Friday, December 31, 2004

Right Direction or OverComplexification?

As the years have gone by, there has been this definite feeling that we are going in the wrong way, in terms of Macromedia Coldfusion.

Both in how we have standards for developing, and in what we focus on.

One article, I remember reading last year on, is that ecommerce retailers have been too focused on what is the correct technology to use. Then they get hooked into bells and whistles rather than what is cost efficient to bring the core results they need.

If I remember correctly, I may be misquoting.

"They should focus more on the ROI goals/results, less on what technology they use to get there."

To me this can also apply to how we use/develop ColdFusion.

I have always seen Coldfusion as a tool to rapidly develop web applications. But over time it has become 20-30 different things to different people. But we have lost that touch with our core, and we still haven't resolved issues of ColdFusion Development...

Here's what we should be focusing on.

1. Built-in tools/addons for Standardized Project Management - As CF Developers we are rarely familiar with Project Mgmt, and it's lack makes us look less professional, then developers of other languages.
2. We need a web-based workflow/source-control system, that is standardized to help remote & local workers, work on different parts of the same project.
3. We need a universally accepted standard of coding style. We may all have variations of this, but we need something as a starting point.
4. We need an automatic documentation system, that forces us to document each component of each application.

We may each have our personal favorites, but that doesn't prevent us from needing a universal style for all cf developers.

As long someone's coding style doesn't make it unreadable to someone else, who may not be a follower of that style.

Face it, if the code isn't readable, how the heck can we improve/fix code?

We must plan for obsolescence and the worse-case scenarios, because it's very sad and frustrating to always be fixing fires.

I think it's time, that we have our top leaders in our community and in Macromedia, create a ColdFusion Standards Group.

The goal would be to create standards in coding, project management, workflow, source control, in a way that can be easily understood and used by everyone.

I am not pitching any methodology over any other. As long as that methodology can release code that is readable by others, who are not of that methodology.

Once we have standards, then we need tools to help us more easily follow that standards.

This is something i have been considering for a while. At first, I thought this could be an add-on to Dreamweaver/Studio. But now I feel for performance and security, it must be a web-based add-on, to IIS/CFMX Web Server.

What do you think?

What is your right direction? And does it make it easier to right code, or for others to read your code, and does it make it faster for you to write code?

Efficiency is one of the keywords for ColdFusion, it's very easy and fast, if you know what you are doing....

Friday, November 19, 2004

Working towards Standards

I keep experiencing these differences of standards, I come to a job, I want to either work with the already existing standards, or to help work to come up with a new one.

It's a political/technical battle all rolled up into one.

There are those workers, who are so busy that don't have time to discuss or work on standards, those who want standards but not sure what kind, those who have a specific standard set they are used to.

I am not a proponent of a specific standard, as long as it's mutually easy to use/remember and helps prevent mistakes being made as a team or individual.

That's goal isn't it, we learn from the mistakes of the past, so like the Boy/Girl Scouts we prepare, prepare and prepare to make sure those mistakes never repeat.

All of this is about the mistake making, learning process.

From Project Mgmt to Coding Standards.

I keep remembering Hal Helm's article on Methodology in CFDJ. On one hand it is important to have a documented and easy to understand standard/methodology, but he is distasteful of there being different ones out there, that aren't a popular/commonly used standard/methodology.

Maybe that's why I get so caught up in the Fusebox/Blackbox/Mach-II arguments. Not that they don't have vital things to be used or learned from, but I have no interest in pushing a certain methodology.

I think we need a fresh new approach to standards and methodologies, that instead of labeling a standard/methodology, but lists each element of good standards/methodologies, and let's everyone pick from the menu on the list, in their own unique priority.

That's the key, we're not all in the same coding environments, so we need to have an open standards approach, of including all ideas, and then each of us can freely pick what things are most important in our own situation.

What kind of key things are important to you, if you had a list of elements for an Open Standards System?

Wednesday, October 13, 2004

Why have a Project Management System?

During my earlier days, I would always wonder, why make a hassle by having a complicated, time-wasting project mgmt system. Just assign me the task, and I'll write notes, and it will get done. Well after 5+ years of experience, I have realized as boring as Project Management, it's really about preventing wasting of time and energy.

As I have learned, as I get more experience, is that being a good coder, is more than just writing good code. It means, mastering all the elements of a web development environment.

But today, we're talking about Project Management, or as I like to call it:

If you were psychic knew exactly what the client wanted, didn't have to waste time by asking any questions, and can just take a few minutes to generate immediate results.

Often, we are experiencing the matador/bull symptom, of seeing an attractive red cape to attack, but not seeing the matador's blade behind it.

This is where those who do not like anal retentive details, of what managing their assignments, are structured so that they are focusing the correct amount of time, on the correct elements that lead to a complete project that meets needs.

Let's analyze that last statement, "A Complete Project that meets needs."

First off, how do we make sure we really know what their needs are?

Secondly, since the person/people who have these needs are usually not the people fulfilling those needs. How can we make sure that we can meet those needs in the way the client wants?

Thirdly, about half-of-us, have to deal with feature-it-us, where in the middle of the project, someone wants to add new features that were NOT a part of the original discussion, and the rest of us are do not have to face it.

Where being Bulls, we just want to kill the matador, or bounce him around at least, we do not want to dance around, or try for Bull gymnastics. Just to keep focused on our goals, and get there....

Where as Matadors, just have needs, that they use the cape and blade to communicate with, not always clearly, and not always decisively.

So we as Bulls and Matadors must come together to work out a way to solve our mutual problems.

The problems are:

1. What the heck is this application, this isn't want I wanted.
2. Hey I definitely said red here, blue here, and lots of green over here.
3. I spent days, weeks of sleepless nights on this project, and you still aren't satisfied?

So how do we solve this problem?

1. You must have a clear and precise idea of what the project is, what the logic flow is like, maybe have some drawings of what the pages look like. What colors will be used, etc. Imagine the coder doing the work, is deaf, blind and dumb, and that you have to use all your senses to completely describe this project.

2. Then as a coder, you have to plan step by step, in detailed fashion, how you plan to execute this project. The client went to all the work to detail out their request, now you must explain how you plan to do the work. I usually ask myself questions, to help fill this out, such as: What kind of graphics need to be created/changed, What database changes need to be done, What form processing or logic flow changes are needed, and how will I know. Use flow charts, diagrams, the more details, the easier it will be to follow your plan.

3. Now client and coder sit down, and review the Project Plan, to make sure the coder's idea of the project are the same as the client's. And then you can go over how long each step will take. What priority this project has over any other work that needs to be done. Then the client either approves or disapproves the Project Plan. If approved, start coding/executing the plan, if disapproved, then must re-start Project Plan to fill any missing spaces, or to clear up any miscommunication.

4. If approved start executing the project plan.

5. Debugging, at this point, as you iron out the bugs, create a functional testing checklist- A list of items that help the average joe, know whether the changes done were accurately done. Coders know code, but end-users know whether they were able to balance their checkbook or not. So help your code testing, by identifying what functionality this new project adds/changes, and how to test the success/failure of that change/addition.

6. Functional Testing, preferrably, the client or a fellow coder, or anyone who has not looked at or worked on the application. You want someone with a clear mindset evaluate the success of your new application. You repeat steps 4,5,6 until all functional testing, says GO!

7. Deploy Mechanism, we each have different development/production environments, but it pays to have a system/person that deploys the code to the right servers at the right time. Remember to turn Trusted Cache off, to make the code active.

8. Test Again on Production, using the Functional Testing Checklist, to make sure that everything works as smoothly on production, as it did on development.

Then document all of this, so you can organize, prioritize, and learn from each project. I mean, wouldn't it be nice, if you were working on the same project as you did 6 months ago, to have that documentation available to help remind you of what work was last done on that application?

Remember this isn't about being Anal Retentive, it's about avoiding communication mistakes.

What kind of miscommunication mistakes have you experienced? Or do you hate Project Management? Please comment.

Tuesday, October 12, 2004

SQL Turbocharging

To really make your pages fly, depends on a lot of different things, such as indexing, table relationships, and good sql statements.

Normally we have two starting points, whether you are creating a table from scratch or whether you are working on existing code.

Let's say you are working with an existing database, and that you want the queries for this specific application to work fast.

First thing, you have to consider is the freshness of data needs. How fresh does the data have to be? This information, can help shape the coldfusion side of your queries, whether you use caching or not.

Second thing, is the control of the datetype enforcement of incoming variables to be applied to your query. For security it always pays to enforce the specific data type, either by using cfqueryparam, or other means.

Third thing, what is the logic of the data you are grabbing, are you looping from 1 query to another query? This is ALWAYS a bad practice for performance.

One of the hardest things I learned, is that you should combine the two queries, instead of doing the looping. But it will take patience and practice to formulate the query so as to return the same data.

Remember each ODBC/JDBC connection to the database adds a hit of time,threads and performance. Less queries is always better.

Now you have this query, now we want to make sure the tables are prepared to deliver the best results.

I always start by making sure tables have a primary key that is also clustered.

You can run the below script to get a list of tables, with primary keys, and whether or not they are clustered or not.

select i.table_name, cu.column_name, tc.constraint_name, case objectproperty(object_id(i.table_name), 'tablehasclustindex') when 0 then 'no' when 1 then 'yes' end as [has clustered index]
from information_schema.table_constraints tc
inner join information_schema.constraint_column_usage as cu on tc.constraint_name = cu.constraint_name
full outer join information_schema.tables i on tc.table_name = i.table_name
where objectproperty(object_id(i.table_name), 'isusertable') = 1
order by i.table_name asc, [has clustered index]

It's a great script for documenting the indexes of all tables in your database.

Now that the tables have primary keys that are clustered, then you look at additional indexes, to make sure the joins are indexed.

For example if you are always WHEREING against a username, but it's not the primary key, then you'd want to make sure it is indexed. Now only the primary key should be clustered, it also does pay to use a easy to understand name of each index.

Once all tables have indexes, that are using in your joining queries, then it is time to start creating relationships.

Creating Relationships in SQL Server can be a painful process, but also a great tool to help improve performance.

You start by specifying a relationship that has no referrential integrity. Referrential Integrity means whether it is strict in making sure that when table 1 is updated, table 2 has to be updated right after. You should consult with a dba, if you want to make sure you are making the right decision for the type of relationship.

But I always start out with a non-enforcing relationship, because that takes less effort in maintenance, and can cause less problems.

Now you don't want to create 5 billion relationships, but you want to look through all the main queries of your applications, for what joins that are commonly used.

So now your tables have primary keys, that are clustered, and have indexes for joined queries, and have relationships with joined tables.

The Database is now setup to deliver speedier data.

Now it is up to how you query that database.

SQL Server is different from Oracle, in that your from statement, the order of tables in it, can help/hinder performance.

In SQL Server 2000, the big thing is your where clauses and your joins.

This is where we take out the SQL Query Analyzer, turn on Show Explanation Plan, grab a sample query, and looking step by step.

An explanation plan is a powerful and graphical way of looking of how SQL Server plans to deliver the data you requested. I start out by looking for anything that takes more than 20% in resources, I look for table scans, or any sign, that a query isn't performing the way I want to.

Let's say you have two tables, Table A has 100,000 records, and Table B has 200 records. Now obviously if you have to do a join, you don't want to scan thru all 100,000 records from Table A, just to get 2 matching records from Table B?

You have to play with the joins, trying subqueries, or in statements, or exists, to create the query that delivers the right data, without unnecessary data scanning.

It pays to think about what kind of data and how much data you need first, then create the query to grab that data...

This is one thing that takes experience, and even I, still have a lot more to learn.

I hope this has helped by putting together all the main ingredients to improving performance.

After all, what's the point of creating a nice web application, if it can't load for the user, or it keeps crashing your site?


Thursday, October 07, 2004

Managing Sql Server Database

Have been learning from Ibrahim Hafidh of Imagetrend, all about profiling, writing better sql. The goal being to both better monitor sql server, and to help identify areas of improvements.

I have been testing out this sweet trial version of Embarcadero Performance Analyst, which for $5000+ is a very sweet, web-based sql performance analyzer.

However, since I wanted to come up with a more practical aka cheaper solution.

Ibrahim and my fellow workers, were learning all about working with SQL Profiler, especially in watching the duration profile, in actually being able to see sql statements that were being run thru sql server. Which is a very powerful tool.

It makes me feel stupid sometimes, seeing all these built-in tools, and not realizing how easy it is to use.

Well starting with this morning, I am sql profiling, reviewing queries to be worked on, if only i could figure out how to identify what coldfusion page, that query came from. That would really help.

Also I have been playing with Information Schema today, to help identify what tables do/don't have primary key's and of those, which are clustered indexes.

Face it, if you have a lot of tables to review, it helps to know and prioritize which ones need the most work.

I modified the above script, actually combining it with another one. Perhaps this already existed, but it's still a nice tool, at least for my own sake.

select i.table_name, cu.column_name, tc.constraint_name, case
objectproperty(object_id(i.table_name), 'tablehasclustindex') when 0 then
'no' when 1 then 'yes' end as [has clustered index]
from information_schema.table_constraints tc
inner join information_schema.constraint_column_usage as cu on
tc.constraint_name = cu.constraint_name
full outer join information_schema.tables i on tc.table_name =
where objectproperty(object_id(i.table_name), 'isusertable') = 1
order by i.table_name asc, [has clustered index]

Run this in SQL Query Analyzer, and see how surprised you are/aren't on how well you created/modeled/optimized your tables.

I have been working with SQL Server 2000 for about 2 years, and the more I learn, the more I realize I have yet to learn.

Also I would like to strongly recommend Ken England's book on Microsoft SQL Server 2000 Performance, Optimization and Tuning Handbook. He's the really first guy, to present high level information, from a first person perspective, that is easy to read and understand. He's a great teacher...

For most coders, you could care less what the issue is with the table design/structure. What matters for you, is delivering the right data, for the best performance.

Here's where Ken England's book comes in handy. He explains, that when you look at Explanation Plans, what makes the explanation plan pick the correct indexes, for joins, scans, depends on the selectivity of the index.

Part of this, as far as I can figure requires that:

1. You have a primary key that is clustered
2. Any fields that you use to do joins, where's need to be indexed.
3. It helps to define relationships between tables, that you join against frequentally.

Here is his formula for determining the selectivity of an index, to be used for the Query Optimizer in your sql statements.
selectivity = (the nmber of rows returned / the count of rows in the table) * 100

if 5,000 of the rows in our 100,000-row table were returned, the selectivity of our query would be:

selectivity = (5000 / 100000) * 100 = 5%

if 90,000 of the rows in our 100,000-row table were returned, the selectivity of our query would be:

selectivity = (90000 / 100000) * 100 = 90%

the more selective a query the fewer rows returened and the more likely that an index will be chosen by the query optimizer.


So basically our goal as coders, is to help make it easy for sql query optimizer to deliver our data with the best performance.

I myself, still have to play around with this more to get my head, which is going to take time...

What kind of things have you learned to optimize your queries performance?


Wednesday, October 06, 2004

Is XML the Answer?

What an interesting idea, something I've been waiting to hear about for a long time. Xml has some definite uses, but it's not the solution for everything.

I have had some thoughts about using SQL Server to generate xml-formatted data, and then use IE's data island to display recordsets that are just too huge to manage in cfquery.

But I have subconsciously realized, that would not be a real solution, that if you have a page bringing too much fresh data, that you need alternatives.

This is all apart of my philosophy of using what works for me. Or in shorter form, if it works, use it.

But this also goes towards the idea that every programming paradigm, is the solution for all problems.

I like the fact that he's willing to dump a programming paradigm, if it doesn't get the job done...

I am not saying to drop oo, different methodologies, only to be replaced by another trend/fad of what to do...

Stick with what you know, but don't be afraid to realize that no solution solves every problems....

Common Sense, isn't always that common...

Have a good day everyone...

Tuesday, October 05, 2004

Balanced SQL

As I been going thru pages upon pages of cf code with sql statements, I realize how important it is to balance the needs of the end-user against scalability.

As much as we create these incredible applications, sometimes we go over the edge in demanding too much from sql, code to deliver what should not be delivered.

Have you had that experience, of reading someone else's code, and thinking, what the heck is this supposed to do?

Or why did they do it that way?

Or the, I want tons of fresh data, with no issues for performance?

There's a give and take, of what we can deliver...

I mean if it's an application, with only 1 end-user, that's quite a different scalability challenge, then discover tons of users using this archaic application.

It's both fascinating and frustating, how much we have to learn, to be really good coders.

We think that being a good coder, is knowing the syntax, using the latest methodology, reading the latest books.

They are but a tip of the iceberg....

So my message of today, is really plan for how many users will use your application, and plan for the far future.

It's a simple phrase.

Think Longterm.

Wednesday, September 29, 2004

Rush, Rush, Code, Code

I've been reading Joel Spolsky's book Joel On Software based on his blog, It's quite a compelling book, because it has a lot more details to help get us to a higher level of productivity, efficiency and organization....

In most of my recent jobs, I've been in the position of having to clean up code, SQL for both performance and easy to understand, read and use.

This book is great, because I can learn what else I need to know.

After having read Rand's post on Incrementalists & Completionists, this helped me understand, why I am so driven to reach such a high level organization/productivity.

Each of us is shaped by the tasks, paths, and choices we make, and I have no less been shaped by mine.

The point of this, is that unless we get a better understanding, we'll end up being burned out and frustrated.

Because in my experiences, I was so driven to get to this high plateau that and if we didn't all work together to get there, in a short period of time, I'd get frustrated and burned out.

So I have realized I have to pick and choose, and incrementally over time, improve the quality of the site, code.

Joel On Software is a really great written book and blog.

The book covers topics from Writing Specs, Bug Tracking, Managing Developers, and as they say, much much more.

I think I'd love to learn about other people's experiences, about cleaning up after other people's code.

Then there's my additional experience of being the one to introduce Project Management, Source Control, Scalable Code to each company.

As Joel Spolsky says, for our industry there is no developer/software management/project management bible, that helps companies start out on top, in stead of having to spend too much time cleaning up code.

For me the goals, most in my mind are:

1. How can I make sure that the site never goes down anymore.
2. That bugs are recorded and assigned to be fixed, instead of always creating new features/fixes that may introduce it's own set of bugs.
3. How can I help the creation of a project mgmt process that will be easy to learn and manage, and yet still make sure we make no communication/project mistakes.
4. How can I encourage my fellow developers to stick to using our project mgmt intranet etc...

Now that I am working on a coding team, I see how important a good methodology is. It's half fadish psychology and half setting standards that will actually be followed.

I mean no offense to fusebox, mach-2 users, I do approve of anyone that uses a standards and works to release quality products.

It does help not having to re-invent the wheel all the time. Or making sure that I bring all the stuff from the previous job to the next one, to help them learn project management, bug tracking, best practices...

I think what'd be nice if we as an industry can create a best practice repository, that can be reviewed, approved, and then made available...

That's it for now...

Sunday, September 19, 2004

Index Tuning Wizard - Man's Best Friend

Index Tuning Wizard is a handy tool for any ole Mr. Clean Project. Basically it is a great tool to improve the performance of your databases.

First off, there is only 1 way to access it via Query Analyzer, I believe the option is not available easily, otherwise.

My method of running it, is to pick a slow running coldfusion page, and look for queries taking more than 100 ms.

Then you plop it into query analyzer, and make sure you enable Show Execution Plan, for additional information.

Then you click on Query, then Index Tuning Wizard. This is where the options are simple but the consequences can be great.

First of all, make sure you make up a backup, and that the site is down, not creating any more database interactions while you make these changes.

Also if you are working on big tables, the actual process of automatically selecting and adding indexes can take much longer, so be prepared to give it the time and resources it needs.

Then I start with the medium level of index seeking, and i keep checked 'Keep Current Indexes', so that it doesn't drop anything.

Then it will take it's time to come up with indexes, and just be patient, and let it happen, and then you have some additional choices.

Such as whether you have a dev sql server, you may want to save the scripts it creates to take action to then be run on dev, after your production database is up to snuff.

All of this is what I just did for my current job, this Saturday. To help improve performance.

All Of what I try to teach is to help any of us, have a stable and scalible platform of database,coding, and standards.

Maybe this is boring, because it isn't the hip or trendy xml, java, but how relevent is it to pursue that, if your platform for your sites, isn't great.

Cover your bases, and make sure you are optimal before you explore anything non-related to coldfusion's main purpose.

Tuesday, September 07, 2004

Error Proofing Your Code

The best way to start error-proofing your code is by following these steps:

1. Identify every incoming piece of data, and document it and it's datatype
2. Identify any possible conflict issue between different scopes of data either incoming or created by the application.
3. Then document each section of your app, with the above information, so you can tackle your app 1 step at a time.

Now that you have the documentation, now comes the setup.

First of all, in my personal style and attempt to prevent scope creep, I came up with my own variable naming scope.

For any application, at the top of the page i create a list of variables that are going to be used within the app, as well as incoming data.

!-- global variables --
cfparam name="local_firstname" default=""
cfparam name="local_lastname" default=""
cfparam name="local_price" default="0.00" scale="2"

Then i move all cfparams through out the application, to the top of the app, for easier sorting and eliminating duplicates.

Now then comes the population from wherever your datasource is: url, form, query, cgi, file, client, cookie or local.

Then you put a cfoutput surrounding the whole form, then you have created an error proof form.

Let me explain.

By having a default values for every form element, you make sure that the form can be used either for an edit content or add content piece. Which really speeds up the process of creating forms.

Now all of this is part of my philosophy of preventing errors from occuring in the first place, rather than trying to catch for them.

I'll walk thru a section of an app, and try to identify places it can fail, and do what i can to prevent it.

More to come...

Tuesday, August 31, 2004

Creating a solid Development Environment

Well here I am, I finally got a new job, in fact today is my first day!

I am pretty excited to be at where I am.

So I get to be part of a company that wants to grow, and this is their first time having more than 1 developer in-house.

So we have to basically take what they have and help take us to the next step in terms of the old-standards:

1. Project Management System - To avoid all the old waste of time, communication errors, and to basically make sure we turn out a product that won't bite us in the butt 2-3 weeks after completion.
2. Source Control System - It's surprising how many people still don't know what it is. But it's still needed, and there is plenty of growth in this niche need to have new features and functionality. But my current favorite is QVCS by, nice product for a great price.
3. Project Management Intranet - I have created 2-3 of these, and basically you can either do it on paper, or on excel or create an intranet. It comes from the need to document, prioritize and improve communication
4. Coding Standards Documented - There has to be a common grounds, however different the styles are, but once it's determined, you need to document it, so that the next new person, has a much easier starting point.
5. Site Bible - One of my ideas, but have never seen well implemented, including by me. Basically imagine if everyone in your development and it died, what information would you need to hand to a brand new team? It's the binder that you put all your documentation from servers, to applications.

What else does a good development enviroment need?

What new or old things do you have that I missed, or could use?


Tuesday, August 24, 2004

The importance of meticulousness

It sounds like a boring topic, but this is what it takes to be real good. As I get older and older, I realize the difference between those who are the best, and those who are not. It's not a matter of talent, but a matter of meticulousness.

Or better put, those who dedicate themselves to being the best, to mastering their craft, will do better in the long run against people who are more talented, but less driven.

That inner drive is something that has taken me a long time to learn, and is part of all our maturation process.

Today has been a good day, I just had job interview at a nice company, and I was again reminded how important craftsmanship is over talent.

It may sound contra-trendy, but it makes sense.

Bill Gates, Donald Trump, Steve Jobs, Red McCombs, may have some talent or genius, but their success is more do to their drive and persistence.

In each of my jobs and careers, and I have had many of both, especially at my old age of 41. People my age, have had to deal with the change in job culture, instead of people having jobs lasting 20-40 years, staying at the same company, retiring with great pensions, we have to deal with changing jobs frequentally to keep moving up the ladder.

I don' t know if that's because of the transformation from an industrial age to an information age.

So times and people change, sometimes for the better sometimes not. The key in my mind is what can we learn from it, how can we improve where we are.

I think one thing I do, that I'd like to pass on, is that I use mental imagery of craftsmanship and persistence, when I am in doubt.

I think about ancient japanese swordsmiths, perfecting their sword over however many months and years it took to create a beautiful and efficient weapon. They had a lot of pride in their work, and since their name, reputation and life depended on it. They would live or die by their work.

I also think of at times, of my father, as a pediatric dentist for 30-40 years, dealing with whiny kids, who demanded perfect care on their teeth, to create a better and healthier future. Imagine the patience and dedication it takes to be the best, because only the best is acceptable to him...

What kind of images motivate you to stay or move towards the best?

What motivates you, who are your idols of craftsmanship?

What books do you read to remember your best work?

Thank you.

Wednesday, August 18, 2004

OT:Barry Ween Custom Desktop

Barry Ween Custom Desktop

As I go slightly mad, with the job hunting, I have created this little homage to Barry Ween, a comic created by artist/writer Judd Winick.

It's just something for fun.

Have fun!

Saturday, August 14, 2004

OT: 2004 Hall of Fame Photos

Carl Eller's Acceptance Speech
Carl Eller's Acceptance Speech

Well finally after lots of computer problems, I managed to organize all my photos, and post some of my best on the photoblog.

Please check them out and let me know what you think.

It was a very touching experience, to see someone finally rewarded for decades of hard work.

A good work ethic does pay off, but sometimes it takes patience...

If you like I have a screen saver all the photos and some nice football music, I can email to anyone who asks...

Wednesday, August 11, 2004

Coding Horror Stories

If you want to hear and share some of your coding horror stories, now is that time. Both for humor and the sad realization of what it really takes to develop quality applications.

In a way this is very funny for me, because my technical personality sort of emulates my personality I developed in working in Fast Food. I was the Prep God, make sure everything is setup correctly, and perfectly, for the afternoon/evening shifts.

And here I am, Mr. Clean, always having to clean up after other people's work.

It's because of spending most of my coding career cleaning up, that I have gradually become a stickler for standards.

I remember one of my job hunting jigs, at this downtown saint paul firm, seemed a bit raw or at the edge, but there are all types of people/personalities/companies in this niche.

Anywho, then they started to show me some of their code.

  • No Indenting
  • No Comments
  • No Logical Structure
  • No clue what the code did, or what steps it took to get there

At that time, I was still somewhat of a beginner, and I wasn't sure how to address their lack of quality. After all, I wanted a job, but I didn't want to work for idiots.

One of my jobs was a nightmare in terms of communication. At the time I was at a web shoppe, who had a legal firm as a client.

Legal firms are very tough, this one was particularly indecisive.

They wanted us to create a multiple-approval layer of content approval. It sounds professional, but in reality, it made them sound very indecisive, and just added unnecessary complexity to the content management application.

It also made them show that they had serious trust issues with their own personnel. But that's legal firms.

Another time, I was working on a severely modified version of AbleCommerce, which is sounds like a really good starting point for an Ecommerce Application. But most pre-built applications, usually end up to be badly written, badly documented, badly coded applications.

They did their job, but made it very difficult to fix or improve upon them.

Especially since the company that modified it, was using coding standards picked up from ColdFusion 2 coding.

So you can imagine all the extra pound signs, that still ring in as a nightmare in my mind.

What kind of horror stories do you have? What kind of coding mistakes piss you off, or just plain drive you insane, please share your stories...

Sunday, August 08, 2004

OT: Carl Eller is enshrined in the Hall of Fame - Pro Football Hall of Fame

Carl Eller is enshrined in the Hall of Fame - Pro Football Hall of Fame

Well this weekend, I was honorably invited by Carl Eller to witness his enshrinement to the Pro Football Hall of Fame.

From 1998-2001, we ran a site called Carl Eller's Viking News, which didn't turn out that profitable, but was an amazing experience.

From the first 1998 Mankato Training Camp, where I saw #18 Randy Moss, as he was interviewed by Carl Eller, and to meeting Red McCombs for the first time.

To helping John Randle, Cris Carter in jest help sign their autographs...

My father and I arrived, in Canton Ohio, Friday afternoon. But the amazing experience of the Hall of Fame didn't start till Saturday morning.

We woke early at 6:15 am to go to the Enshrine Parade, and we were escorted by the police motorcycles and cars with blazing sirens on our way to the Parade. The parade lasted over 3 hours in freezing weather, 15 different high school marching bands, all kinds of floats, former Hall of Famers, and all kinds of local celebrities.

Quite an experience.

Then Saturday Night was the mind-blowing Enshrinee Dinner, which started off with a cocktail party outside, you can see all the celebrities/fans/sponsors schmoozing.

At the dinner, we shared our table with former Viking WR Frank Gilliam, who retired in 1977. While our room was hosted by former hall of famer, Mel Renfro.

After which we went into the huge stadium, as we watch different speakers/presenters.

Paul Tagliabue, then the TV/Radio Awards, a lot of really funny jokes, that can not be said in public.

Then the former hall of famers, helped our four Enshrinees with their new Golden Hall of Fame Jackets...

It was real dark inside, and there was like the lighted area, like a fashion walkway/catwalk.

Then on Sunday, another Police Escorted Ride, in our buses to the Hall of Fame. There were so many people there, that they had 3 huge banquets, Gold, Silver and Bronze. With 3 huge tents of food and drinks. I am not even sure I even recognized some of what I had.

Then we went to Fawcette Stadium, where were bloodly lucky to be on the ground floor way near the front.

I saw quite a few former Vikings, such as former Hall of Famer, Vikings Coach Bud Grant, former player Chuck Foreman RB, former player Bob Lurtsema and a bit more.

But the stadium was really dominated by Bronco's fans.

After hearing Regis Eller, introduce his father Carl Eller, then hear his amazing and really emotional speech. Then we heard part of Bob Browne Jr's introduction of his father, the heat had gotten to me, so we had to leave.

I am glad to be back to Minnesota, and thanking my lucky stars for such an honor.

Thursday, August 05, 2004

Site Search Bookmarklet

In the last couple of days, I've been playing with Bookmarklets, as ways to enhance my browsing experience.

One of the cool things, I have found is ways to search for things on your current page, and present that in IE's sidebar.

So that instead of replacing the current page, you can use put content in the sidebar.

Site Search Bookmarklet - Just drag to your links toolbar.

The way this works, you select some text on a webpage, until it is highlighted, then click on this bookmarklet on your links toolbar, and google will search that site for that highlighted text.
Neat, huh?

My next project is to detect an rss feed on a webpage, and display the transformed version of that rss/xml feed in the ie side panel...

Tell me what you think.
Have fun!

Applying CF Best Practices - joe rinehart on CF and more

This is my attempt to show how to apply Joe Rinehart's Best Practices list...

1. Never trust user input

I always have my own version of a local variable name, that can not be overwritten without my consent. This is both good for security and data type validation, but makes it easy for form displays.

Because when I create my forms to edit/add content, I always create the edit form, with cfoutput surrounding my local variables.

But when you use cfparam to pre-define your local variables, then it really makes it easy to use the same code from editing content to adding content.

2. cfqueryparam or caching and data type validation

The benefits vary somewhat based on what database you are using. In Oracle, cfqueryparam also allows you to take advantage of Bind Variables, to improve performance.

Using it with SQL Server just allows you to use it's datatype validation methods.

And you can use cfqueryparam in where, select, anywhere where there is dynmamic variables affecting the sql code.

Remember cfqueryparam isn't needed if you the dynamic part isn't being controlled or sent by a user. Although it is a good practice.

You have to evaluate which caching method benefits you most, based on whether it's frequentally changing data, or frequentally needed data.

3. Coding for Portability

This is in general a good idea, to use variables, to hold data that changes based on what server you are on, and store them in your application scope, in the application.cfm file.

It does get to be a pain to have change hundreds of files, if your datasource or folder structure changes.

4. Code Formatting

Please use the same syntax of commenting, like some older coders put their comments to the right of the commands, and some above the commands. Stick to one style.

This also includes indenting, which is my favorite easy way of following the logic flow of an application...

5. Use existing public code

Well there are a lot of resources out the of udf's, custom tags, all kinds of ways to do a specific task.

Try looking around, and seeing if a free piece of code can do what you want, before you try creating from scratch.

6. Variable Scoping

I've written enough to explain the importance of variable scoping. It really comes down to being careful to make sure the variables hold the data you want them to.

I always use cfparam to create a local variable, which then i can logically pass url, query, form, file, cookie, client data.

The key is control what data gets into what variable.

8. Performance Matters

CFMX has really advanced it's debugging information, it shows allt he includes that your application calls, and makes it easy to identify bottlenecks.

If an include takes more than 250 mx, then it bolds/reddens that debug info, to show you to be aware of this includes performance issues.

Take care of your queries, logic flow, keep an eye how much processing power it takes to do your job.


In conclusion.

Best Practices are always evolving, because all of us want to learn from our and other's mistakes so that we can do a professional job.

Because none of us want to go back to a project we thought we completed, only to find out it crashed the site, or didn't work, or had x number of errors.

The more thorough and precise in our coding, the better we will get in the long haul..

What best practices do you use, that are not in the list? What practices do you think make sense or no sense?

Please comment and let us know!


Wednesday, August 04, 2004

Stored procedures applied per situation

Stored procedures are bad, m'kay?

It really depends on your situation, in terms of stored procedures vs no stored procedures.

1. Is this for an intranet, extranet or internet site?
2. How many different records are you planning to pump thru the procedure?
3. How many different logic flows, looping would you need?

I do agree that stored procedures aren't the solution for everything, and perhaps there was this drive towards stored procedures as a solution for bad coldfusion/sql programming, to help produce great results in reasonable performance.

It just depends on your needs.

If it's a 1-time query, then you don't need to save it as a stored procedure, unless for example, it's for a monthly/yearly report, in that case you would save it, so that you won't have to redo it, the next time you need it.

The real issue is whether you pay attention to how your queries affect both the load on coldfusion and sql server, but how it affects the load time of your visitors/users.

Sidenote: There seems to be two major opinionated camps, those that think sticking to best practices and reasonable coding standards is a great idea, and those that think it's not that important or even a waste of time.

I know that I and a few other bloggers, really recycle those standards and best practices and standards. And I'll acknowledge discussing them isn't hip or cool, but it's really out of a desire to encourage everyone to perfect their craft.

Because when you release crappy code, that isn't documented, or crashes after it's taken from Access to SQL Server, or any number of the experiences I and thousands of coders have experienced in cleaning up after other people's work.

It's just a pain in the butt, it makes you look bad, and it makes us waste time trying to do the job you should have done...

Maybe we should start sharing some of our horror stories... - joe rinehart on CF and more - joe rinehart on CF and more

A very nice re-introduction of Coldfusion's Forgotten Best Practices.

I know it's one of my favorite topics, to hone into our cultural mindset the importance, of sticking to the best coding/programming practices.

So that we can demonstrate that we are great coders, and that coldfusion is a great platform.

How about this, who are your favorite coders, or favorite applications you like to code or, some website program you did, that you are proud of?

Let's share our glory moments while we can!

Tuesday, August 03, 2004 - Blog - Blog

This is a great starting point, but I feel sometimes, that it's not really a logical argument that is being presented back and forth.

After all, once someone is sold on a particular technology, it's like a religion trying to discuss alternatives.

Don't take me wrong, this is a great starting point...

But what I'd really like to see is a comparrison of top sites in the same niche industry comparing head to head, in terms of sales, how long the servers stay up, how many times they had to reboot, anything else we can use as a fact, to prove that coldfusion is the more profitable and more stable solution.

I believe that this is part of the a new trend of trying to prove things by example, rather than just explaining why coldfusion beats java, asp,, php etc....

I'd love to hear examples of top sites that compete, like a cf vs php or cf vs etc...Bring up your favorite/top cf sites...

Let's show coldfusion competes in the real world.

Query Bottlenecks

One of the biggest barriers to scalability and long-term performance is how we logically grab the flow of data from the database.

So I am creating a list of things to not do, and to do.

1. Do not loop thru cfquery, because each cfquery creates a seperate jdbc/odbc connection which takes additinal memory/processing power etc. So look at the different queries and see if you can combine them.

But it depends on the relationships, if for example you have two queries and the second is looped over by the first one, perhaps you can do a join.

Remember the power of group by in the cfoutput statements....

You can use this technique, based on the join you use, to combine the queries and still get the same results.

Remember the issue is to reduce the number of cfqueries, as much as possible.

2. Do not ever, ever do select *. It's always best as a standard and for performance, to specify in the correct order, the exact field you need for your query.

For example if you have a table of 40 fields, and you run a query that needs only five, think of how much wasted memory/cpu, you are using improper coding.

If coldfusion is going to be the best language, then we have to always code for longterm, scalable performance, even if it takes us longer to type in.

So once you have performance nailed down, then it is a matter of improving upon it.

1. Such as moving complicated, highly used joined queries into views or stored procedures.
2. Review the performance of the page, by debugging, look at the cfincludes and cfqueries for those that take too long.
3. Learn all about caching, when and where.

Cachedwithin, Cachedafter, or cfqueryparam

CFQUERYPARAM is a very powerful tool, both to help validate incoming url/form data to a specific data type. Which is great for preventing cross-scripting, and errors.

For oracle databases, it actually improves performance, because of something called bind-variables. Basically if you have 1 query running dozens of times, but just a different value in the where clause, then bind-variables, allow the database to treat the queries as the same query, which caches it and improves performance.

In the end, query bottlenecks, are caused by poor planning, and poor coding. These can be overcome, and they should be overcome.

Coldfusion has so much potential to be the most scalible web development language/server, if we code that way.


Monday, August 02, 2004

Cracking into ColdFusion - SitePoint ColdFusion Blog

Cracking into ColdFusion - SitePoint ColdFusion Blog

I find this extremely fastinating, because who knows what special needs we have, that we need to roll our own versions of cfml tags?

Like for me, example, I'd love to roll a version of cfdump that is printable, besides all that beautiful dhtml.

I find it can take coding to a different path, to add things we'd all wish for.

I mean once you have a stable platform, that is scalible and delivers on it's goods. Then it's a matter of customizing it towards your specific needs.

Friday, July 30, 2004

Blogging and Ecommerce....

One of the things I have been researching, while on the job hunt, is the profitability trends and technologies for blogging.

It seems there are room for growth in profitability, such as the major recommendation of putting Google AdSense on your blog. But that seemed to be lackluster, not quite enough.

For starters, there are major technical limitations to a blog, especially if you have one of the free ones, no coldfusion, asp, php, jsp or .net support, just straight html,css and javascript.

But it seems empty that there are no tools or means to integrate a shopping cart, or to browse the categories of recommended products.

There simply is no data connection, or way to integrate some higher technology, unless you try to do javascript converted content, but that doesn't always reach all your customers.

I for example, use blogger/blogspot because it is free. But that is also gives me a set of possible Blogger Tags to add functionality to my blog.

Ecommerce sounds so many times, like a magic wand of big cash flowing in, but it isn't. It's a process of merely trading products/services for money that could be used to buy other products/services.

We could try to implement XML Databinding to allow browsing of categories of products, but that is only supported in IE, and has issues with the data being from a different domain as the site being displayed at.

In addition, bloggers can't create seperate blank pages, that you can do programming on. You just write a blog, edit the template, and that's about it.

You could deliver content in javascript via coldfusion, put together a database of products, and then deliver the content as needed to help visitors browse? But how do you get them to browse, there is no seperate browsing page. Do you do a popup? No, people find them annoying.

So there you have this driving need to make more money with your blog, but the technology and usability limits are very tough to break...

Do we need a new standard of blogging? To add more functionality?

After all if blogs make money, maybe as more than just an affiliate, think of how that will grow the blogging industry, if people can sell securely their products.

Or even if you are just an amazon affiliate, if you can organize all your recommended books/products whatever into a browseable categories?

I think it's time for some innovation...

What do you think?

Thursday, July 29, 2004

Must have site search features

After reading the latest news from InternetRetailer, about how Tower Records increased conversion rates by changing site search, from Endecca to Mercado Software. I had to see it for myself.

I had to see if the relevence was good, the interface was clean, to see if it could find what I was looking for.

Now I may not be a big music hound, but I did remember April Lavigne.

Now it occured to me, that most recent search software should have spelling correction, right? Wrong!

I typed in Lavine, thinking, okay this new search would be able to recognize my miss-spelling and find April Lavigne, well I was wrong.

This led me to think, not so much to blame a particulare site search software, but to identify what kind of features does a site search really need?

1. It needs to know how to interpret search query requests to bring the relevent results.
2. Then it needs to provide an interface to search/browse/sort thru those results

Sounds simple right?

Well let's for example show you what it took Celebros, a very nice site search company from Israel, to create spelling corrections.

Basically they had to go thru every piece of data, every title, category, to hopefully provide clue that if a customer typed x they really meant y.

That took days and weeks to come up with spelling corrections, related searches, categorization etc...

But it still failed thru no fault of their own.

It's a failure of an assumption, that the customers really know the product data, or that they can easily figure out what to say to get what results they want....

Before you can create an efficient and profitable site search, you have to understand the customer, not the client, but the people who are going to query your site search....

Most customers tend to divide into three or more searching categories:

1. Looking for something specific
2. Just browsing, surfing
3. Researching to Buy

Each has different needs, and you have to create a solution that meets all their needs.

Then you need to realize you can create your own cheaper solution using whatever database you have. Whether it be SQL Server 2000's Full-Text Searching, Verity, Oracle SQL Statements.

The key is how to create good experiences when you fail to deliver relevent results, I mean, they may be relevent to the people who work at your company, but not to the customer.

So then there is two approachs, do you spend lots of time or money trying to create a data set, that will fix any search queries?

Or do you try your best, then watch the Conversion Rates, and hope things grow....

At first any new search solution for your site, will show improvements, especially if it's an experience change...

I mean, people can adapt to the worst site search, and even being offered two choices will choose the old one, just because they are more used to it.

So my theory is that you need an adaptive/learning site search, that immediately gives a good experience to the customer, as much as possible, such as offerng a discount/promotion for failing to get good search results or even a live chat link to help not lose this customer, and make sure they can find what they are looking for....

Then you need to get feedback from customer and from your site search programming to identify what types of search failures are occuring, and then to inform your site search/merchandising team to tweak the site search data, to provide the correct results, for the search query the customer used...

Then over time, you'll have a more liquid and more human site search...

It's either that or calling the boy scout to go visit the customer and bring them across the street to your store?


Tuesday, July 27, 2004

Learning Site Search

One of my covert projects I was working on, was a Learning Site Search. The concept being that to help the search engine learn as it goes, rather than trying to force it to have all the answers at first.

Let's face it, most site searches are not going to be perfect. They can get somewhat close, but only after months of working on the data, and still it will have problems.

1. To create a good site search, you have to know what the customers want, and how they'll phrase their searches
2. You have to know how to identify when the customer is not getting the results they want.

I wanted this site search, to have some unusual features, that helped it be more usable and better marketing.

First the site search would be based on SQL Server 2000's Full-Text Indexing, which I was very comfortable with.

Then there would be the toolbars/configuration of that search to help a customer modify the search, such as going from page to page, highlighting keywords, etc.

Now my first objective, was to relevancy, I had decided, that after looking at the research on site search usability, that most customers' really do not look past the 2nd page. I mean how many of us have that much time to waste, looking for something that's hard to find or isn't organized well?

Then I decided to have 20 results per page, so that no matter what search result, the focus of the merchandising team was on helping to make those two pages the most relevant of results.

After having worked with other search firms, such as Celebros, for which they have some very nice features, such as spell replacement. I wanted to add that feature to this search, but considering that with a product line of 38400, it would be pretty hard to pre-determine all spelling corrections, synonyms, etc.

So this is the backbone of the Learning Site Search, the need for two forms of feedback. One set of feedback has to be from the customers, to help identify keywords that don't get proper results. Then the second has to be from internal programming identifying when we get too many results, or irrelevant results.

So we now have a site search that after being notified either by customer or internal programming when a keyword is returning bad results. Then there has to be a way to configure or customize the results.

I created several different tables.

Synonyms - What if this keyword in a different way of saying it, gets the better results, so show the original word, but use this substitute to get the better results.
Spelling - To show the original word, and use the correction to substitute it.
Rank Drop Off - This is a new feature, I remember reading where after rank % between each record gets too high, it just becomes too irrelevant. So that for example for x keyword, any result below 30% was of no value,then we could stop showing results 30% or below. It makes it easier to have us show only the most RELEVANT results.
Rank Multipliers - I am searching thru 2 different tables in a total of 3 different ways, so to provide another way to customize results, we can dynamically change the priority of which table, which method via these dynamic rank multipliers.
Keyword Status - To help identify which keywords have which problems, including a space for comments by Merchandising Team.

Now Full-Text Indexing has a built-in rank counting field, to determine the density of a keyword in certain fields, and in addition I was searching 3 different methods. What I needed next was to convert to a percentage mode of ranking.

So after complex querying, I grabbed the top ranking result item, and then divided, following results by that, giving me a 100-0% results in percentage. Which then prepares for my rank drop off needs.

Now back to feedback methods, we did not want to ask for comments for customers, but just to click on 1-4 links to help us identify if this result was incorrect in any way...

Now all that's setup, then it's just a matter of testing it, against your previous search.

Now this will grow in data, as it gets customized. But remember, this will learn, and grow, and require less effort over time.

Good luck...

P.S. As you may know, I am currently looking for work, I would love to be able to help you with your site search, to be more profitable and to be more relevant. Being a coder does not prevent you from identifying with your customers.

Monday, July 19, 2004

Time to move on...

Well time changes, and we all change, so I just lost my job, for all those firm cfpurists haters, can go hip-hip hurray...
Well here we are start of a brand new week, life brings you struggle sometimes....
I am not saying I am mistake free, that I am perfect, nor do I think that there exists such people.
We all try, I wasn't really happy, they weren't really happy, so ce sera, time to move on....
I already have ideas for some business products I want to create...
Even though I had issues with the company I worked for, I still love the ecommerce industry.
There are always great challenges in both getting visitors to purchase something and then make sure you can fulfill them, and perhaps get repeat customers...
Month after month, I read all about all this need for huge web analytics software, to analyze tons of logs, for personalization and segmentation.
But when I've seen articles written by people who've implemented such methods, they are not getting what they want out of it....
I really feel the whole industry needs a new approach on how to point customer a to the great products that the customer is really looking for.
But instead what happens, those people who deal with the different parts of their multi-channel company, each channel's marketing people decide what navigation, what categories, what search, what tools, what help, and rarely is there any synchronization or common sense.
Then come the usability geeks, or anyone who has some common sense, see that there is issues with customers satisfaction in finding things.
It's just too easy to forget why we have each channel.....
To help the customer find the product...
But if each channel does things differently, how likely are those multi-channel customers to be confused???
Well anyways, last night as I was reading Christopher Rowley's, The Founder, a short sci-fi book. I felt inspired or envisioned to create a new product.
Basically a two-fold product:
1. To identify when and where, as fast as possible, the customer is not finding what they are looking for.
2. To create ways to help immediately, and friendly help the customer.
It's not quite well thought out yet...
And I have some other ideas as well...
Basically I want to find a way to create some cheap tools that get rid of the need for these unrealistically expensive tools like coremetrics.
I mean, really, it's nice to hack the data, but the data is not the customer, and you are not responding to what their needs are but to your delayed perceptions of them.
Anyways that's my thoughts...

Tuesday, July 13, 2004

Managing your Deployments

Currently, where I work, we have a source control system, with a dev, staging and two production environments.

Which can be a godsend in terms of rollback reliability, but can become pains in the rear, for deployment of projects that are affected by each other...

I think we need a new way of thinking about deployments, or have more usable tools to do what people call builds.

There has to be several parts.

1. The ability to remotely turn trusted caching on/off at a certain date/time
2. The ability to associate a group of files from different folders as a build to be ftped/deployed to any of a number of servers, that is equiped to de-build it upon receipt.
3. Then turn trusted caching on/off for all deployed servers, then to wait 10-15 minutes, then the code is deployed and caching can be turned back on.

I personally am frustrated with this situation, and am looking for ideas/solutions.

The hardest one is the ability to turn on/off trusted cache on multiple servers at a set scheduled date/time, without someone having to manually go into CF Admin and do that.

What if you had 10, 20 or 40 servers that you had to deploy new code for?

Isn't it amazing?

Well sometimes innovation comes out of need?

What kind of needs do you feel are unmet?

Thursday, July 08, 2004

Bring me Blackstone!

Macromedia - Developer Center : The Blackstone Tour Report

I have to admit, I'm rather excited to see this, any news is good news.

I also love how Ben Forta says:

"Is Macromedia committed to ColdFusion?" "Is ColdFusion dead?", and so on. Threads like this have been popping up for years now. So, just to make things perfectly clear, here are some facts (as presented to the groups):

There continues to be plenty of ColdFusion development going on. Google today reports millions more CFML pages than it did a year ago. That's good news, especially when taken in context. After all, most ColdFusion deployment is within organizations (intranets, extranets, portals, and so forth); areas that Google never even sees.
Macromedia continues to sell lots of ColdFusion. Sales are solid and have been for a while. That's a very positive sign indeed, especially for a product that is rapidly approaching its 10th birthday.
Lots of developers are using ColdFusion MX, the latest version of ColdFusion. When polling attendees at user groups, over half had upgraded to ColdFusion MX (including MX 6.1). That's good too.
Macromedia continues to be very committed to ColdFusion. ColdFusion MX (and the massive financial and resource investment that went into that release) is a testament to that. Then on top of the solid foundation and architecture of ColdFusion MX, we already have thousands of man hours invested into the development and testing of Blackstone, and we’re not even in the beta cycle yet.
To borrow (and slightly adapt) a quote: "Rumors of its demise have been greatly exaggerated."

Way to go Ben, that's the way to tell it to them.

Wednesday, July 07, 2004

What does the customer really want?

It starts with who is your customer?

Who are you developing it for?

Sometimes it can get confusing, between what your boss(s) want and the client wants.

I keep reiterating this, because it's still obvious that we have issues in the industry with communications.

Maybe there should be a committe, of designers/developers to help standardize the industry, in terms of training, salary, tiers of skill level, certification, and all the professional skills we develop with time.

It's just the sad part of being self-taught, we just miss some things, because we never had the opportunity to learn them, or be aware that we did not know we needed them..

If we webbers, are more geek or nerd, that does not highlight us for having great communication or planning skills.

I keep thinking about that old Simpsons/Child Rant, on a road trip, "Are we there yet?"

And yet, I think where do we want to be as an industry in the next 5-10 years?

Is the real change going to be technilogical, or in ourselves, and the demands for what skills we have to have to keep our job.

What does it take to keep your job?

Wednesday, June 30, 2004

Search and Learn

My big project this summer, is the revision of my companies, site-search. We have over the last two years, tried external solutions. But it really turns out that we have a unique data mix and categorization mixture, so it makes it really hard to have an automatic fix.

So while my boss is out to France, for his yearly vacation, I will working on Search Revision No. 4

What will be unique about this version?

It will be a learning search.

The way I look at it, you can either spend months creating/adjusting/tweaking data, spelling corrections, synonyms beforehand, or you can do it gradually over time.

My bible of search from Nielsen Normal Group E-Commerce User Experience on Search, I keep hoping for a 2004 version.

I have two goals, make it easy for us to gradually improve the search, and improve the usability of the interface to deliver what the customers want, not just what we think they want.

When you communicate with someone, there are two parts, sender and receiver, but when you are searching for someone, it becomes a bit more difficult to really understand each other.

When we adapt to a new situation or interface over time,we create a mental paradigm of how to do things in this interface/world/universe.

Like I know that if I type X, I'll get the Y results, I was really after.

So to make this learning search really work, we'll have to get direct/indirect feedback from our search users, our Magellan wanna-bes. Searching for the fountain of woodworking.

For direct, I have to find a way to get direct/indirect feedback from customers, when there is:

Too Much Results
Too Few Results
Bad Results
Misspelled Keywords

On one hand, how likely are customers going to be helpful in improving the search? If their core needs of finding what they are looking for, is not met???

So I have to provide the direct feedback tool, as long as that information is only available internally.

But I also have to identify indirect feedback, based on the results themselves.

My goal in terms of results is two pages of extremely relevant results. Because reports show that people are not willing to scroll thru more than 3 pages before they get frustrated.

So I can measure indirectly based on how many records the FullText Query returns, as well how many pages, but after that it gets a bit shady as to how to measure bad results.

Now beyond the measuring, into adjusting results.

Once I have some adjusted data, such as changing the rank multipliers that change the value I place on the data I am searching. I am searching thru 2 different tables, 3 different ways totally. So I have a dynamic field that can be adjusted to say table 1 is more important for keyword X's results, then table 3 is.

In addition, I check for synonyms, misspelled words, to replace what the current keyword is to help fix it for the customer.

So this is not quite a simple project.

Perhaps this can be even called an Enterprise Project??

Well more to come later, on this challenge of the lifetime...

Later :)

Thursday, June 24, 2004

Frameworks: The Good, Bad and the Ugly

D.Ross.Blog: Frameworks, Methodologies

Let's get this conversation out in the open.

Let's take some quotes from my infamous comments...

"That's my point, I am sure it is a good methodology, and I am all for improving our standards of coding to make ColdFusion is the solution for the web applications.

However, no methodology should sacrifice core values of performance and usability.

I understand for example, how it helps to have a common methodology for a team.

I am not here because I like slamming methodologies, I just want to make sure we are choosing methodologies for the right reason.

Because no matter how a methodology improves our coding standards/skills, it doesn't matter as long as it's coded to be the most efficient in performance and usable to the clients."

My point is that several-fold:

1. We should not just jump to a framework because everyone else does, or because it's a trend or fad. It must be because it fulfills the needs we have as a team/coder and because it delivers on our core values.

2. I don't care if it's the most elegant framework in the world, but if it takes away from the core values, then it's really not yet a good framework.

Here are my core values, and I hope they are similar to yours:

1. Performance and Scalibility: As my web apps get used by more and more people, I have to make sure that the web app is built for handling that, by using the most EFFICIENT/FASTEST LOADING ways of delivering functionality/content or whatever.

2. Url Usability: It sounds trivial, but it's actually a very important subtle thing. People like to be able to bookmark, or understand what page they are at. The old fusebox method of having all apps be on one page with different url variables, just doesn't cut it. Especially in the day of trying to do search engine optimization.

I do know it can do the seo friendly urls, but it should be on seperate pages. Which can be fusebox if you like that just includes the fusebox code. But we need to go a products page or a shopping cart page. There needs to be a visible difference in the address bar between pages.

3. The framework should be built to be easy to understand for people who don't even use that framework.

Point-in-case, at work we have several fusebox apps, that we don't even bother trying to fix because it's so convoluted trying to find where the actual code is and what not.

Now, I do approve of frameworks, as long as they are done right.

Of course it is much better than going blindly with no standards or no framework....

But we have to evaluate what is a good framework and what is a bad framework.

There are plenty of innovations that fusebox and other frameworks have brough to the cf world, that I like.

Like the idea of wireframing your apps, I still want to play with that as a great way of planning out your app logic.

I have not yet played with mach-ii. I am honestly a late adapter, probably always will be.

And maybe I still think that a good web app, is 1-3 pages with 5-10 cfincludes...

And because my code is commented, indented, planned, easy to read and understand, then I don't have to worry about dying, and having someone try to figure out what the heck my code is trying to do.

Unless coldfusion has changed so much that the speed differences are a lot different than they used to be, then I'll be glad to change my mind.

As long as the reuse method is the most EFFICIENT/RELIABLE way.

And as a nice as cfc's are, they are not it, nor are cfmodules or cffunctions or cfscripts.

CFINCLUDES are still the fastest gun in the REUSE West...

Wednesday, June 23, 2004

Think Longterm

One of the things that is hard to grasp as a newbie, is the importance of thinking longterm.

Because it takes a while to understand the importance of:

1. Commenting
2. Planning
3. Project Management
4. Error Proofing
5. Testing

There is probably more to the list, but most newbies, think if the app works, that's all that matters.

And I was the exact same way, give me a goal, and I couldn't wait to code.

But then begin the painful lessons:

1. This isn't exactly what I had asked for.
2. I have to change something in your code.
3. I get too many errors on this app, did you error proof it?
4. This application, 1 out of 100 gets the job done, what's happening 99% of the other times? Did you test it?

The reason why, I and other classicists/old schoolers/purists emphasize these things so much, is because you are dead on aiming for a hungry bull just ready to rip you to pieces.

We all make mistakes, and that's okay as long as we learn from them.

And to help you prevent them from the long haul, you need to take a different approach to your coding.

Take pride in your work, knowing that it is going to do it's thing for a long time 5-20 years, who knows??

However then make sure the app/project will last that long by following the methods that help make sure you get there.

Let's briefly go over these top-notch methods:

1. Commenting, Elyse over at Anticlue has a nice article on commenting, here

But simply said, commenting can be simple or complex, but the more details you put in there, the easier it will be to fix/repair in the future.

Which is the whole purpose, put in the details now, to make it easy for you later.

Also it's much more important, if people besides yourself are going to look at your code. Take pride in your code, make it look nice, neat and indented.

2. Planning, I can't say enough about planning. Planning is both the skills you develop to architech the project/app, but it's also making sure you have a clear/written and documented vision of your project.

3. Project Managmeent, it sounds like a pain in the ass, but really it's godsend. Because this prevents communication mistakes.

I don't know how many times, it's happened to me, where I thought I knew exactly what the client/customer/boss wanted for the project, only when I started coding, they said, "What the heck is this? This isn't what I wanted?"

Isn't that a pain???

So learn from this, use a simple-to-complex project management system to prevent communication mistakes, as well as not wasting anyones time until everyone knows what everyone else wants and will get done.

4. Error-Proofing, this is one of my concepts, if someone else came up with it before me, I apologize. Error Proofing simply means preparing your code to handle any possible error.

This means what copy/app changes you want to happen based on what kind of error.

Also this can mean some cool looking error reporting, Qforms has some interesting ideas on form error handling.

Also it means, knowing your incoming data, protecting your datatypes. If your app depends on results from a query, what if you get no data one time? How does your app handle it?

5. Testing, this is one the that my current boss is a master of. You give him an app, and he will break it into smithereens. :)

Which is both funny and a good thing, because it takes a lot of work to do testing, and prepare your code.

You gotta lose your ego, come up with a testing system, that always changes and evolves, and make sure your code is really PERFECT, before it gets public.

That's all for now.

Remember think longterm, do you really want to have to fix some crappy app, you did 2-3 years ago, but was written so horribly no english person can read it?

I don't think so.

Tuesday, June 22, 2004

Ensuring CF As an E-Commerce Platform

ColdFusion Developer's Journal

Well here it is, my first official CFDJ article, if this is well received, I would love to write as much as I can.

I know this may sound weird, but I've waited a long time to be worthy enough, or ready enough. I'm just happy to be able to give something back.

Anyways enough of my blutherings.

Feel free to check it out at the link at the top, and give me your bloody honest opinion.

Thank you everyone!

Macromedia MXNA Aggregator in Flash

Phoenix ColdFusion UserGroup: Welcome to the Phoenix ColdFusion UserGroup

I know I may have come across as an acerbic anti-flash person. But when I see something cool, and it makes sense and is usable. I gotta get behind the bandwagon.

It may sound simple but something like this on the right side of the Arizona CFUG site, just rocks my boat.

It's perfect, especially when any of my posts show up :)

Another flash integration of the MXNA Aggregator, that I am just beginning to drool over.

This interface is very nice, although I still miss the ability to right-click and open into a new window. It still is a very beautiful application.


:} - A Good way to share learning


The reason I really like this community q&a site, is because there is so much knowledge and experience that we can share and learn from.

I don't claim to know it all, but I would like to learn it all, or as much as humanly possible.

Next time you have a problem or puzzle, or just want to fill some time helping other cfers, checkout

What's on your bookshelves?

It turns out that most of us, are trained by book knowledge, that and experience of course.

So I come to you today, to share what's on my book-shelves, and curious to here what you have on yours?

  1. Oracle PL/SQL Tips and Techniques

  2. Learn SQL Server 2000 Administration

  3. The Guru's Guide to Transact-SQL

  4. The Guru's Guide to SQL Server Stored Procedures, XML and HTML

  5. SQL Server 2000 Programming

  6. Instant SQL Programming

  7. Visual DHTML for the world wide web

  8. XML in 10 Minutes

  9. SQL Server 7.0 in 10 Minutes

  10. SQL in 10 Minutes

  11. Stupid Web Tricks

  12. Community Building on the Web

  13. Designing Web Usability

  14. Homepage Usability

  15. Website Automation Toolkit

  16. XSLT for Dummies

It is still sort of amazing the value of the printed word, as a means of communication and education.

Each of these books is sort of a step on a giant ladder, as we progress thru our skill-sets and our gradual shifts of paradigms.

Even to this day, I like to read >Website Automation Toolkit, because it was written before content management systems or backoffices, really existed. So here was this Paul Helsinki, who had writted for the Web Techniques Magazine, had come up with a couple different concepts of using perl to create a content/site management system.

I still get some new ideas, or refresh my concepts when I read a good old book like this.

Of course, I really got my web start, using the Instant HTML Book by WROX, a well-used book of mine,as a HTML reference.

In a way it is very pleasant to look back and see where we were, as long as we do not get too proud or egotistical about it.

Have a good day everyone!

Monday, June 21, 2004

How do you train yourself?

For me my path to coldfusion, was half a drive to know more, learn more, be more.

I was working at a Tech Support job for a local ISP. Getting that rapid burn out we all get after time, and wanted to learn more.

For example, I saw a need for an intranet that held all the massively disorganized information, on helping customers. So I started to learn javascript, html, especially from my favorite html book Instant HTML - Programmer's Reference by Steven Wright.

I pushed myself to find out how to create a framed interface, (yes, I know frames suck, but can be a good solution for intranets.)

Anyways I self-taught myself, html, javascript.

And as that went on, I became the webmaster, hostmaster and what not.

But I was never settling for whatever I currently knew, I always wanted to know, learn and master more.

So I was always bothering the web developer, at the ISP's web department to learn more, to learn, what the heck is coldfusion, etc....

And with time,grew to master more and more of my skills.

But as I have gotten older, I have seen how so many (apparently, but not confirmed) people were trained by their companies, or were sent on conferences by their companies.

I just have always wondered, what it took to get more education, the support of even half-way from companies.

Also I wonder what all of your learning paths are like?

I realize things have changed since the dot-com bomb, but if we're all to get better, and there really is no formalized training system, how can we improve other than thru conferences?

Especially if we're all so self-taught, that must inhibit our real awareness of each other's work in terms of quantity and quality.

That's my thought of the day...

How do you train yourself?

Thursday, June 17, 2004

Plurals, where's my plurals??

Well yesterday and today of all things, I am working on re-thinking my companies site search functionality.

For which we are using SQL Server 2000's fulltext capability, I have been exploring and tweaking for the last 2 years. I thoroughly enjoy attempting to configure it right.

There are so many ways to go wrong.

Such as:

1. What are the customer's looking for?
2. Where's the data that's most likely to have the best pointer to the correct results?
3. How do I display results, and tools, and aids, to help the customer in case of difficulties, or just to make sure the search is usable.

My two search bibles, which I will never part even in death, are:

E-Commerce User Experience, Search Section/Chapter written by Nielsen Norman Group for which we know the most about the author, Jakob Nielsen, Usability Guru.

Then there is this color pdf from Cognetics, on Search Interfaces for Information Portals.

Also I still love Kristoffer Bohmann's article on Effective Search Results

In this deadly art of search design/development, we few mortals must dare to solve two main riddles.

1. How the heck can i find what the customer is looking for??
2. How should I display the results back.

Well yesterday to today, I've been working on question or battle #1.

Figuring out how to hack the data.

In our case we are searching 3 different ways, and then pre-determining the rank multiplier value for each, as well as a new feature I am adding called Rank Drop Off.

Rank Drop Off means, I got too much wanking results, and who's really going browse thru 30 pages of results. So to help eliminate the flock from the schlock, on the fly I determine the ranking percentages, and then drop off, anything below a certain percentage.

Now of the results, that come back, I am want to make sure no duplicates occur. Unfortunately we have product groupings that have the same title, (another item on my wishlist for cleanup), so I created a sql view to help give me only unique, active and searchable product groupings.

I also have to grab the top rank value of the first row, and use that to on the fly calculate a percentage of that ranking value for all the rest of the records.

I had to bring out my trusty rusty handy dandy The Guru's Guide to Transact-SQL, to look up joins, subqueries, all that funky stuff. That we in the Blue's Clues sql training class have learned.

So day later, I have all the layout, design, different modes, and yet I can not counter the PLURALS!!!


It could be the end of world, for us, stay tuned, as we find out what happened to the DEADLY PLURALS!!!!

Anyways, I can't get the freaking plurals to work.

I say table, hoping to get table and tables back.

freetexttable (product_groupings, title, '#filter#',80)

I have tried putting it in quotes, adding an * before and after.

I know there is an inflectional mode if you use a different method other than freetexttable, but freetextable has the rankings I need.

Plus there is just too much data, to hand-code a synonym or plurals table.

Do you oh wise wizards of Blue's Clues, have any ideas or suggestions.

That is beside the old "Jump off a cliff, Craig!".

Which is in my employment contract to not do, on thursdays! Especially in summer.

Anyways, that's my brain in a jar.

Wednesday, June 16, 2004

Coldfusion does not do Ecommerce Well??? - Blog

Go down near the end of the page, and you'll see the comment by Ben Forta on ecommerce.

I work for an ecommerce site, so hearing that ColdFusion is not a good solutions, really just hit me through the roof.

To make it easy, here's the Ben Forta quote:

Craig, the e-commerce space has never been where CF does well. Sure, it did ok there in the .COM era, but that was an exception, not the rule. Most of CF development happens on intranets and extranets and portals, internal stuff, and that is what CF is best at.

How can I deal with this? Can Ben Forta be wrong, or have we as coders just not persuaded the world that we can create stable, profitable ecommerce sites?

Perhaps, if you understood my personal work situation a little bit, that may add to your understanding.

For the last two years, I have been working for an ecommerce company. From day one, my job has been to clean up the horrible coding that was put in place, by a company using a packaged ecommerce system.

Every day, for the last year, I have been fighting with my boss, to help finish the cleaning process, to both clean up the code, database, files, folders everything.

Only each year, we never get it done, and he keeps coming up with reasons, why any other technology would automatically solve all our problems.

So here I am, trying to fight for my sanity, and wonder what can I do?

I am not a quitter, so every day, I work on different ways to improve the quality and stability, just so we can cleanly add new features and functionality.

And the funny thing is, each year, my boss finds a new technology, tries to get bids for pricing and either the prices are way out of our reach, or they really are not good solutions.

In other words, when you have a complicated site, it's unrealistic to just buy a package, and expect to easily deploy all your custom needs and features.

Also, it was way more expensive than coldfusion was.

So that adds to my relief and frustration, trying to save my job, trying to do the right thing no matter what.

And then this year, we decided to finish the clean up work, but instead of asking me to help finish it up, or to hire some desperately needed additional personnel, he decides to outsource all the final cleanup work.

Now during all this time, I have been starting my process of writing my first CFDJ article, hopefully to come out in june. All about why coldfusion is a great solution for ecommerce.

I get more and more fired up, I had presumed, that the reason coldfusion was so down-trodded upon, was that we really hadn't done good work.

I mean the kind of good work, that doesn't require you to come back a week later, and fix all the errors, you never fixed in the first time.

Have we as an industry done really good, high quality work??


I mean I still hear of people, that do not see the need to comment their code, or think worrying about performance is lame.

So here I am, my boss just left for 3 weeks in France like he usually does every year. Summer is a our slow time, which is the perfect time to finish cleaning up the code, and totally revamping our ecommerce coding, database and all.

But instead, it's much more important that he goes to France.

Anyways, I don't mean to get on his case.

But I am on fire here, and wanting to show and prove why coldfusion is a great answer for Ecommerce.

But if Ben Forta doesn't believe it's good for Ecommerce, what am I to do?

That's why I get upset about all this flash, oo, java talk, because I am not sure, that as cfer's we've really mastered the basics?

I mean, are the customers happy with the work you did, has it improved their profitability,because you did it, or it was done in ColdFusion?

My point of all this, is that this is a fight, not some hoping things work out right.

I am dedicating this blog to prove to the world that ColdFusion is a great solution for speedy, stable and profitable ecommerce sites.

But that means, that we all, have to be coding at higher levels of quality.

Project Management
Correct SQL Coding Stability and Performance
Bug Tracking
Error Proofing

Whatever it takes to prove that Coldfusion is not dead, nor dying.

That's my humble freaking opinion.