Wednesday, June 30, 2004

Search and Learn

My big project this summer, is the revision of my companies, site-search. We have over the last two years, tried external solutions. But it really turns out that we have a unique data mix and categorization mixture, so it makes it really hard to have an automatic fix.

So while my boss is out to France, for his yearly vacation, I will working on Search Revision No. 4

What will be unique about this version?

It will be a learning search.

The way I look at it, you can either spend months creating/adjusting/tweaking data, spelling corrections, synonyms beforehand, or you can do it gradually over time.

My bible of search from Nielsen Normal Group E-Commerce User Experience on Search, I keep hoping for a 2004 version.

I have two goals, make it easy for us to gradually improve the search, and improve the usability of the interface to deliver what the customers want, not just what we think they want.

When you communicate with someone, there are two parts, sender and receiver, but when you are searching for someone, it becomes a bit more difficult to really understand each other.

When we adapt to a new situation or interface over time,we create a mental paradigm of how to do things in this interface/world/universe.

Like I know that if I type X, I'll get the Y results, I was really after.

So to make this learning search really work, we'll have to get direct/indirect feedback from our search users, our Magellan wanna-bes. Searching for the fountain of woodworking.

For direct, I have to find a way to get direct/indirect feedback from customers, when there is:

Too Much Results
Too Few Results
Bad Results
Misspelled Keywords

On one hand, how likely are customers going to be helpful in improving the search? If their core needs of finding what they are looking for, is not met???

So I have to provide the direct feedback tool, as long as that information is only available internally.

But I also have to identify indirect feedback, based on the results themselves.

My goal in terms of results is two pages of extremely relevant results. Because reports show that people are not willing to scroll thru more than 3 pages before they get frustrated.

So I can measure indirectly based on how many records the FullText Query returns, as well how many pages, but after that it gets a bit shady as to how to measure bad results.

Now beyond the measuring, into adjusting results.

Once I have some adjusted data, such as changing the rank multipliers that change the value I place on the data I am searching. I am searching thru 2 different tables, 3 different ways totally. So I have a dynamic field that can be adjusted to say table 1 is more important for keyword X's results, then table 3 is.

In addition, I check for synonyms, misspelled words, to replace what the current keyword is to help fix it for the customer.

So this is not quite a simple project.

Perhaps this can be even called an Enterprise Project??

Well more to come later, on this challenge of the lifetime...

Later :)

1 comment:

  1. This blog is awesome! If you get a chance you may want to visit this survey software site, it's pretty awesome too!