I mentioned in my previous post that I was stunned to realize that search as a a technology was really only about 18 years old. I just didn’t realize. No wonder it doesn’t work well.

The other thing that I’ve observed is that most people, including architects, designers, product managers, etc., don’t really think through the question of how the users will invoke search. What are these people searching for? Why are they searching for it at all? How will they approach the job at hand?

As one of the stakeholders at CM&A around search, indexing, analytics, and visualization, I get asked a lot of questions about search and what we’re planning to do, about it. I also get to ask questions back like, “What kinds of search do you do?” and, “How well does it work?” Most of the time the answer is that it doesn’t work very well and “I prefer to search the internet as opposed to my own intranet”. It’s an interesting statistic that about 50% of all keyword searches fail. That is, the user has to modify the query keywords and try again. And that’s just the interfaces like Google, which are pretty minimalistic.

I admit it, I’m not a big fan of PowerLink and according to most customers I talk to, they aren’t either. I think that one reason is in how customers and employees approach the question of search and how it was implemented is a disconnect. So that’s today’s topic.

I started a set of research which I call “the psychology of search” and is really focused on how people approach the problem/question. I think there are 3 kinds of search:

Recovery, which is looking for a specific thing in a specific place (“Honey, have you seen my car keys?”);

Discovery, which is about looking something up (“Okay, what’s on the calendar for today?”); and

Exploration, which is akin to information research (“Hmm, what are the competitors doing?”).

The graphic below might sum this up:

These are not the academic definitions – I will refer you to Broder’s Taxonomy for that – but customers seem to get the picture for this pretty well.

These generalized categories arose from this slightly more detailed list of search types we’ve seen people execute:

• Specialized Search -Seeking specific information whose characteristics are tightly defined
• Generalized Search -Unrefined, broad, loosely defined. Not concerned with completeness of results…
• Legal Investigation
  • Defendant – an exhaustive search of all sources and assets for specific information. Generally tightly defined. Completeness of results is essential. Results must bear intense scrutiny from opposing council. Must provide transparency into idiosyncrasies.
  • Plaintiff – a detailed examination of assets for specific information. Typically more loosely defined. Resembles Knowledge Discovery in that plaintiff’s attorney’s will want to “cast a broad net” for information.
• Knowledge Discovery (KM) -R&D, Patent Analytics. Generally an unstructured search with a few keywords
• Business Intelligence -Data mining, trolling, pattern recognition, trending analysis. Usually performed on historical data archives to refine operational behaviors or processes
• Internet Search -The main categories for such queries are shopping, finding various web-mediated services, downloading various type of file (documents, etc), accessing certain data-bases (e.g. Lexis-Nexis type data), finding servers (e.g. for research ) etc.
• Situational Awareness -Watching or monitoring specific data sources for activities, trends, events. Generally accomplished by text mining of sources like LexisNexis, etc.
• Navigational -Characterized by knowing you have something but not remembering where you put it. Could have very high query rates if widely deployed. People prefer categorical navigation.

I think once you realize that there are all kinds of search types and that these need to be supported well, you will start thinking about the kinds of search and indexing technologies you will use to help your customers. Until this happens, though, search will be a very dissatisfying experience.