I suspect that this entry will get a few comments. I hope it does.

I was putting together my deck for EMC World and thought that maybe it would make good blog material. People are always asking me about search, classification, analytics, and visualization anyway, so I thought, why not?

I started looking into the history of search engines and I was very surprised to find that this is really new to the technology scene. I mean, full text is really only 18 years old. When I think about it, for me, search was always there. I worked in an advanced research shop in Mountain View, CA back in the early 80’s and was always on the internet. All these arcane applications that most people never heard of: Gopher, Finger, Archie, etc. (Unless you are really old, like me and Dave Reiner). I was using Netscape before it was a company. Okay, I’m old. I’m not a web native. I’m a web immigrant. But full text apparently got started just before I started at this R&D company, so to me, search was just natural. I thought it was always there. Little did I know.

A bit of a historical perspective:

  • In 1945, Vannever Bush (Wikipedia lists him as, “A leading figure in the development of the military-industrial complex and the military funding of science in the United States, Bush was a prominent policymaker and public intellectual (”the patron saint of American science”) during World War II and the ensuing Cold War, and was in effect the first presidential science advisor”).
  • In the 1960’s, Gerard Salton invented the SMART Information Retrieval System. Many important concepts in information retrieval were developed as part of research on the SMART system, including the vector space model and relevance feedback.
  • In 1960, Ted Nelson created Project Xanadu with the goal of creating a computer network with a simple user interface. It didn’t succeed. http://en.wikipedia.org/wiki/Project_Xanadu
  • In 1963, Ted Nelsoncoined the phrase, “hypertext“.
  • In 1972, ARPA invented the ARPANet which, “was the world’s first operational packet switching network, and the predecessor of the global Internet.
  • In 1990, Alan Emtage creates the first search engine, “Archie” (short for Archives”)
  • 1991: Tim Berners-Lee combines with TCP/IP and invents the WWW. First WWW site, http://info.cern.ch/, posted August 6, 1991.
  • 1994: Berners-Lee founds the World Wide Web Consortium (W3C) at MIT

Today: We know we have too much data.

I find it interesting to look at some old presentation materials from User Conferences (Documentum folks call these User Conferences “Momentum”; EMC’ers call it EMC World. Same thing).

Here’s a Momentum presentation by a customer (Nortel Networks) back in 1998:

The meat of the discussion: back in 1998, 5 years after the invention of the internet, Nortel already has the “too much data, not enough information, and I can’t find any of it” problem. Wow.