Six Search Technologies You Should Know About

The days of the one-size-fits-all search algorithm are over. As impressed as we all were with Google’s PageRank algorithm back in 2001, the results you’d get from that algorithm today would be laughable in comparison to the results we have come to expect from search engines.

Why? Well, in part because SEOs have gamed the original algorithm to death at this point, but mostly because search technology has gotten so much better since then.

Here, then, are six search technologies that will (or are) reshaping the search landscape.

1. Collaborative Filtering. Loyal readers may wonder why I don’t just change the title of this blog to “Collaborative Filtering Thoughts” since I mention this technology about once a post. Collaborative filtering is technology that matches your interests to people similar to you, best expressed in Amazon’s “People who bought this book also bought . . .” Many Web 2.0 applications are based on this principle, such as StumbleUpon, del.icio.us, and Flixster. I’m very bullish on collaborative filtering, simply because I believe that the “wisdom of crowds” can be far more effective than even the best algorithm for many types of searches (product reviews, restaurant recommendations, someday perhaps even dating!).

Examples of Collaborative Filtering: Collarity, Launchcast, Flickster, Amazon.com.

2. Personalization. As the name implies, personalization uses data a search engine has about you to serve more relevant results in the future. For example, if I continually do searches for “lake trout” and “fly fishing”, a personalization engine will likely conclude that my search for “laker” is not for a basketball team, but rather for a fish. Personalization has the potential to be very powerful, but it also comes with a price – privacy concerns. Ultimately, this technology will only work if consumers really trust a search engine to protect and honor their personal data.

Examples of Personalization: Google Personalization.

3. Semantic Search. Semantic search identifies similarities in words and phrases. Thus, if I searched for “telephone”, the search results might show me results that contained sites about “cell phones.” In the paid search world, this might mean that you buy the word “mortgage” on broad match, but end up getting matched with terms like “refinance” and “home equity” because the search engine considers these words to be semantically related to one another.

Examples of Semantic Search: Hakia, Yahoo “also try” results.

4. Clustering. Clustering, or clustered search, tries to categorize words or phrases into a taxonomy (or groups) of related themes. I might type in “auction” and a clustering engine would show me categories like “Online Auctions” and “Fine Art Auctions” and beneath each category I would find sub-categories like “eBay”, “Ubid” and “Sotherby’s.” Clustering is a great way to provide additional navigation options to users who might feel overwhelmed by the raw search results.

Examples of Clustering: Clusty, eBay search results.

5. Local Search. Local search can be described in two ways – either it’s interactive maps like Google Maps, or it’s geotargeting based on the IP address or user registration information. For example, as a registered user of Yahoo, Yahoo knows that I live in the Bay Area (I gave this info to them when I signed up, and I was actually honest). As a result, I get a lot of ads for local events, car dealers, and Bay Area real estate brokers. But even if I didn’t register, Yahoo could still have a good idea of my location by looking at my IP address, or by cookie-ing my searches.

Examples of Local Search: Google Local, Krillion.

6. Human-Edited Search Engines. As funny as it may sound, humans are making a comeback (here’s hoping I don’t see the top of the Statue of Liberty on my next beach walk . . .). As the number of Web sites multiple, it becomes harder and harder for anyone to filter out the all the noise and get to the good stuff. So rather than do the hard work yourself, why not farm out the effort to someone else? That’s the theory behind search newcomers like ChaCha, Mahalo, and really Wikipedia when you think about it.

Examples of Human-Edited Search Engines: ChaCha, Mahalo.

Phew, that’s a lot of different search options. Ultimately, I’m still throwing my hat in the ring with collaborative filtering, but you can bet that each of the technologies above – as well as many that have yet to be developed – will play a role in the future of search!

 

Why the Internet Will Make Roger Ebert Obsolete

Anyone over 30 no doubt remembers watching “Siskel and Ebert at the Movies” growing up. A “two thumbs up” review from the two movie reviewers could easily drive millions of dollars at the box office for a lucky movie.

And despite Siskel’s death, Ebert still continues the show, now with Richard Roeper as his cohort. There’s no doubt, however, that the importance of a “two thumbs up” rating these days has diminished, and it’s not because of Siskel’s death.

Indeed, the importance of all types of reviewers, be it movie reviews, restaurant critics, or even consumer reports, seems to be growing less and less by the day. No doubt the Internet is a major driver of this trend.

To date, the Internet has largely impacted reviewers by simply giving consumers access to more choices. You no longer need to wait until Sunday night to watch Ebert review a movie – you can instead go on to RottenTomatoes.com, or a movie blog, or a chat board, and get dozens of opinions about the latest blockbuster.

The same is true for getting information about autos, electronics, books, hotels, and so forth – a quick search on Google will give you hundreds of opinions about any area you want.

The downside of the explosion of information is that it becomes difficult to wade through the data and find the stuff that is actually relevant to you. Unfiltered information is useless and actually makes you want to go back to an expert like Ebert that you trust.

Most Internet review sites today, however, have solved this problem, usually by asking reviewers to give a star-rating (sometimes on multiple factors) to whatever they are analyzing. You can then see an overall summary of the collective wisdom without having to read 500 reviews.

But the best is clearly yet to come. As any loyal reader of this blog knows, I love collaborative filtering – the Amazon.com idea of “people who bought this item also bought this other item.” So far, the online review sites have not utilized this principle. For example, if you go to TripAdvisor.com, you can always find dozens of reviews about a hotel, but it is difficult to know whether a particular reviewer shares your criteria for a good hotel.

With collaborative filtering, TripAdvisor could start to match you up with people who have similar taste in hotels. So if I really liked the Motel6 in San Antonio but hated the Ritz in Half Moon Bay, and you also had the exact same opinion on these hotels, the next time I was thinking about going to Dallas, I could review your picks for Dallas and likely find a hotel that really works for me.

Collaborative filtering actually works on a much greater scale: if there are 50,000 people reviewing hotels across the world, the odds are pretty great that whatever city I am visiting that someone with my particular taste will have already reviewed a hotel in that city.

There is a new web site called Flickster.com that is doing this with movies. I’m assuming that it will work along this model – the more reviews I submit to the site, the more it understands my preferences and matches me with other reviewers. Over time, I should be able to type in any movie and get a pretty good idea of whether I will like it.

This is a lot more accurate than listening to Roger Ebert because it is a lot more personal. I’m sure that if I sat down with Roger and told him about movies that I liked and disliked, he could come up with some good suggestions (for the record, I have actually talked to Roger Ebert several times, as I used to review movies when I was in college in Chicago). But even personal recommendations from Roger would not be as powerful as the collective knowledge of thousands of moviegoers.

There’s no doubt that this model will soon be applied to movies, restaurants, hotels, and perhaps even doctors, lawyers and other service providers. But I actually think that the most interesting application of collaborative filtering will come with respect to eTail – online shopping.

Right now, every comparison shopping engine uses a star-review system to enable customers to provide feedback on their shopping experience. eBay and Amazon have a slightly less accurate system, basically asking the customer if the experience was positive, negative, or neutral. And Google’s “Quality Score” measures the frequency that users click on a Google ad, go to a Web site, and immediately return back to Google (click the back button), a measure of the lack of relevancy for a particular site.

All of these systems, however, are still the ‘old school’ wide swath of information. From general information, you get a general feeling for whether a merchant is good or bad, but you don’t really know if that merchant is right for you in particular.

For example, let’s say you don’t care whether the merchant has 24 hour customer service, but it is vital that the merchant offers Saturday delivery. You can’t easily glean this information from 500 user reviews.

But with collaborative filtering, this becomes very possible. As you review stores, the collaborative filtering system starts to understand what is important to you. It can then match you with users who have reviewed other stores. After a while, the system’s accuracy cannot be doubted. You can do a search for “digital camera” and the system can not only recommend cameras that you are likely a good fit for you, but also recommend merchants that will provide the best customer experience for your needs.

Most consumers would gladly pay a few dollars more for a product if they knew that they were going to have an awesome shopping experience. I think that that is actually the promise of comparison shopping, a promised that is definitely not fulfilled by a vague star-rating.

The end result of such a system would be tremendously valuable to consumers. After all, a company that consistently provided horrible service simply would never be matched with new consumers – there would be no way to hide from your business quality.

It would also likely spell the end of an organization like Consumer Reports, which in most ways is no different than Roger Ebert.

The Internet is making today’s experts obsolete and creating new experts who’s status is determined by statistics, not the mass media. While this will put people like Roger Ebert out of a job, the end result is a great victory for consumers – product and business transparency that will make shopping easier and more personal and hold businesses accountable for bad service or products.