The Search Engine Family Tree

Information doesn't exist in a vacuum and neither do search engine indexes. In order to get the most out of your search engine marketing efforts, you need to know how each of the engines interact with each other. Useful for both organic search engine optimization as well as pay per click advertising, Bruce Clay's Search Engine Relationship Chart® is a map to the ecosystem of the engines, ad networks and important directories on the Web.

Though most people look at the chart and inherently understand the value, they're often not certain how to use it in order to get the most out of it. Some of the common questions about the chart are related to the position of the circles on the page. Is Google on top because they're the most important? DMOZ is right in the middle; does that make them the most important? What exactly are we trying to say by putting MSN way at the bottom? In all cases the answer is simply: it fit better.

Bruce Clay first published his Search Engine Relationship Chart® in 2000. Back then, there were more major players in the search game and things were, to say the least, somewhat cluttered. The chart had 26 companies on it, everyone from Yahoo! to Magellan to that upstart company Google. Fifteen of those companies took their primary results from their own index; five of those supplied secondary results to other engines. Without a roadmap, it was an impossible task to keep it all straight. But over the years, things started to change. What was once a cluttered mess became a tidy interplay of a select group of companies.

You can see the progression from cluttered and confused to clean and neat on our SERC histogram. The organization of the chart mirrors the organization of the search industry. Where once there were many major players, today there are only a few. The power of search is growing by leaps and bounds. With mass adoption by consumers and increased interested by advertisers, search engine marketing is the most popular online marketing strategy today and estimates predict only more growth in the future. Already eMarketer estimates that $7.1 billion is spent in the search marketing space, dollars that translate into increased competition from all quarters.

But while the power of search grows, it's also consolidating in the hands of fewer companies. No longer do search experts talk about being in all the top engines. At most, they talk about only four, and in many cases, just one: Google.

The most recent version of the chart is a clean white document with just two clusters on it, Google above and Yahoo below. From these center hubs, several other engines receive primary, secondary and directory results. Paid results from AdWords and Yahoo! Search Marketing are served to ad partners.

The charts contains several layers of information, first is the question of where the results come from. Though the engines may have different algorithms, many smaller engines have contracts with the larger engines to supply the actual index. Google supplies AOL, for example. When a search engine has its own index, the circle on the chart is bordered in red. Should they be sharing those results out, a red arrow points from the primary engine to the receiving engine. Additionally, some engines get results from other engines merely to supplement their own. These connections are shown in blue and indicate that while the receiving engine may have their own index, they are not solely reliant on it.

Directory results are also shown on the chart via green arrows. Directories like DMOZ and the Yahoo directory are considered quality links and are nice things to have. Because they supply listings to the search engines, your site is likely to appear in the indexes. However, they are not the be all and end all of the search engine game, they do not guarantee rankings and it is not necessary to beg, borrow and buy your way into them. Additionally, the directories often have title and descriptions that are vastly different (and usually not as well crafted) as your own optimized Meta data. If you are showing up in an engine with your DMOZ tag and you would rather see the on-page information, the search engines have added support for the Meta robots command "noodp" which will instruct the engines to use the on page Meta data. It does not negate the value of the directory link, just prevents the directory's title and description for your site from being shown. Yahoo is considering adding similar support for the Yahoo Directory. If you are conducting your own search engine optimization efforts, it is highly recommended that you employ this command on your pages.

The last arrow on the chart is yellow. Yellow arrows indicate paid results, like AdWords or Yahoo! Search Marketing. Check out these distribution networks carefully because your PPC campaign has the option of appearing on these sites. If you are using AdWords, you can choose to appear either only on Google's search results or all Google's search partners (AOL, Ask, AdSense for Search) and content network (AdSense for Publishers). For Yahoo, your options are to appear on Yahoo and its search partners' results pages or on its content network (Yahoo Publishing Network).

The online version of the Search Engine Relationship Chart® is a Flash document and incorporates several other features.

You can click on any company and be taken to information about that engine, including information about search engine submission policies, how long they've been in business, what kind of technology they are using and how to get into the index via either free or paid methods. The page also summarizes a few tips for best results and lists the content received or provided by other companies. Links on the information page will take you directly to the submission page for the listed engine. As always, it is good to remember that submission is not a guaranteed method of being indexed.

In order to make the relationships between companies even more transparent, hover your mouse over a company's circle without clicking on it. The other unconnected companies will fade out, leaving behind the content partners. This system works both ways whether the search company or directory is feeding out information, as with Google below, or merely pulling in data from elsewhere, as with Netscape.

With the growth of the search industry and the narrowing of the field, it is more important than ever to know where and how search engines are connected to each other and how you can achieve the most visibility for the least cost. The SERC won't do your optimization for you but it's a valuable tool in guiding your efforts. Conversion and quality are of paramount importance in your search engine marketing campaigns.