The Clash of the Titans – A Fresh Perspective

It’s fascinating to see the press mull over the search wars that are currently being waged.

Here is an article from Business Week entitled “The Counterattack on Google“. This article outlines much of how Google’s Adwords system employs a yield based mechanism to monetize their search results, as opposed to the current bid to position model pioneered by Overture, which Yahoo currently employs. In recent months, Google has added the Quality Score variable to improve the rankings in favour of greater relevance and less price emphasis. The virtuous cycle is that the happier the users are, the more they will search on Google, and this strategy is most definitely working. It’s working so well in fact, that Yahoo has rebuilt the system and relaunching it (currently dubbed Project Panama), which they hope will compete with Google.

I’m not going to go into all the current flaws in the Yahoo system which is being relaunched soon, but I’m going to address the strategic flaws in the Yahoo/MSN battle with Google.

This is something that both companies have really overlooked, and also the analysts trying to figure out how MSN & Yahoo are going to compete, which they need in order to forecast the share prices going forward. The flaw is that these analysts don’t realise that there are some critical flaws in the strategies of the two giants playing catchup. I don’t believe that unless there is a critical change in strategy, that MSN/Yahoo will beat Google in the short term and Google is going to grow very quickly in foreign markets in particular.

So what are MSN/Yahoo doing wrong?

The Internet is the world’s largest economy. What Google does right is that it allows advertisers based in any geographic area, to use a centralised system and buy advertising across multiple countries, without relying on reloading their campaigns, etc etc. Someone targeting all French speaking countries, for a French language product, simply selects Canada, France, Belgium, etc and all IP targeted traffic from those countries are served with the ads in questions.

Google has solidified their position in global markets, including smaller ones such as South Africa, by allowing anyone in the Internet Economy to participate in their global ad marketplace. This further increases relevancy. As a South African, when I search Yahoo.com or MSN.com, I don’t see commercial ads targeting me, but I do on Google. Yahoo & MSN insist on a decentralized model, which makes no sense to me whatsoever and companies in South Africa are unable to target local users on MSN.com & Yahoo.com.

MSN & Yahoo have strong brands outside the US, but they are not able to easily monetize their search, because of this decentralized “region by region” strategy. Google’s has really built a classy system around allowing anyone from anywhere to participate in their global marketplace in any language and any currency. This is where I believe the Yahoo\MSN will fall short in the long term.

By looking at Google’s earnings growth outside the US, I believe that not enough focus has been placed on the global market by the other search engines and also, US advertisers do try to target foreign countries with their products. Kudo’s to Google on this one.

Furthermore, this strategy will leads to other complications in the Contextual products in that Yahoo\MSN are not able to target international publishers very well. If the ads that Yahoo serves through YPN (Yahoo Publisher Network) only have US advertisers – and Yahoo US Advertisers only wants this traffic (i.e. no international traffic) – they will fail to deliver the solution that serves the needs of the publishers and hence does not compete with Google’s AdSense solution. Yahoo needs to focus on taking on Google globally, not just in the US. Many of the Yahoo search solutions are too locally (US) focused. It looks like MSN may follow this route as well.

Google has succesfully used Geo-Targeting to build it’s business, but MSN & Yahoo have gone with proprietary marketplaces – a key differentiator. Google tries to provide relevance while monetizing as many searches are possible, regardless of where they are based.

If Yahoo/MSN wants to take on Google, I do believe that they need to go back to the drawing board and rework their global strategy, and not to focus purely on each geography separately. They need to increase the monetization potential per search. Focus on the Internet as a medium, not a geographic location. In order to be relevant, you must act as a marketplace, connecting advertisers to searchers, by placing restrictions or hindrances such as geographic location, currency, language, etc, they will not maximize the revenue per search.

Take this example:

For every hundred searches on a search engine this could be the sample demographic:

40 US – English
10 US – Hispanic
30 Other Countries – English
10 Other Countries – French
10 Other Countries – Other Languages

For these hundred searches on Google.com – Google could yield a Clickthrough Rate of 100% because it has advertisers targeting every search, in it’s global marketplace.

If you looked at Yahoo or MSN, they would only be able to yield good results past US English, and maybe France. Also, because the auctions are run in multiple countries, they do not get the benefit of overlaying multiple bids and relevancy to get higher CPC’s and a mix thereof. Forget CPC auction benefits for just a second – their listings are now less relevant and have lower yields per search – something their stock prices are reflecting.

This is the brutal truth, whether they like it or not. MSN has got a great new angle with their demographic targeting capabilities and this shows that they are focused on revenue per search, but I’m not happy that the strategy is correct, especially given MSN’s strong international presence and brand. MSN have also integrated Search into MSN Messenger, which gives them an even stronger international base from which to grow their search capabilities. The problem is that when users search, they’re not going to see the local plumber advertising unless MSN has a local marketplace.

The funny thing is that I’ve had this discussion with MSN & Yahoo representatives and both companies are either dismissive or blasé about this particular strategic approach. If you carefully analyse the numbers, you’ll see that search is going to grow alot faster in many smaller markets, and if MSN & Yahoo are going to roll out marketplace by marketplace (slowly) they’re going to have to battle an even bigger Google ( I know, scary though). This is just one of the major flaws in the counterattack, but there are probably more than I can count.

It’s simple math that for every search, you need to display the most relevant adverts which generate the most money across the widest net of advertisers, and if they’re not doing that, they’ll never beat Google…

The Future of Search Engines?

This post was actually written over a month ago, but the Free Urchin release by Google prompted me to post it now.

I believe that Search Engines, in general, need to seriously look at what drives users to search and how to maximize their revenues from search in order to create a virtuous cycle (the more money you make from search driven by user satisfaction, the more it tends to self perpetuate and so on).

In order to seriously compete with Google, emerging search engines (A9, MSN, etc) need to understand what the driving force behind their revenues (or planned revenues are) are, and then apply it to their business in order to reach deeper into the consumer’s mind (and wallets, or so it seems). I believe that PPC and Natural Search (with revenue sharing links) potentially can merge. Issues like Click Fraud still plagues advertisers, rev share removes this risk. The fact that CPC’s keep rising should also tell you that business are underpaying for their traffic on the PPC engines, that’s what allows affiliates like to us to make a lot of money doing the arbitrage work.

Here is an alternative strategy to search:

The 3 cornerstones of a search is generally considered to be (defined by John Battelle in his book, Search):

1. The Crawler
2. The Index
3. The User Interface,

There is, however a fourth, in my opinion: The Customer Interaction. Poor quality results and websites tend to drive more similar searches per user, and less revenue per user. By monitoring the interaction between a user and a website, you can increase both the quality of the listings and the revenue of the search engine. Not enough time and investigation is spent by the major search engines on understanding post click conversion metrics on natural results, and this is the untapped gold mine of the search engine industry.

(Added: 15 November 2005

Google’s recent move into Web Analytics is definitely due to their foresight into this area. )

To understand Customer Interaction, you need to split the traffic intent of the user between Commercial & Non-Commercial. I’m going to focus on Commercial Natural Search Results for the purposes of this post, but the principles of usability would still apply to results that are of a non-commercial nature.

If the quality of websites in an index was monitored with respect to customer interaction, relative to the keyword that was searched on, you would probably find that search engine spam results would be quickly dropped to the bottom of the listings, organically. Search engine spam is almost akin to email spam – high volume, low conversion – high irritation.

Natural traffic should be independently ranked, based on the user experience for a given keyword, however, independent rankings, should not necessarily mean non-monetization of natural traffic. In the past, Inktomi was used to generate independent “Trusted Feed” organic listings, however, the downfall was that advertisers could not control the keywords that they paid on. The market moved to PPC, but in my opinion, PPC is just a pit stop to Cost Per Action – which is the metric that most smart PPC marketers use. Snap is trying to achieve much of what I describe here, although I don’t believe their knowledge of the affiliate marketing space is up to scratch to execute on this strategy without taking years by building direct relationships with each website. The idea should be to build a platform and leverage the platform to build the business.

At the rate that Google is growing, competing search engines like Yahoo needs a solid and ground-breaking strategy in order to reverse this growth trend, and I believe strongly that it lies in “The Customer Interaction”.

PageRank is dead. Links are becoming less important, due to the rise of RSS feeds. Websites are no longer pointing to other sites, they are syndicating the content on their own sites. Google is full of Search Engine Spam, mostly as a result of duplicate content feeds and syndication – can link popularity ever be trusted again? The use of inbound links as a method of determining authority is and will be eroded over time.

The other flaw with link is that by using CSS, which only become popular after PageRank, essentially you can put a link up once on your website and it propagates throughout the website, seamlessly. Search engine spammers are notorious for placing links from PR9 & PR10 sites, by buying them and increasing their PageRank.

Who can we trust?

How about you let the customer decide what they think is most relevant…

Imagine this:

A user searches on the keyword “Star Wars DVD” in Google. Top 3 results are for Amazon, Buy.com & eBay. Currently, you do not know which of the 3 the user actually purchased from. This is a huge slice of data, missing from your databases – can you imagine the potential to optimize the results based on the User Interaction. Firstly, you would find the most relevant results moving to the top of the listings, and secondly – you would be able to begin finally monetize the natural search market – something even Google have not yet achieved. Remember, that I’m not saying that you should change the results from the index – on the contrary – serve the most relevant results, even if the site does not have a monetization program (i.e. no commerce transaction), but if you’re going to send traffic to Amazon, at least use an affiliate link to gather both data and commissions.

There are a number of ways to achieve what is written above, without impacting the quality of the results.

1. Integrate Commercial Natural Search results with Affiliate Programs

This is actually easier done than said. For example, a user searches for “Blink Malcom Gladwell Book” on Google. The #1 link is (as expected) Amazon. Now, instead of sending the traffic to Amazon for free, and trying to figure out if this is what the user was looking for, send it via an affiliate link, integrated into their API (our Catalog Server does this, for instance – pretty easy). The user is then cookied and based on the data that Amazon sends back, you can check to see if there is Customer Interaction.

You can also improve your listings is you realize that users are not buying the book from eBay, but instead from Borders – this way, the search engine will become like a personalized shopping and searching partner – which is what the ultimate goal of search should be. As we know, Google & eBay work closely together, but for no remuneration on the natural side – imagine if this changed?

The idea is not to build two separate indexes, but instead to allow affiliate redirects to rev share on the basis on a natural result click via a particular merchant.

2. Earning Per Click

By monitoring Earnings Per Click on a website per keyword click level, search engines may find that some of their natural results traffic (sent via affiliate programs), have higher customer interaction rates, therefore higher yields than your PPC search program, an in this way, you can either force your PPC advertisers to pay higher amounts, or display Paid Listings, with natural listings.

3. Partner Database

Search Engines can and should start building a search partner database. Some partners, who may not be running an affiliate program, can simply add snippets of Java Script code to their site (ala Urchin) and allow the search engine to track the user through the site. This would greatly enhance your understanding of the customer interaction and increase detection of bots/spiders. You can also prioritize the high volume websites out, and perhaps eventually move them onto some type remuneration basis. This will also add a face to previously anonymous website, which typically are spam site or phishing sites, etc.

Conclusion:

I have just touched on a few of the endless possibilities engulfing our industry – maybe some are way out, maybe not – but if Snap succeeds, I’d be interested in seeing how close they come to this post.

The overall context of this post is that the search engine needs to have more insight as to the usability of a website in it’s rankings, but surely there is an opportunity to monetize natural listings whilst still delivering rock solid search results…?