The Internet in recent years has, alas, become the victim of a mass commercial onslaught, resulting in extreme competitiveness in the field of Web site marketing. Small businesses and corporate enterprises alike have been investing huge budgets to compete for prominence in their market place, with the intent to drive as many targeted visitors to their Web site as possible. The most popular method for doing this is known as search engine optimization.

Search engine optimization, or SEO is the generic term for ensuring that your web site is optimized to appear as one of the top ranking sites on a search engine. It involves finding popular keywords and phrases that describe your product or service and adding them in specific places, and with certain frequency to your web pages. This can be extended further by creating additional pages to capture searches for hundreds or even thousands of relevant keyphrase variations. By this method it is possible to achieve high rankings and good coverage in the search engines.

Over 85% of Internet users find what they are looking for by using a search engine and few rarely venture past the top 30 listings. Commercial success for a web based company, in general, depends on how well the site is listed in the major search engines, of which there are only really half a dozen of value.

So who are the major players then?

Yahoo! easily out-performs most other portals in terms of search volume. Although they are shifting their model from a web directory to more of a community centre, a massive amount of Yahoo! searches and category mining is still producing a good share of the average web site referrals. The shift from directory to portal is an indication of Yahoo's attempts to increase its market reach by offering other services, thus improving it's market value and in the last 18 months there's been a massive increase in commercial advertising on Yahoo!

Another move by Yahoo has been the introduction of Business Express Review. A payment of $299 will get your web site reviewed in seven days. This does not guarantee inclusion and is only worth investing in if you are confident your Web site meets the very strict criteria. Very recently this has become a recurring fee for new sites that are listed. You should think carefully about the expected ROI before using Business Express.

Google. The number one US search engine founded by Larry Page and Sergey Brin is without doubt one of the fastest and most relevant search engines out there. It's popularity soared during 2000/2001 and currently is seeing over 100million searches per day searching over 2,073,418,204 web pages. Amazingly Google has reached this lofty status primarily through word of mouth advertising. Google spends little or nothing on media advertising and does not carry banner ads or pop-up ads on the site or within its results pages making its results pages very clean and uncluttered. Partnering with Yahoo! back in July of 2000 doubled their visibility making Google one of the top referring sites for anyone with a half decent listing.

Submission to, and inclusion in Google is free and ranking is based on PageRank technology, a method by which a sites ranking is measured by outbound and inbound link quality as well as on the page keyword density.

Overture (formerly Goto) have, in the last 12 months extended their market reach dramatically. By partnering with several of the big name portals including Yahoo!, America Online, Terra Lycos and AltaVista, Overture's search results reach approximately 75% of the U.S. Internet population. Most recently they completed a deal with Yahoo to feed the top paid listings in to the top of Yahoo! search results (at present only in the US). Listings on Overture compete with each other by bidding for keywords and paying 'per click' for rankings. This has made Overture listings extremely commercialised, with rankings based on advertising budget, elevating the presence of companies with deeper pockets.

The more money you have to spend, the more web traffic your site will receive with Overture.

Inktomi are not actually a search engine. They manage a large database of web documents that is scalable and is basically hired out to search services and portals. It is currently implemented by 8 out of the top 10 fortune500 companies and 2 out of 3 of the most visited destination web portals including MSN and America Online. Although inclusion in the Inktomi database is still free, it has become increasingly difficult to get new pages listed. This is since the launch of a pay for inclusion service, which enables you to enjoy benefits such as
24 hour refresh and allowing you to specify which pages are included in the index. Again, paid inclusion means that those with big budgets are the ones who will see the best coverage and positions, making it difficult for the average site owner to compete.

The changing landscape of SEO.

With more and more of the major portals and search engines requiring substantial inclusion, pay-for performance and site review fees it has become increasingly difficult to achieve sufficient coverage to promote your Web site. It requires a pretty serious Web marketing budget, and this is widening the gap between the average medium sized Web enterprise and the corporate giants. With the exception of search engines such as AlltheWeb, Lycos and Google, the landscape looks bleak for Web marketing on a shoestring. Where small businesses used to be able to compete on the same stage with large corporates, there is now a state of imbalance.

Although the situation may appear to be bleak at present, there is hope for the future. A multitude of new search engines and market specific vertical portals (Vortals) are emerging. New search engines, Wisenut and Teoma who have excellent speed, relevancy and very large databases, are tipped to be the next big thing in search and there are already a large number of Vortal sites for markets such as mortgages, cars and online casinos. Although the commercial search engines are producing the traffic right now, I get the feeling that the searching public will vote with their mouse clicks. When Joe Websearcher keeps getting the same commercial sites, and can't find the information they were looking for, Joe Websearcher is liable to re-evaluate his options and look elsewhere.