by: Wendy Suto

As long as the Internet has been around, it has remained a mystery to the mainstream public how Web sites are listed at the top of search engine results. There are many theories of how search engines and search engine optimization (SEO) initially began.

The 1990s

Alan Emtage, a student at the University of McGill, created the first true "search" program in 1990 called Archie (still in use today), to archive Web documents. The following year, Gopher started at the University of Minnesota, and this is when the concept of search engines began. In 1993, Matthew Gray created the World Wide Web Wanderer, the earliest known search engine robot that assists with ranking Web sites. But search engines as we use them today were born in 1994. In that same year, Galaxy, Lycos and Yahoo! were all started, two of which are still widely popular search engines today.

Yahoo! was among the first to implement SEO techniques, even though at the time they were unaware of the potential growth the industry would soon have at the turn of the century. Yahoo! Founders David Filo and Jerry Yang were trying to get their site seen by others on the Internet by giving it more exposure. Some excellent structure and tricky hand-coding, their site became more available for new visitors. They were not questioned about ethical business practices because nobody was sure what was considered ethical or unethical – there were simply no standards in place yet.

As the initial search engines were cataloging the early Internet, many business owners soon learned to appreciate the value of their Web site being listed in the search engines, as they first saw increases in visitors to their Web sites. They began submitting their URLs on a continuous basis, and changed their sites to support the needs of search engine robots. SEO companies started showing up, when they began experimenting with the concept of search engine optimization, with the emphasis initially on the submission process alone. Soon afterwards, the first automatic submission software was released, and it was then the notion of Spam came into existence.

The 2000s

SEO professionals have been seen in a negative light over the last five year, due in part because in early 2001, enthusiastic webmasters quickly realized they could overwhelm search engine result pages by over submitting Web sites. Unfortunately, as the Internet industry developed, search engines quickly became cautious of new SEO companies attempting to generate visitors for their clients at any cost, however unfair or unethical. Tactics such as keyword spamming, doorway pages, cloaking, and hidden white text placed on white backgrounds proved too much for the search engines to tolerate. As a result, the search engines replied with numerous countermeasures, created to filter out any techniques considered spam. That is good news, although it forced ethical SEO companies to start using more subtle techniques to assist their clients Web sites with obtaining rankings in the engines.

The "big 3" search engines, Google, MSN and Yahoo!, have recently come to the realization that SEO as an industry is here to stay, and to maintain effective results, they needed to accept the industry, even embrace it, and engines eventually partnered with successful, ethical SEO companies to establish typical standards for fair and ethical optimization. This is important to help keep information relevant and beneficial to visitors while still being unbiased to people who create the content on their Web sites.

The Current State of SEO

Today, there are major differences in how search engines work and how to get ranked in them. With the assistance of proper search engine optimization, Web sites can now have a equal fighting chance of obtaining high rankings. Because SEO is a highly specialized trade that requires both technical skills and business marketing knowledge, it is only through the combination of these two skills that one can properly implement SEO techniques to obtain high search engine rankings. Many SEO specialists have since now realized it is "search engines or bust."