How Search Engine Spiders Work
There are hundreds of search engines available today, but some are far more complex than others. This article will give you an overview of how some of the most popular ones work.
Let’s start with a smaller engine: InfoSeek. They only index about 200 words of your web page, so it’s important to make sure that you have meta tags on your site, and that the most important things are listed first. The information you put in your meta tags will be used to display a description of your site, and most meta tags can contain about 200 characters of text. The keywords meta tag, however, can have up to 1,000 characters.
These simple rules are important to keep in mind for all search engines. The more important that the information is, the closer it has to be to the beginning of your meta tags or even the beginning of your site’s content. Many search engines won’t even touch your meta tags so it is important that you have the same information in your body that you have in your meta tags (although you obviously cannot simply enter lists and lists of key words as this would be detrimental to your site’s content).
The AltaVista search engine will send Scooter, its spider, to check out your entire site. Scooter can take as long as three months to spider and fully index your site – the average spider only takes 6-8 weeks. Scooter will normally spider somewhere between two and ten pages from your site each week. This means that the longer that your web site lasts, the better it will be indexed which is in example of how search engines implement Darwin’s Theory into their ideology.
Excite used to be a search powerhouse, but has now been dropped as the provider of AOL and Netscape search, so it’s less important than it once was. The algorithm it uses to determine keyword relevance is very complicated: it indexes your pages and then attempts to summarize them by selecting only the most relevant sentences. Expect to have your pages reviewed roughly once every two weeks. Keep in mind, though, that with meta tags have no meaning to Excite when it comes to rankings, even though it will use your description tags as long as the words are relevant to your pages’ content.
Affiliate Networks, Earning Reports, Social Networks, Sumit Money Tricks | | November 11, 2010 | 
Comments (0)
se marketing, search engine marketing, search engine optimization, SEO
