SEO Glossary
Search Engine Spiders
A search engine spider (also known as a web crawler, web spider or web robot) is a program or automated script which browses the Internet in a methodical, automated manner.
This process is called web crawling or spidering. Many sites especially search engines such as Google and Yahoo use spidering as a means of providing up-to-date data.
When a website has been updated it requires the search engine spiders to read the changes before they can be incorporated within the search engine results. This can take anywhere from a matter of hours to a number of week.
This is why good internal linking is required within a website. If links to all pages within a site are not accessible for search engine spiders then they will subsequentially not be able to access and index the pages. Thus resulting in them not being incorporated within the search engine database.
Search Engine Spiders are also known as ants, automatic indexers, bots, and worms. However, these terms are not commonly used within the SEO industry.
Terms related to Search Engine Spiders include:
For more information on Search Engine Spiders please contact us
