Category: Robots Txt

August 18, 2008
pic4

The Robots.txt File

The robots.txt file is used to control web spiders and other web robots on how they crawl your site. More often than not, this file is […]
Top