| Welcome to WebmasterWorld Guest from 220.127.116.11 |
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
|Become a Pro Member|
|I'm not understanding the different makes of bots|
Can I stop some, while allowing others?
| 9:42 pm on Jul 1, 2004 (gmt 0)|
Hi. Is there a way I can stop Google from crawling a site and still allow other bots (Yahoo, MSN) to crawl?
| 10:11 pm on Jul 1, 2004 (gmt 0)|
Sure thing - just add this to your robots.txt file in your document root:
That will stop Googlebot, but not the others.
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved