| Welcome to WebmasterWorld Guest from 126.96.36.199 |
register, login, search, subscribe, help, library, PubCon, announcements, recent posts, open posts,
|Really necessary to add to robots.txt|
| 9:06 pm on Jul 4, 2003 (gmt 0)|
Is it really necessary to add the two lines to robots.txt like the AdSense FAQ says? As long as you are not explicitly banning their bot, why is this necessary?
| 9:54 pm on Jul 4, 2003 (gmt 0)|
It's not necessary. If you're not banning their bots (or all bots) you don't need to worry with adding anything to robots.txt.
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld ® and PubCon ® are a Registered Trademarks of Pubcon Inc.
© Pubcon Inc. 1996-2012 all rights reserved