| Welcome to WebmasterWorld Guest from 18.104.22.168 |
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
|Pubcon Platinum Sponsor|
|are the robots.txt examples still valid from WebmasterWorld?|
| 2:59 pm on Jan 28, 2005 (gmt 0)|
I was wondering if the robotx.txt example provided by WebMasterWorld is still valid to use.
Are the "nice guy/bad guy" listings all set? Is it wise to use this example:
I was also wondering why this is the last portion of the robots.txt:
Isn't that saying to block ALL ROBOTS? What about google, msn, yahoo, teoma, and the linking?
Any info will be great.
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
© Webmaster World 1996-2013 all rights reserved