Welcome to WebmasterWorld Guest from 54.166.152.121

Forum Moderators: goodroi

Message Too Old, No Replies

are the robots.txt examples still valid from WebmasterWorld?

     
2:59 pm on Jan 28, 2005 (gmt 0)

10+ Year Member



I was wondering if the robotx.txt example provided by WebMasterWorld is still valid to use.

Are the "nice guy/bad guy" listings all set? Is it wise to use this example:
[searchengineworld.com...]

I was also wondering why this is the last portion of the robots.txt:
User-agent: *
Disallow: /

Isn't that saying to block ALL ROBOTS? What about google, msn, yahoo, teoma, and the linking?
Any info will be great.
Thanks

 

Featured Threads

Hot Threads This Week

Hot Threads This Month