| Welcome to WebmasterWorld Guest from 188.8.131.52 |
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
|Subscribe to WebmasterWorld|
|Do i need the txt file?|
if i want to allow everyone
| 2:13 am on Jul 2, 2004 (gmt 0)|
If i want to allow all spiders to my site, do i really need the robots.txt file and robot meta-tags on my site?
| 2:16 am on Jul 2, 2004 (gmt 0)|
No you don't but you might want to put a blank file there just so you don't get so many 404's
| 12:55 am on Jul 5, 2004 (gmt 0)|
and ogletree that blank file has to be named "robot.txt".
| 12:59 am on Jul 5, 2004 (gmt 0)|
|that blank file has to be named "robot.txt" |
A common mistake (or perhaps just a typo). the filename should be robots.txt, and it must be placed in the document root directory.
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
© Webmaster World 1996-2014 all rights reserved