| Welcome to WebmasterWorld Guest from 184.108.40.206 |
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
|Pubcon Platinum Sponsor 2014|
|Do i need the txt file?|
if i want to allow everyone
| 2:13 am on Jul 2, 2004 (gmt 0)|
If i want to allow all spiders to my site, do i really need the robots.txt file and robot meta-tags on my site?
| 2:16 am on Jul 2, 2004 (gmt 0)|
No you don't but you might want to put a blank file there just so you don't get so many 404's
| 12:55 am on Jul 5, 2004 (gmt 0)|
and ogletree that blank file has to be named "robot.txt".
| 12:59 am on Jul 5, 2004 (gmt 0)|
|that blank file has to be named "robot.txt" |
A common mistake (or perhaps just a typo). the filename should be robots.txt, and it must be placed in the document root directory.
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved