| Welcome to WebmasterWorld Guest from 126.96.36.199 |
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
|Become a Pro Member|
|Is this enough|
| 9:57 am on Feb 2, 2004 (gmt 0)|
All I have in my robots.txt file is the bit below - is it enough?
| 11:07 am on Feb 2, 2004 (gmt 0)|
What is it you want to do?
The robots.txt you have does nothing.
| 11:17 am on Feb 2, 2004 (gmt 0)|
All I want is for every robot for every SE and every directory to visit my website.
| 11:19 am on Feb 2, 2004 (gmt 0)|
|All I want is for every robot for every SE and every directory to visit my website. |
You can safely just remove your robots.txt file completely in that case.
It's only used for listing search engine spiders that you do *not* want indexing your site.
| 11:20 am on Feb 2, 2004 (gmt 0)|
Well all the robots.txt file will do is to prevent certain spiders from visiting your website. At the moment you do allow all, but if you want to be in a directories you will have to submit to them :)
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved