trillianjedi
msg:1525464 | 11:07 am on Feb 2, 2004 (gmt 0) |
What is it you want to do? The robots.txt you have does nothing. TJ
|
dougie
msg:1525465 | 11:17 am on Feb 2, 2004 (gmt 0) |
All I want is for every robot for every SE and every directory to visit my website.
|
trillianjedi
msg:1525466 | 11:19 am on Feb 2, 2004 (gmt 0) |
| All I want is for every robot for every SE and every directory to visit my website. |
|
You can safely just remove your robots.txt file completely in that case. It's only used for listing search engine spiders that you do *not* want indexing your site. TJ
|
sem4u
msg:1525467 | 11:20 am on Feb 2, 2004 (gmt 0) |
Well all the robots.txt file will do is to prevent certain spiders from visiting your website. At the moment you do allow all, but if you want to be in a directories you will have to submit to them :)
|
|
