All I have in my robots.txt file is the bit below - is it enough?
Dougie.
User-Agent: * Disallow:
trillianjedi
11:07 am on Feb 2, 2004 (gmt 0)
is it enough?
What is it you want to do?
The robots.txt you have does nothing.
TJ
dougie
11:17 am on Feb 2, 2004 (gmt 0)
All I want is for every robot for every SE and every directory to visit my website.
trillianjedi
11:19 am on Feb 2, 2004 (gmt 0)
All I want is for every robot for every SE and every directory to visit my website.
You can safely just remove your robots.txt file completely in that case.
It's only used for listing search engine spiders that you do *not* want indexing your site.
TJ
sem4u
11:20 am on Feb 2, 2004 (gmt 0)
Well all the robots.txt file will do is to prevent certain spiders from visiting your website. At the moment you do allow all, but if you want to be in a directories you will have to submit to them :)