Welcome to WebmasterWorld Guest from 184.108.40.206
Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.
|Thread Subject||Messages||Started by||Last Message|
|what is robots.txt file?
||3||antipodes||5:57 am Jan 4, 2003|
|Getting ERROR_DISALLOW with Inktomi after adding new robots.txt
||7||bebox||1:41 pm Dec 29, 2002|
A couple of questiosn for Brett
|4||victor||5:00 pm Dec 27, 2002|
|Robots.txt Validator bug?
Christian Storm from Turnitin Robot maintainers says my robots.txt is wrong
|3||scottspence||11:15 am Dec 17, 2002|
|How do I create a robot.txt?
||9||palmpal||1:35 pm Dec 15, 2002|
|robots.txt to stop the BAD bots
There was a past post with the best robots.txt file?
|11||Googly||11:18 pm Dec 12, 2002|
|3||junior||9:06 am Dec 5, 2002|
|3||darkroom||9:50 am Dec 3, 2002|
|robots.txt and mod_rewrite, meta robots, etc.
||4||Trisha||4:17 am Nov 30, 2002|
||7||jamesf4218||9:48 am Nov 25, 2002|
|Ask Jeeves robots.txt entry
what UA should be used
|3||Kerrin||4:40 pm Nov 7, 2002|
|logic check on robots.txt
will robots.txt properly disallowing a links page prevent link pop penalty?
|9||wooden||6:47 pm Nov 5, 2002|
|Which robots to exclude
||8||biggles||1:56 am Nov 4, 2002|
|robots.txt validator bug
The robots.txt validator accepts malformed files (missing User-agent).
|4||mortenf||11:02 pm Nov 3, 2002|
|What is a Robots.txt?
||4||Noximus||3:48 pm Oct 28, 2002|
still a bit unclear
|3||flexsez||7:43 pm Oct 23, 2002|
|Confused about robots.txt
need a little help
|2||Yamagotchi||3:19 pm Oct 21, 2002|
|robots.txt file question?
|3||printlist||11:46 am Oct 9, 2002|
||25||hollyhats||3:17 pm Oct 5, 2002|
|Beginners guide to robots
|6||Monkscuba||1:34 pm Sep 29, 2002|
|Robots.txt and htaccess
Script to write these files, is there such a thing?
|4||mack||12:55 am Sep 26, 2002|
how to use robots txt
|4||jilla||4:27 pm Sep 22, 2002|
|302 code and robots.txt
will google spider check the robots.txt when the server code is 302?
|2||coco||11:39 am Sep 10, 2002|
|Google WAP Proxy and robots.txt
||14||carfac||8:10 pm Sep 7, 2002|
Where is it?
|3||Nick_W||6:21 am Sep 7, 2002|