Welcome to WebmasterWorld Guest from 184.108.40.206
Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.
|Thread Subject||Messages||Started by||Last Message|
|robots.txt to stop the BAD bots
There was a past post with the best robots.txt file?
|11||Googly||11:18 pm Dec 12, 2002|
|3||junior||9:06 am Dec 5, 2002|
|3||darkroom||9:50 am Dec 3, 2002|
|robots.txt and mod_rewrite, meta robots, etc.
||4||Trisha||4:17 am Nov 30, 2002|
||7||jamesf4218||9:48 am Nov 25, 2002|
|Ask Jeeves robots.txt entry
what UA should be used
|3||Kerrin||4:40 pm Nov 7, 2002|
|logic check on robots.txt
will robots.txt properly disallowing a links page prevent link pop penalty?
|9||wooden||6:47 pm Nov 5, 2002|
|Which robots to exclude
||8||biggles||1:56 am Nov 4, 2002|
|robots.txt validator bug
The robots.txt validator accepts malformed files (missing User-agent).
|4||mortenf||11:02 pm Nov 3, 2002|
|What is a Robots.txt?
||4||Noximus||3:48 pm Oct 28, 2002|
still a bit unclear
|3||flexsez||7:43 pm Oct 23, 2002|
|Confused about robots.txt
need a little help
|2||Yamagotchi||3:19 pm Oct 21, 2002|
|robots.txt file question?
|3||printlist||11:46 am Oct 9, 2002|
||25||hollyhats||3:17 pm Oct 5, 2002|
|Beginners guide to robots
|6||Monkscuba||1:34 pm Sep 29, 2002|
|Robots.txt and htaccess
Script to write these files, is there such a thing?
|4||mack||12:55 am Sep 26, 2002|
how to use robots txt
|4||jilla||4:27 pm Sep 22, 2002|
|302 code and robots.txt
will google spider check the robots.txt when the server code is 302?
|2||coco||11:39 am Sep 10, 2002|
|Google WAP Proxy and robots.txt
||14||carfac||8:10 pm Sep 7, 2002|
Where is it?
|3||Nick_W||6:21 am Sep 7, 2002|
|A question on robots.txt files
~/ghealton/tmp/ or /tmp/ in /~ghealton/robots.txt
|3||ghealton||4:05 pm Aug 30, 2002|
which engines IGNORE it?
|6||charon||1:53 pm Aug 19, 2002|
|2||hurlimann||6:59 pm Aug 18, 2002|
nervous beginner seeks reassurance
|5||jgar||4:51 pm Aug 12, 2002|
|google and robots.txt - do I need one?
||5||cheater_copperpot||5:45 am Aug 7, 2002|