Welcome to WebmasterWorld Guest from 184.108.40.206
Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.
|Thread Subject||Messages||Started by||Last Message|
|Which robots to exclude
||8||biggles||1:56 am Nov 4, 2002|
|robots.txt validator bug
The robots.txt validator accepts malformed files (missing User-agent).
|4||mortenf||11:02 pm Nov 3, 2002|
|What is a Robots.txt?
||4||Noximus||3:48 pm Oct 28, 2002|
still a bit unclear
|3||flexsez||7:43 pm Oct 23, 2002|
|Confused about robots.txt
need a little help
|2||Yamagotchi||3:19 pm Oct 21, 2002|
|robots.txt file question?
|3||printlist||11:46 am Oct 9, 2002|
||25||hollyhats||3:17 pm Oct 5, 2002|
|Beginners guide to robots
|6||Monkscuba||1:34 pm Sep 29, 2002|
|Robots.txt and htaccess
Script to write these files, is there such a thing?
|4||mack||12:55 am Sep 26, 2002|
how to use robots txt
|4||jilla||4:27 pm Sep 22, 2002|
|302 code and robots.txt
will google spider check the robots.txt when the server code is 302?
|2||coco||11:39 am Sep 10, 2002|
|Google WAP Proxy and robots.txt
||14||carfac||8:10 pm Sep 7, 2002|
Where is it?
|3||Nick_W||6:21 am Sep 7, 2002|
|A question on robots.txt files
~/ghealton/tmp/ or /tmp/ in /~ghealton/robots.txt
|3||ghealton||4:05 pm Aug 30, 2002|
which engines IGNORE it?
|6||charon||1:53 pm Aug 19, 2002|
|2||hurlimann||6:59 pm Aug 18, 2002|
nervous beginner seeks reassurance
|5||jgar||4:51 pm Aug 12, 2002|
|google and robots.txt - do I need one?
||5||cheater_copperpot||5:45 am Aug 7, 2002|
|Robots.txt for mirror sites
Anyone have a good one?
|7||ScottM||10:38 pm Aug 6, 2002|
|Wildcard blocking of dynamic - robots.txt
|5||Lisa||6:13 am Jul 31, 2002|
|Robots.txt exclude file types
||2||Lisa||7:03 am Jul 29, 2002|
||5||webboy1||12:04 pm Jul 22, 2002|
How to make a valid positive list?
|5||matthias||6:20 am Jul 9, 2002|
|Google - robots.txt - indexing
||9||Brett_Tabke||9:40 am Jun 24, 2002|
|Do I need a robots.txt file for Googlebot to read?
Will I get indexed by Google if I don't have a robots.txt?
|27||johnd||10:03 pm Jun 18, 2002|