Welcome to WebmasterWorld Guest from 22.214.171.124
Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.
|Thread Subject||Messages||Started by||Last Message|
|How to Validate robots.txt and do I even have one?
||13||neophitemoneymaker||11:58 pm May 1, 2005|
|Page indexed in Google despite robots.txt entry
How did they find it in the first place?
|23||pmkpmk||9:17 pm May 1, 2005|
|The Great SearchEngineWorld robots.txt Survey #2 is Under Way
||9||Brett_Tabke||2:35 pm Apr 29, 2005|
|No-Follow, allow indexing of a file but not it's links?
||2||JAB_Creations||8:32 am Apr 27, 2005|
|How to stop Yahoo Slurp?
Slurp ingnores my robots.txt
|16||atlrus||9:47 pm Apr 26, 2005|
|How to remove ezboard forum from Wayback?
||3||wmw99||12:04 pm Apr 26, 2005|
|Is there a definitive robots.txt to disallow bad bots
bad bots and bad bots only
|6||Clark||11:40 am Apr 24, 2005|
|Disallow keyword Ignored?
Who guarantees me that disallow keyword isn't ignored
|5||bkausbk||9:02 am Apr 24, 2005|
|Allowing and disallowing spiders
Easy one - can somebody confirm my understanding?
|3||Zeberdy||9:19 am Apr 21, 2005|
|How to disallow getlastpost
||2||annelisabeth||9:19 am Apr 21, 2005|
|Robots.txt Help request.
I need help with stopping spyder robots.
|10||westmore||6:22 am Apr 21, 2005|
How many can I have?
|8||Andromeda||8:07 pm Apr 20, 2005|
I need a confirmation for robots.txt from you experts
|11||RaniaD||11:51 am Apr 20, 2005|
|I want to disallow extra for Slurp
is this syntax right?
|2||baze22||12:55 pm Apr 19, 2005|
|Robots.txt vs Robots META tag
Does one over-rule the other?
|3||obsos||7:14 am Apr 18, 2005|
|Validation of robots.txt
||3||designei||10:50 am Apr 17, 2005|
|Help w/ robots.txt Google won't toch it
|2||JackieB||11:04 am Apr 16, 2005|
|What was wrong with this robots.txt if anything
||4||montenegro||9:25 am Apr 16, 2005|
|block cgi bin on a virtual host
cgi bin is not in root directory
|3||Reid||6:26 am Apr 16, 2005|
|Block specific words from being indexed
||13||Jer1024||7:34 pm Apr 15, 2005|
|Allowing robots to index secure directories in root
||13||naomi||2:46 pm Apr 14, 2005|
|Block certain links
||2||liviu||1:13 pm Apr 14, 2005|
Do I NEED robots.txt?
|3||TheRookie||11:47 pm Apr 12, 2005|
Where does it go?
|1||mark1||10:55 pm Apr 10, 2005|
is googlebot coming and going
|4||Steve_w||9:00 pm Apr 10, 2005|