Welcome to WebmasterWorld Guest from 107.20.75.63

Forum Moderated by: goodroi

Sitemaps, Meta Data, and robots.txt


Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
Thread SubjectMessagesStarted byLast Message
robots.txt validator bug
The robots.txt validator accepts malformed files (missing User-agent).
4 mortenf 11:02 pm Nov 3, 2002
What is a Robots.txt?
4 Noximus 3:48 pm Oct 28, 2002
Spider Blocking
still a bit unclear
3 flexsez 7:43 pm Oct 23, 2002
Confused about robots.txt
need a little help
2 Yamagotchi 3:19 pm Oct 21, 2002
robots.txt file question?
What's this?
3 printlist 11:46 am Oct 9, 2002
Robots code
25 hollyhats 3:17 pm Oct 5, 2002
Beginners guide to robots
Help needed!
6 Monkscuba 1:34 pm Sep 29, 2002
Robots.txt and htaccess
Script to write these files, is there such a thing?
4 mack 12:55 am Sep 26, 2002
robots.txt
how to use robots txt
4 jilla 4:27 pm Sep 22, 2002
302 code and robots.txt
will google spider check the robots.txt when the server code is 302?
2 coco 11:39 am Sep 10, 2002
Google WAP Proxy and robots.txt
14 carfac 8:10 pm Sep 7, 2002
Yahoo's robots.txt
Where is it?
3 Nick_W 6:21 am Sep 7, 2002
A question on robots.txt files
~/ghealton/tmp/ or /tmp/ in /~ghealton/robots.txt
3 ghealton 4:05 pm Aug 30, 2002
Robots.txt question
which engines IGNORE it?
6 charon 1:53 pm Aug 19, 2002
robots.txt
Just checking
2 hurlimann 6:59 pm Aug 18, 2002
robots.txt
nervous beginner seeks reassurance
5 jgar 4:51 pm Aug 12, 2002
google and robots.txt - do I need one?
5 cheater_copperpot 5:45 am Aug 7, 2002
Robots.txt for mirror sites
Anyone have a good one?
7 ScottM 10:38 pm Aug 6, 2002
Wildcard blocking of dynamic - robots.txt
To: GoogleGuy
5 Lisa 6:13 am Jul 31, 2002
Robots.txt exclude file types
2 Lisa 7:03 am Jul 29, 2002
Robots.txt
5 webboy1 12:04 pm Jul 22, 2002
robots.txt
How to make a valid positive list?
5 matthias 6:20 am Jul 9, 2002
Google - robots.txt - indexing
9 Brett_Tabke 9:40 am Jun 24, 2002
Do I need a robots.txt file for Googlebot to read?
Will I get indexed by Google if I don't have a robots.txt?
27 johnd 10:03 pm Jun 18, 2002
Quick tip
Check out your robots.txt[2] ( 1 2 )
31 GoogleGuy 12:34 pm Jun 18, 2002

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members