Welcome to WebmasterWorld Guest from 54.144.231.243

Forum Moderated by: goodroi

Sitemaps, Meta Data, and robots.txt


Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
Thread SubjectMessagesStarted byLast Message
Which robots to exclude
8 biggles 1:56 am Nov 4, 2002
robots.txt validator bug
The robots.txt validator accepts malformed files (missing User-agent).
4 mortenf 11:02 pm Nov 3, 2002
What is a Robots.txt?
4 Noximus 3:48 pm Oct 28, 2002
Spider Blocking
still a bit unclear
3 flexsez 7:43 pm Oct 23, 2002
Confused about robots.txt
need a little help
2 Yamagotchi 3:19 pm Oct 21, 2002
robots.txt file question?
What's this?
3 printlist 11:46 am Oct 9, 2002
Robots code
25 hollyhats 3:17 pm Oct 5, 2002
Beginners guide to robots
Help needed!
6 Monkscuba 1:34 pm Sep 29, 2002
Robots.txt and htaccess
Script to write these files, is there such a thing?
4 mack 12:55 am Sep 26, 2002
robots.txt
how to use robots txt
4 jilla 4:27 pm Sep 22, 2002
302 code and robots.txt
will google spider check the robots.txt when the server code is 302?
2 coco 11:39 am Sep 10, 2002
Google WAP Proxy and robots.txt
14 carfac 8:10 pm Sep 7, 2002
Yahoo's robots.txt
Where is it?
3 Nick_W 6:21 am Sep 7, 2002
A question on robots.txt files
~/ghealton/tmp/ or /tmp/ in /~ghealton/robots.txt
3 ghealton 4:05 pm Aug 30, 2002
Robots.txt question
which engines IGNORE it?
6 charon 1:53 pm Aug 19, 2002
robots.txt
Just checking
2 hurlimann 6:59 pm Aug 18, 2002
robots.txt
nervous beginner seeks reassurance
5 jgar 4:51 pm Aug 12, 2002
google and robots.txt - do I need one?
5 cheater_copperpot 5:45 am Aug 7, 2002
Robots.txt for mirror sites
Anyone have a good one?
7 ScottM 10:38 pm Aug 6, 2002
Wildcard blocking of dynamic - robots.txt
To: GoogleGuy
5 Lisa 6:13 am Jul 31, 2002
Robots.txt exclude file types
2 Lisa 7:03 am Jul 29, 2002
Robots.txt
5 webboy1 12:04 pm Jul 22, 2002
robots.txt
How to make a valid positive list?
5 matthias 6:20 am Jul 9, 2002
Google - robots.txt - indexing
9 Brett_Tabke 9:40 am Jun 24, 2002
Do I need a robots.txt file for Googlebot to read?
Will I get indexed by Google if I don't have a robots.txt?
27 johnd 10:03 pm Jun 18, 2002

Featured Threads

Hot Threads This Week

Hot Threads This Month