Welcome to WebmasterWorld Guest from 54.198.143.223

Forum Moderated by: goodroi

Sitemaps, Meta Data, and robots.txt


Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
Thread SubjectMessagesStarted byLast Message
Robots.txt Validator bug?
Christian Storm from Turnitin Robot maintainers says my robots.txt is wrong
3 scottspence 11:15 am Dec 17, 2002
How do I create a robot.txt?
9 palmpal 1:35 pm Dec 15, 2002
robots.txt to stop the BAD bots
There was a past post with the best robots.txt file?
11 Googly 11:18 pm Dec 12, 2002
Robots.txt file
disallow target_blank
3 junior 9:06 am Dec 5, 2002
robots.txt
robots.txt
3 darkroom 9:50 am Dec 3, 2002
robots.txt and mod_rewrite, meta robots, etc.
4 Trisha 4:17 am Nov 30, 2002
robot.txt syntax
7 jamesf4218 9:48 am Nov 25, 2002
Ask Jeeves robots.txt entry
what UA should be used
3 Kerrin 4:40 pm Nov 7, 2002
logic check on robots.txt
will robots.txt properly disallowing a links page prevent link pop penalty?
9 wooden 6:47 pm Nov 5, 2002
Which robots to exclude
8 biggles 1:56 am Nov 4, 2002
robots.txt validator bug
The robots.txt validator accepts malformed files (missing User-agent).
4 mortenf 11:02 pm Nov 3, 2002
What is a Robots.txt?
4 Noximus 3:48 pm Oct 28, 2002
Spider Blocking
still a bit unclear
3 flexsez 7:43 pm Oct 23, 2002
Confused about robots.txt
need a little help
2 Yamagotchi 3:19 pm Oct 21, 2002
robots.txt file question?
What's this?
3 printlist 11:46 am Oct 9, 2002
Robots code
25 hollyhats 3:17 pm Oct 5, 2002
Beginners guide to robots
Help needed!
6 Monkscuba 1:34 pm Sep 29, 2002
Robots.txt and htaccess
Script to write these files, is there such a thing?
4 mack 12:55 am Sep 26, 2002
robots.txt
how to use robots txt
4 jilla 4:27 pm Sep 22, 2002
302 code and robots.txt
will google spider check the robots.txt when the server code is 302?
2 coco 11:39 am Sep 10, 2002
Google WAP Proxy and robots.txt
14 carfac 8:10 pm Sep 7, 2002
Yahoo's robots.txt
Where is it?
3 Nick_W 6:21 am Sep 7, 2002
A question on robots.txt files
~/ghealton/tmp/ or /tmp/ in /~ghealton/robots.txt
3 ghealton 4:05 pm Aug 30, 2002
Robots.txt question
which engines IGNORE it?
6 charon 1:53 pm Aug 19, 2002
robots.txt
Just checking
2 hurlimann 6:59 pm Aug 18, 2002

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members