Welcome to WebmasterWorld Guest from 54.144.68.27

Forum Moderated by: goodroi

Sitemaps, Meta Data, and robots.txt


Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
Thread SubjectMessagesStarted byLast Message
Getting ERROR_DISALLOW with Inktomi after adding new robots.txt
7 bebox 1:41 pm Dec 29, 2002
Webmasterworld's Robots.txt
A couple of questiosn for Brett
4 victor 5:00 pm Dec 27, 2002
Robots.txt Validator bug?
Christian Storm from Turnitin Robot maintainers says my robots.txt is wrong
3 scottspence 11:15 am Dec 17, 2002
How do I create a robot.txt?
9 palmpal 1:35 pm Dec 15, 2002
robots.txt to stop the BAD bots
There was a past post with the best robots.txt file?
11 Googly 11:18 pm Dec 12, 2002
Robots.txt file
disallow target_blank
3 junior 9:06 am Dec 5, 2002
robots.txt
robots.txt
3 darkroom 9:50 am Dec 3, 2002
robots.txt and mod_rewrite, meta robots, etc.
4 Trisha 4:17 am Nov 30, 2002
robot.txt syntax
7 jamesf4218 9:48 am Nov 25, 2002
Ask Jeeves robots.txt entry
what UA should be used
3 Kerrin 4:40 pm Nov 7, 2002
logic check on robots.txt
will robots.txt properly disallowing a links page prevent link pop penalty?
9 wooden 6:47 pm Nov 5, 2002
Which robots to exclude
8 biggles 1:56 am Nov 4, 2002
robots.txt validator bug
The robots.txt validator accepts malformed files (missing User-agent).
4 mortenf 11:02 pm Nov 3, 2002
What is a Robots.txt?
4 Noximus 3:48 pm Oct 28, 2002
Spider Blocking
still a bit unclear
3 flexsez 7:43 pm Oct 23, 2002
Confused about robots.txt
need a little help
2 Yamagotchi 3:19 pm Oct 21, 2002
robots.txt file question?
What's this?
3 printlist 11:46 am Oct 9, 2002
Robots code
25 hollyhats 3:17 pm Oct 5, 2002
Beginners guide to robots
Help needed!
6 Monkscuba 1:34 pm Sep 29, 2002
Robots.txt and htaccess
Script to write these files, is there such a thing?
4 mack 12:55 am Sep 26, 2002
robots.txt
how to use robots txt
4 jilla 4:27 pm Sep 22, 2002
302 code and robots.txt
will google spider check the robots.txt when the server code is 302?
2 coco 11:39 am Sep 10, 2002
Google WAP Proxy and robots.txt
14 carfac 8:10 pm Sep 7, 2002
Yahoo's robots.txt
Where is it?
3 Nick_W 6:21 am Sep 7, 2002
A question on robots.txt files
~/ghealton/tmp/ or /tmp/ in /~ghealton/robots.txt
3 ghealton 4:05 pm Aug 30, 2002

Featured Threads

Hot Threads This Week

Hot Threads This Month