Welcome to WebmasterWorld Guest from 54.144.79.200

Forum Moderated by: goodroi

Sitemaps, Meta Data, and robots.txt


Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
Thread SubjectMessagesStarted byLast Message
Bots ignoring my robots.txt
In place for a week, validates, but no use
12 roddy 7:10 am Nov 5, 2003
Blocking a spider from a directory
3 JVB_Mktg 4:30 am Nov 5, 2003
Robots.txt
To use... or not to use? That is the question
6 panic 1:29 am Nov 5, 2003
Robots.txt Validator
We're sorry, this robots.txt does NOT validate.
2 midge 12:48 pm Nov 3, 2003
Robot.txt - What is it, and why use it
4 laffe 4:36 am Nov 2, 2003
robots.txt syntex
How to block a directory.
6 jdancing 7:52 pm Oct 31, 2003
Robots.txt during development
Anyone advise on this?
2 stuntdubl 3:30 am Oct 30, 2003
bad Bots
3 cgj5204 4:40 am Oct 29, 2003
Restrict Specific Domain Names from Being Indexed
Using the robots.txt file on Google to restrict certain domains...
3 jmh1970 1:36 am Oct 29, 2003
New Forum : Robots.txt
1 Brett_Tabke 9:24 pm Oct 28, 2003
Put your robots.txt on a diet
Three easy ways to reduce bloat in robots.txt
20 jdMorgan 8:36 pm Oct 28, 2003
Blocking image file indexing without robots.txt
Can this be done?
2 Winedark 8:08 pm Oct 28, 2003
Robots.txt size
How long can a robots.txt be?
4 WyrdoManx 9:47 pm Oct 22, 2003
Robots.txt - Am I Missing Somthing?
Query strings and general syntax is suspected...
13 Nick_W 3:40 pm Oct 21, 2003
How to correctly set robots.txt for a subdomain?
please help
5 PatrickDeese 6:36 pm Oct 15, 2003
Robots.txt for Query Strings
Can I stop bots from following links with query stings
10 Nick_W 6:08 pm Oct 15, 2003
DMOZ Robots.txt
Crawling no longer allowed
12 Yidaki 5:38 pm Oct 14, 2003
Robots.txt Question :)
Question regarding how to use robots txt
8 Prana 9:33 pm Sep 27, 2003
robots.txt
3 domains in root
9 Alternative_Future 11:33 pm Sep 23, 2003
Robots.txt disallow: /index.php?
then /index.php?param=example still allowed?
5 Yidaki 12:17 pm Sep 21, 2003
Robots.txt exclusion question
Robots.txt exclude all but one directory?
4 olof_bailey 8:52 pm Sep 20, 2003
robots.txt
Where does it go?
4 zoobie 5:58 pm Sep 19, 2003
404 robots.txt and traffic going down
8 silverbytes 4:29 pm Sep 14, 2003
Robots.txt
4 johnnstacy1 8:19 am Sep 1, 2003
More than one Robots.txt per server?
18 apphoto 6:05 pm Aug 30, 2003

Featured Threads

Hot Threads This Week

Hot Threads This Month