Welcome to WebmasterWorld Guest from 54.144.243.34

Forum Moderated by: goodroi

Sitemaps, Meta Data, and robots.txt


Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
Thread SubjectMessagesStarted byLast Message
Do search engines cache robots.txt?
10 trillianjedi 11:18 pm Apr 16, 2003
Site hacked and robots.txt placed, taken off google.
robots.txt, hacked, taken off google
3 Mohammed 10:43 am Apr 15, 2003
robots.txt
Disallow other URLs for one site.
3 spinweb 6:09 pm Apr 11, 2003
Question about simple robots.txt file
13 nadsab 12:56 pm Apr 10, 2003
Robots.txt
Is this valid?
5 Alternative_Future 9:52 am Apr 3, 2003
Get a Robot
Get the robot file
7 Glenn 2:45 pm Apr 2, 2003
robots.txt question
2 deltron23 9:01 pm Mar 28, 2003
External Javascript and bots and robots.txt?
2 JeremyL 6:07 pm Mar 12, 2003
Robot.txt and Templates
The <meta>Robot tag in templates, with specific disallow in robot.txt
5 juniperwasting 4:12 pm Mar 11, 2003
My Robots.txt Screw Up.
case study in how to kill well performing serps in 1 easy step
5 webdevsf 1:45 am Mar 8, 2003
How do I apply the robots.txt?
Seems important and need to know more!
3 askjoe 6:24 pm Mar 7, 2003
I don't want to be listed yet
Is disabling anonomas access enough?
8 currington 9:53 am Mar 5, 2003
empty robots.txt
yes or no
6 HuhuFruFru 8:32 am Mar 5, 2003
question about an empty robots.txt
5 HuhuFruFru 8:36 pm Mar 4, 2003
logs and robots.txt
7 tyrojds 7:42 pm Mar 4, 2003
Why robots.txt if you want everything indexed?
Does it even matter?
9 Larry_C 3:49 pm Mar 3, 2003
Robots.txt question
excluding every folder expect one
5 JVB_Mktg 7:54 pm Mar 2, 2003
Logs show a code 206 with robots.txt
code 206 - partial content
2 nativenewyorker 11:03 pm Mar 1, 2003
robots.txt syntax
spaces or slashes?
3 keyplyr 5:14 pm Feb 27, 2003
Strange Incident with robots.txt
error code 401
3 neh2008 4:57 pm Feb 17, 2003
robots.txt to block bots access to images
got to save bandwidth
4 stavs 3:00 pm Feb 14, 2003
ROBOTS.TXT Location
ROBOTS.TXT Location
4 neh2008 5:27 pm Feb 11, 2003
Robots.txt
I better get this right...
6 JVB_Mktg 4:19 am Feb 11, 2003
Robots.txt
How? Sub-Domains
2 MarkHutch 8:37 pm Feb 6, 2003
frames and robots.txt
4 snowfishin 9:15 pm Feb 3, 2003

Featured Threads

My Threads

Hot Threads This Week

Hot Threads This Month