Welcome to WebmasterWorld Guest from 50.17.74.162

Forum Moderated by: goodroi

Sitemaps, Meta Data, and robots.txt


Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
Thread SubjectMessagesStarted byLast Message
Blocking Google
I need to block google from my index page
4 loner 4:25 pm Apr 2, 2007
help needed
1 hugh 4:03 pm Apr 2, 2007
What's the Proper Way to Block Sub Folders
Looking for the best way to block 2nd level sub folders
5 graywolf 7:43 pm Mar 28, 2007
ROBOT and PR rank?
how does the robot effect your pr rank?
7 moppy56 1:48 pm Mar 28, 2007
Blocking Partial Contents of Folder Without Using Robots Tag
10 files (7 want indexed, 3 want blocked)
5 SandySEO 8:49 pm Mar 27, 2007
Another bad bot to ban
this one is using 302 redirects .. bad bad
1 Bewenched 3:43 am Mar 25, 2007
excluding a subdomain
Using the robots.txt file to exclude a subdomain
10 joebray 9:47 am Mar 23, 2007
G. ignores robots.txt at other ports, gives false page count
Google shows in its index pages that it never saw, even prohibited
1 OnetSzukaj 2:58 pm Mar 22, 2007
How to use Robots.txt as an INCLUSION method
It's exclusion only if you use it that way
1 incrediBILL 12:32 am Mar 22, 2007
TripAdvisorBot
Anyone seen this?
3 bobbarnes 10:28 pm Mar 16, 2007
problem with msnbot and wildcard matching
2 Achernar 1:36 pm Mar 16, 2007
How Can I Make Bots Visit My Web Site?
3 emadido 2:34 pm Mar 15, 2007
why i can't reaply the topic
why i can't reaply the topic
3 unknown 8:43 am Mar 14, 2007
robotsxx.txt not found.
2 benevolent001 1:16 pm Mar 13, 2007
Something a little different
2 futureX 1:56 pm Mar 12, 2007
Why is WebmasterWorld robot validator giving back HTML
3 Clair 7:11 pm Mar 10, 2007
When Robots.txt And Meta Robots Collide
What happens?
13 Easy_Coder 1:57 am Mar 10, 2007
OSC, dupe content, robots.txt and 301 redirects
Need to ensure bots 301 to new SEO URL
5 spina45 8:23 pm Mar 9, 2007
Disallow / access to our directory
disallow
2 drew68 3:47 pm Mar 6, 2007
Block the main page of a directory but not its contents?
6 bresso 7:39 am Mar 6, 2007
Robots.txt and pointing domains
3 Shanada 6:58 am Feb 28, 2007
Will Googlebot be able to crawl these pages of my website?
4 saqitude 3:40 pm Feb 25, 2007
robots.txt question
2 seomack 8:14 am Feb 19, 2007
Is correct this robots.txt?
2 MalastiC 9:44 pm Feb 18, 2007
Sitemap and robot.txt
3 spuddy 3:53 pm Feb 15, 2007