Welcome to WebmasterWorld Guest from 54.226.179.247

Forum Moderated by: goodroi

Sitemaps, Meta Data, and robots.txt


Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
Thread SubjectMessagesStarted byLast Message
How Can I Make Bots Visit My Web Site?
3 emadido 2:34 pm Mar 15, 2007
why i can't reaply the topic
why i can't reaply the topic
3 unknown 8:43 am Mar 14, 2007
robotsxx.txt not found.
2 benevolent001 1:16 pm Mar 13, 2007
Something a little different
2 futureX 1:56 pm Mar 12, 2007
Why is WebmasterWorld robot validator giving back HTML
3 Clair 7:11 pm Mar 10, 2007
When Robots.txt And Meta Robots Collide
What happens?
13 Easy_Coder 1:57 am Mar 10, 2007
OSC, dupe content, robots.txt and 301 redirects
Need to ensure bots 301 to new SEO URL
5 spina45 8:23 pm Mar 9, 2007
Disallow / access to our directory
disallow
2 drew68 3:47 pm Mar 6, 2007
Block the main page of a directory but not its contents?
6 bresso 7:39 am Mar 6, 2007
Robots.txt and pointing domains
3 Shanada 6:58 am Feb 28, 2007
Will Googlebot be able to crawl these pages of my website?
4 saqitude 3:40 pm Feb 25, 2007
robots.txt question
2 seomack 8:14 am Feb 19, 2007
Is correct this robots.txt?
2 MalastiC 9:44 pm Feb 18, 2007
Sitemap and robot.txt
3 spuddy 3:53 pm Feb 15, 2007
How to exclude from crawling subdomain
exclude the crawling subdomain
4 fendy 1:00 pm Feb 15, 2007
Confused over Google spider and robots.txt
4 flowermark 12:56 pm Feb 15, 2007
Which bots to block?
robot.txt
2 cyclesolutions 9:51 pm Feb 12, 2007
Accidentally blocked root access, doh!
3 ninjaNeen 6:51 pm Feb 8, 2007
robots.txt usage and examples
Using robots.txt to improve SEO
3 cduke250 3:50 pm Feb 7, 2007
disallowing subdomain access rule
1 coburn 1:37 pm Feb 7, 2007
can put two robots at one line at robot.txt to ban two spiders?
robots.txt, ban google, yahoo bots
4 johnlim9988 3:54 am Feb 7, 2007
Blocking subdir, but allowing crawl in deeper dir?
7 Vulcan315 3:52 pm Feb 6, 2007
robots noindex on an iframe page
What would this do?
2 dataguy 1:20 pm Feb 3, 2007
Slowing down MSN bot?
Doubling it's bandwidth usage each month.
4 bouncybunny 3:33 am Feb 3, 2007
Slurps and wildcards
5 Vimes 1:36 am Feb 3, 2007

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members