homepage Welcome to WebmasterWorld Guest from 54.237.134.62
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconSomething a little different
 
2futureX 1:56 pm on Mar 12, 2007 (gmt 0)
newpost thread iconWhy is WebmasterWorld robot validator giving back HTML
 
3Clair 7:11 pm on Mar 10, 2007 (gmt 0)
newpost thread iconWhen Robots.txt And Meta Robots Collide
  What happens?
13Easy_Coder 1:57 am on Mar 10, 2007 (gmt 0)
newpost thread iconOSC, dupe content, robots.txt and 301 redirects
  Need to ensure bots 301 to new SEO URL
5spina45 8:23 pm on Mar 9, 2007 (gmt 0)
newpost thread iconDisallow / access to our directory
  disallow
2drew68 3:47 pm on Mar 6, 2007 (gmt 0)
newpost thread iconBlock the main page of a directory but not its contents?
 
6bresso 7:39 am on Mar 6, 2007 (gmt 0)
newpost thread iconRobots.txt and pointing domains
 
3Shanada 6:58 am on Feb 28, 2007 (gmt 0)
newpost thread iconWill Googlebot be able to crawl these pages of my website?
 
4saqitude 3:40 pm on Feb 25, 2007 (gmt 0)
newpost thread iconrobots.txt question
 
2seomack 8:14 am on Feb 19, 2007 (gmt 0)
newpost thread iconIs correct this robots.txt?
 
2MalastiC 9:44 pm on Feb 18, 2007 (gmt 0)
newpost thread iconSitemap and robot.txt
 
3spuddy 3:53 pm on Feb 15, 2007 (gmt 0)
newpost thread iconHow to exclude from crawling subdomain
  exclude the crawling subdomain
4fendy 1:00 pm on Feb 15, 2007 (gmt 0)
newpost thread iconConfused over Google spider and robots.txt
 
4flowermark 12:56 pm on Feb 15, 2007 (gmt 0)
newpost thread iconWhich bots to block?
  robot.txt
2cyclesolutions 9:51 pm on Feb 12, 2007 (gmt 0)
newpost thread iconAccidentally blocked root access, doh!
 
3ninjaNeen 6:51 pm on Feb 8, 2007 (gmt 0)
newpost thread iconrobots.txt usage and examples
  Using robots.txt to improve SEO
3cduke250 3:50 pm on Feb 7, 2007 (gmt 0)
newpost thread icondisallowing subdomain access rule
 
1coburn 1:37 pm on Feb 7, 2007 (gmt 0)
newpost thread iconcan put two robots at one line at robot.txt to ban two spiders?
  robots.txt, ban google, yahoo bots
4johnlim9988 3:54 am on Feb 7, 2007 (gmt 0)
newpost thread iconBlocking subdir, but allowing crawl in deeper dir?
 
7Vulcan315 3:52 pm on Feb 6, 2007 (gmt 0)
newpost thread iconrobots noindex on an iframe page
  What would this do?
2dataguy 1:20 pm on Feb 3, 2007 (gmt 0)
newpost thread iconSlowing down MSN bot?
  Doubling it's bandwidth usage each month.
4bouncybunny 3:33 am on Feb 3, 2007 (gmt 0)
newpost thread iconSlurps and wildcards
 
5Vimes 1:36 am on Feb 3, 2007 (gmt 0)
newpost thread iconrobots.txt blocking my index page?
  Hello
4hola2 2:08 pm on Feb 2, 2007 (gmt 0)
newpost thread iconRobots.txt command? "do not search anything outside this."
 
5OWizeOne 1:08 pm on Feb 1, 2007 (gmt 0)
newpost thread iconHow do I block yahoo's SE spider and allow their image spider?
  Does Yahoo send a different spider for the image search?
4Clark 5:18 pm on Jan 27, 2007 (gmt 0)
Displaying Page 30 of 71<< Back 1 ... 20 21 22 23 24 25 26 27 28 29 [ 30 ] 31 32 33 34 35 36 37 38 39 ... 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved