homepage Welcome to WebmasterWorld Guest from 54.166.122.86
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconhelp needed
 
1hugh 4:03 pm on Apr 2, 2007 (gmt 0)
newpost thread iconWhat's the Proper Way to Block Sub Folders
  Looking for the best way to block 2nd level sub folders
5graywolf 7:43 pm on Mar 28, 2007 (gmt 0)
newpost thread iconROBOT and PR rank?
  how does the robot effect your pr rank?
7moppy56 1:48 pm on Mar 28, 2007 (gmt 0)
newpost thread iconBlocking Partial Contents of Folder Without Using Robots Tag
  10 files (7 want indexed, 3 want blocked)
5SandySEO 8:49 pm on Mar 27, 2007 (gmt 0)
newpost thread iconAnother bad bot to ban
  this one is using 302 redirects .. bad bad
1Bewenched 3:43 am on Mar 25, 2007 (gmt 0)
newpost thread iconexcluding a subdomain
  Using the robots.txt file to exclude a subdomain
10joebray 9:47 am on Mar 23, 2007 (gmt 0)
newpost thread iconG. ignores robots.txt at other ports, gives false page count
  Google shows in its index pages that it never saw, even prohibited
1OnetSzukaj 2:58 pm on Mar 22, 2007 (gmt 0)
newpost thread iconHow to use Robots.txt as an INCLUSION method
  It's exclusion only if you use it that way
1incrediBILL 12:32 am on Mar 22, 2007 (gmt 0)
newpost thread iconTripAdvisorBot
  Anyone seen this?
3bobbarnes 10:28 pm on Mar 16, 2007 (gmt 0)
newpost thread iconproblem with msnbot and wildcard matching
 
2Achernar 1:36 pm on Mar 16, 2007 (gmt 0)
newpost thread iconHow Can I Make Bots Visit My Web Site?
 
3emadido 2:34 pm on Mar 15, 2007 (gmt 0)
newpost thread iconwhy i can't reaply the topic
  why i can't reaply the topic
3unknown 8:43 am on Mar 14, 2007 (gmt 0)
newpost thread iconrobotsxx.txt not found.
 
2benevolent001 1:16 pm on Mar 13, 2007 (gmt 0)
newpost thread iconSomething a little different
 
2futureX 1:56 pm on Mar 12, 2007 (gmt 0)
newpost thread iconWhy is WebmasterWorld robot validator giving back HTML
 
3Clair 7:11 pm on Mar 10, 2007 (gmt 0)
newpost thread iconWhen Robots.txt And Meta Robots Collide
  What happens?
13Easy_Coder 1:57 am on Mar 10, 2007 (gmt 0)
newpost thread iconOSC, dupe content, robots.txt and 301 redirects
  Need to ensure bots 301 to new SEO URL
5spina45 8:23 pm on Mar 9, 2007 (gmt 0)
newpost thread iconDisallow / access to our directory
  disallow
2drew68 3:47 pm on Mar 6, 2007 (gmt 0)
newpost thread iconBlock the main page of a directory but not its contents?
 
6bresso 7:39 am on Mar 6, 2007 (gmt 0)
newpost thread iconRobots.txt and pointing domains
 
3Shanada 6:58 am on Feb 28, 2007 (gmt 0)
newpost thread iconWill Googlebot be able to crawl these pages of my website?
 
4saqitude 3:40 pm on Feb 25, 2007 (gmt 0)
newpost thread iconrobots.txt question
 
2seomack 8:14 am on Feb 19, 2007 (gmt 0)
newpost thread iconIs correct this robots.txt?
 
2MalastiC 9:44 pm on Feb 18, 2007 (gmt 0)
newpost thread iconSitemap and robot.txt
 
3spuddy 3:53 pm on Feb 15, 2007 (gmt 0)
newpost thread iconHow to exclude from crawling subdomain
  exclude the crawling subdomain
4fendy 1:00 pm on Feb 15, 2007 (gmt 0)
Displaying Page 30 of 72<< Back 1 ... 20 21 22 23 24 25 26 27 28 29 [ 30 ] 31 32 33 34 35 36 37 38 39 ... 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved