homepage Welcome to WebmasterWorld Guest from 54.166.122.86
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconEfficient way to block multiple bots?
 
11JAB_Creations 2:33 pm on Apr 3, 2007 (gmt 0)
newpost thread iconBlocking Google
  I need to block google from my index page
4loner 4:25 pm on Apr 2, 2007 (gmt 0)
newpost thread iconhelp needed
 
1hugh 4:03 pm on Apr 2, 2007 (gmt 0)
newpost thread iconWhat's the Proper Way to Block Sub Folders
  Looking for the best way to block 2nd level sub folders
5graywolf 7:43 pm on Mar 28, 2007 (gmt 0)
newpost thread iconROBOT and PR rank?
  how does the robot effect your pr rank?
7moppy56 1:48 pm on Mar 28, 2007 (gmt 0)
newpost thread iconBlocking Partial Contents of Folder Without Using Robots Tag
  10 files (7 want indexed, 3 want blocked)
5SandySEO 8:49 pm on Mar 27, 2007 (gmt 0)
newpost thread iconAnother bad bot to ban
  this one is using 302 redirects .. bad bad
1Bewenched 3:43 am on Mar 25, 2007 (gmt 0)
newpost thread iconexcluding a subdomain
  Using the robots.txt file to exclude a subdomain
10joebray 9:47 am on Mar 23, 2007 (gmt 0)
newpost thread iconG. ignores robots.txt at other ports, gives false page count
  Google shows in its index pages that it never saw, even prohibited
1OnetSzukaj 2:58 pm on Mar 22, 2007 (gmt 0)
newpost thread iconHow to use Robots.txt as an INCLUSION method
  It's exclusion only if you use it that way
1incrediBILL 12:32 am on Mar 22, 2007 (gmt 0)
newpost thread iconTripAdvisorBot
  Anyone seen this?
3bobbarnes 10:28 pm on Mar 16, 2007 (gmt 0)
newpost thread iconproblem with msnbot and wildcard matching
 
2Achernar 1:36 pm on Mar 16, 2007 (gmt 0)
newpost thread iconHow Can I Make Bots Visit My Web Site?
 
3emadido 2:34 pm on Mar 15, 2007 (gmt 0)
newpost thread iconwhy i can't reaply the topic
  why i can't reaply the topic
3unknown 8:43 am on Mar 14, 2007 (gmt 0)
newpost thread iconrobotsxx.txt not found.
 
2benevolent001 1:16 pm on Mar 13, 2007 (gmt 0)
newpost thread iconSomething a little different
 
2futureX 1:56 pm on Mar 12, 2007 (gmt 0)
newpost thread iconWhy is WebmasterWorld robot validator giving back HTML
 
3Clair 7:11 pm on Mar 10, 2007 (gmt 0)
newpost thread iconWhen Robots.txt And Meta Robots Collide
  What happens?
13Easy_Coder 1:57 am on Mar 10, 2007 (gmt 0)
newpost thread iconOSC, dupe content, robots.txt and 301 redirects
  Need to ensure bots 301 to new SEO URL
5spina45 8:23 pm on Mar 9, 2007 (gmt 0)
newpost thread iconDisallow / access to our directory
  disallow
2drew68 3:47 pm on Mar 6, 2007 (gmt 0)
newpost thread iconBlock the main page of a directory but not its contents?
 
6bresso 7:39 am on Mar 6, 2007 (gmt 0)
newpost thread iconRobots.txt and pointing domains
 
3Shanada 6:58 am on Feb 28, 2007 (gmt 0)
newpost thread iconWill Googlebot be able to crawl these pages of my website?
 
4saqitude 3:40 pm on Feb 25, 2007 (gmt 0)
newpost thread iconrobots.txt question
 
2seomack 8:14 am on Feb 19, 2007 (gmt 0)
newpost thread iconIs correct this robots.txt?
 
2MalastiC 9:44 pm on Feb 18, 2007 (gmt 0)
Displaying Page 30 of 72<< Back 1 ... 20 21 22 23 24 25 26 27 28 29 [ 30 ] 31 32 33 34 35 36 37 38 39 ... 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved