homepage Welcome to WebmasterWorld Guest from 54.237.247.172
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconremoving permanent redirect - honoring content in page
  removing permanent redirect
2thu_household 2:56 pm on Dec 4, 2006 (gmt 0)
newpost thread iconDo we need robots at all? New member here
  Confused as to whether I need to add robots txt
5plenusvita 9:23 pm on Dec 2, 2006 (gmt 0)
newpost thread iconBlock all spiders, but only allow well known spiders?
 
2foxfox 3:19 pm on Dec 2, 2006 (gmt 0)
newpost thread iconmsnbot visits robots.txt and exits
 
7pemba76 12:13 pm on Nov 28, 2006 (gmt 0)
newpost thread iconUpload of incorrect robots.txt
  What a screw up - any permanent damage?
4TinkyWinky 10:12 am on Nov 28, 2006 (gmt 0)
newpost thread iconDoes excluding Googlebot create a penalty?
  If other bots are allowed, is this a red flag to Google
2AndyA 9:26 pm on Nov 27, 2006 (gmt 0)
newpost thread iconwhat to put into robots.txt?
 
11Hitman3266 5:09 pm on Nov 27, 2006 (gmt 0)
newpost thread iconHow long does it take for SEs to learn about a robots'txt
 
4bouncybunny 12:04 pm on Nov 20, 2006 (gmt 0)
newpost thread iconYahoo! Slurp Now Supports Wildcards in robots.txt
 
17OldWolf 2:54 am on Nov 20, 2006 (gmt 0)
newpost thread iconRobots.txt and password-protection
 
3hwy2hel 10:56 pm on Nov 16, 2006 (gmt 0)
newpost thread iconYahoo robots txt
  nedd to disallow all urls with?
2cheaperholidays 4:54 pm on Nov 16, 2006 (gmt 0)
newpost thread iconrobots.txt? OR .htaccess?
  disallowing or protecting directories
2johnjoyce 4:27 pm on Nov 16, 2006 (gmt 0)
newpost thread iconWildcards in robots.txt for forum software
 
6Asia_Expat 1:40 am on Nov 14, 2006 (gmt 0)
newpost thread iconrobots.txt syntax
  '$' and '?'
2sssweb 8:40 pm on Nov 11, 2006 (gmt 0)
newpost thread iconrobots.txt help
 
2miracle 7:16 pm on Nov 8, 2006 (gmt 0)
newpost thread iconLink Checker / Site map generator
  One that observes robots.txt and meta tags
5fish_eye 5:33 am on Nov 7, 2006 (gmt 0)
newpost thread iconDisallow by ISP
  Where to get a list of ISPs and the countries they are in?
1youfoundjake 7:13 pm on Nov 3, 2006 (gmt 0)
newpost thread iconMade the mistake of putting sub folders I want allowed
  disallow everthing but?
2Sunnyvale 7:07 pm on Oct 31, 2006 (gmt 0)
newpost thread iconGoogle remove url tool for supplemental urls a waste of time?
  Should I bother removing duplicate supplementals?
3Tonerman 4:20 pm on Oct 30, 2006 (gmt 0)
newpost thread iconallow major engines, block the fleas
  and to add further control
5youfoundjake 5:16 pm on Oct 26, 2006 (gmt 0)
newpost thread iconDissallowing Dynamic Content
  Blocking "index.php?"
2trillianjedi 6:58 pm on Oct 25, 2006 (gmt 0)
newpost thread iconCombining htaccess and robots.txt?
 
7Lady18 11:37 pm on Oct 24, 2006 (gmt 0)
newpost thread icondisallowing a 'staging server'
 
2rden17 4:11 pm on Oct 24, 2006 (gmt 0)
newpost thread icon.htaccess
 
2ssgumby 9:53 pm on Oct 21, 2006 (gmt 0)
newpost thread iconcloaking robots.txt
  to avoid hacking
4idolw 12:26 pm on Oct 18, 2006 (gmt 0)
Displaying Page 33 of 72<< Back 1 ... 23 24 25 26 27 28 29 30 31 32 [ 33 ] 34 35 36 37 38 39 40 41 42 ... 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved