homepage Welcome to WebmasterWorld Guest from 54.226.168.96
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Accredited PayPal World Seller

Visit PubCon.com
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconmy robots file
  need help
3experienced 12:26 am on Dec 12, 2006 (gmt 0)
newpost thread iconDisagreement between Google and a validator
  Robots.txt Checker says it's so, Google robots anayler says the other way
6Gusgsm 9:49 am on Dec 11, 2006 (gmt 0)
newpost thread iconGooglebot tripping bad bot script
 
5RobertRogers 5:29 pm on Dec 10, 2006 (gmt 0)
newpost thread iconIs this bad robots.txt
 
1scoobydooaustralia 8:02 pm on Dec 8, 2006 (gmt 0)
newpost thread iconduplicate pages .aspx?id=
  Not sure what to do next
2schatzm 9:01 pm on Dec 6, 2006 (gmt 0)
newpost thread iconIs this a bot?
  Unwanted visitor to my website
2brancook 8:57 pm on Dec 6, 2006 (gmt 0)
newpost thread iconTwo domains mirror each other
  Is it possible to a command to a robots.txt file to apply to just one?
4Jeremy_H 9:08 pm on Dec 5, 2006 (gmt 0)
newpost thread iconDo I need to disallow files that aren't linked to?
  for example, 'included' PHP & javascript pages
9sssweb 3:34 pm on Dec 4, 2006 (gmt 0)
newpost thread iconremoving permanent redirect - honoring content in page
  removing permanent redirect
2thu_household 2:56 pm on Dec 4, 2006 (gmt 0)
newpost thread iconDo we need robots at all? New member here
  Confused as to whether I need to add robots txt
5plenusvita 9:23 pm on Dec 2, 2006 (gmt 0)
newpost thread iconBlock all spiders, but only allow well known spiders?
 
2foxfox 3:19 pm on Dec 2, 2006 (gmt 0)
newpost thread iconmsnbot visits robots.txt and exits
 
7pemba76 12:13 pm on Nov 28, 2006 (gmt 0)
newpost thread iconUpload of incorrect robots.txt
  What a screw up - any permanent damage?
4TinkyWinky 10:12 am on Nov 28, 2006 (gmt 0)
newpost thread iconDoes excluding Googlebot create a penalty?
  If other bots are allowed, is this a red flag to Google
2AndyA 9:26 pm on Nov 27, 2006 (gmt 0)
newpost thread iconwhat to put into robots.txt?
 
11Hitman3266 5:09 pm on Nov 27, 2006 (gmt 0)
newpost thread iconHow long does it take for SEs to learn about a robots'txt
 
4bouncybunny 12:04 pm on Nov 20, 2006 (gmt 0)
newpost thread iconYahoo! Slurp Now Supports Wildcards in robots.txt
 
17OldWolf 2:54 am on Nov 20, 2006 (gmt 0)
newpost thread iconRobots.txt and password-protection
 
3hwy2hel 10:56 pm on Nov 16, 2006 (gmt 0)
newpost thread iconYahoo robots txt
  nedd to disallow all urls with?
2cheaperholidays 4:54 pm on Nov 16, 2006 (gmt 0)
newpost thread iconrobots.txt? OR .htaccess?
  disallowing or protecting directories
2johnjoyce 4:27 pm on Nov 16, 2006 (gmt 0)
newpost thread iconWildcards in robots.txt for forum software
 
6Asia_Expat 1:40 am on Nov 14, 2006 (gmt 0)
newpost thread iconrobots.txt syntax
  '$' and '?'
2sssweb 8:40 pm on Nov 11, 2006 (gmt 0)
newpost thread iconrobots.txt help
 
2miracle 7:16 pm on Nov 8, 2006 (gmt 0)
newpost thread iconLink Checker / Site map generator
  One that observes robots.txt and meta tags
5fish_eye 5:33 am on Nov 7, 2006 (gmt 0)
newpost thread iconDisallow by ISP
  Where to get a list of ISPs and the countries they are in?
1youfoundjake 7:13 pm on Nov 3, 2006 (gmt 0)
Displaying Page 32 of 71<< Back 1 ... 22 23 24 25 26 27 28 29 30 31 [ 32 ] 33 34 35 36 37 38 39 40 41 ... 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved