homepage Welcome to WebmasterWorld Guest from 54.237.197.160
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Visit PubCon.com
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconrobots.txt syntax
  spaces or slashes?
3keyplyr 5:14 pm on Feb 27, 2003 (gmt 0)
newpost thread iconStrange Incident with robots.txt
  error code 401
3neh2008 4:57 pm on Feb 17, 2003 (gmt 0)
newpost thread iconrobots.txt to block bots access to images
  got to save bandwidth
4stavs 3:00 pm on Feb 14, 2003 (gmt 0)
newpost thread iconROBOTS.TXT Location
  ROBOTS.TXT Location
4neh2008 5:27 pm on Feb 11, 2003 (gmt 0)
newpost thread iconRobots.txt
  I better get this right...
6JVB_Mktg 4:19 am on Feb 11, 2003 (gmt 0)
newpost thread iconRobots.txt
  How? Sub-Domains
2MarkHutch 8:37 pm on Feb 6, 2003 (gmt 0)
newpost thread iconframes and robots.txt
 
4snowfishin 9:15 pm on Feb 3, 2003 (gmt 0)
newpost thread iconDo proxies and Internet caches respect robots.txt?
 
9Scooter24 12:06 am on Feb 3, 2003 (gmt 0)
newpost thread iconrobots.txt file - what program to write it in?
  I use Frontpage 2002
6Web_Footed_Newbie 4:00 pm on Jan 29, 2003 (gmt 0)
newpost thread iconRobots.txt - What program to use to write?
  I use Frontpage 2002
4Web_Footed_Newbie 3:34 pm on Jan 29, 2003 (gmt 0)
newpost thread iconPotbot not respecting robots.txt
 
3Scooter24 1:59 pm on Jan 24, 2003 (gmt 0)
newpost thread iconrobots.txt and robots meta tag not mutually exclusive?
  robots.txt robots meta tag
4Check6 1:35 pm on Jan 24, 2003 (gmt 0)
newpost thread icon"User-Agent"?
  ...or, just User-Agent ?
13pendanticist 3:22 am on Jan 22, 2003 (gmt 0)
newpost thread iconrobots.txt command to disallow indexing
  I want to only disallows robots to index a specific directory
2wellnesscafe 4:50 am on Jan 18, 2003 (gmt 0)
newpost thread iconQuick question about robots.txt
  Excluding dynamic pages
4TheDave 4:09 am on Jan 15, 2003 (gmt 0)
newpost thread iconWhy would rogue spiders obey robots.txt?
 
2irock 5:29 am on Jan 9, 2003 (gmt 0)
newpost thread iconwhat is robots.txt file?
 
3antipodes 5:57 am on Jan 4, 2003 (gmt 0)
newpost thread iconGetting ERROR_DISALLOW with Inktomi after adding new robots.txt
 
7bebox 1:41 pm on Dec 29, 2002 (gmt 0)
newpost thread iconWebmasterworld's Robots.txt
  A couple of questiosn for Brett
4victor 5:00 pm on Dec 27, 2002 (gmt 0)
newpost thread iconRobots.txt Validator bug?
  Christian Storm from Turnitin Robot maintainers says my robots.txt is wrong
3scottspence 11:15 am on Dec 17, 2002 (gmt 0)
newpost thread iconHow do I create a robot.txt?
 
9palmpal 1:35 pm on Dec 15, 2002 (gmt 0)
newpost thread iconrobots.txt to stop the BAD bots
  There was a past post with the best robots.txt file?
11Googly 11:18 pm on Dec 12, 2002 (gmt 0)
newpost thread iconRobots.txt file
  disallow target_blank
3junior 9:06 am on Dec 5, 2002 (gmt 0)
newpost thread iconrobots.txt
  robots.txt
3darkroom 9:50 am on Dec 3, 2002 (gmt 0)
newpost thread iconrobots.txt and mod_rewrite, meta robots, etc.
 
4Trisha 4:17 am on Nov 30, 2002 (gmt 0)
Displaying Page 68 of 72<< Back 1 ... 58 59 60 61 62 63 64 65 66 67 [ 68 ] 69 70 71 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved