homepage Welcome to WebmasterWorld Guest from 54.196.195.207
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Visit PubCon.com
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconDo proxies and Internet caches respect robots.txt?
 
9Scooter24 12:06 am on Feb 3, 2003 (gmt 0)
newpost thread iconrobots.txt file - what program to write it in?
  I use Frontpage 2002
6Web_Footed_Newbie 4:00 pm on Jan 29, 2003 (gmt 0)
newpost thread iconRobots.txt - What program to use to write?
  I use Frontpage 2002
4Web_Footed_Newbie 3:34 pm on Jan 29, 2003 (gmt 0)
newpost thread iconPotbot not respecting robots.txt
 
3Scooter24 1:59 pm on Jan 24, 2003 (gmt 0)
newpost thread iconrobots.txt and robots meta tag not mutually exclusive?
  robots.txt robots meta tag
4Check6 1:35 pm on Jan 24, 2003 (gmt 0)
newpost thread icon"User-Agent"?
  ...or, just User-Agent ?
13pendanticist 3:22 am on Jan 22, 2003 (gmt 0)
newpost thread iconrobots.txt command to disallow indexing
  I want to only disallows robots to index a specific directory
2wellnesscafe 4:50 am on Jan 18, 2003 (gmt 0)
newpost thread iconQuick question about robots.txt
  Excluding dynamic pages
4TheDave 4:09 am on Jan 15, 2003 (gmt 0)
newpost thread iconWhy would rogue spiders obey robots.txt?
 
2irock 5:29 am on Jan 9, 2003 (gmt 0)
newpost thread iconwhat is robots.txt file?
 
3antipodes 5:57 am on Jan 4, 2003 (gmt 0)
newpost thread iconGetting ERROR_DISALLOW with Inktomi after adding new robots.txt
 
7bebox 1:41 pm on Dec 29, 2002 (gmt 0)
newpost thread iconWebmasterworld's Robots.txt
  A couple of questiosn for Brett
4victor 5:00 pm on Dec 27, 2002 (gmt 0)
newpost thread iconRobots.txt Validator bug?
  Christian Storm from Turnitin Robot maintainers says my robots.txt is wrong
3scottspence 11:15 am on Dec 17, 2002 (gmt 0)
newpost thread iconHow do I create a robot.txt?
 
9palmpal 1:35 pm on Dec 15, 2002 (gmt 0)
newpost thread iconrobots.txt to stop the BAD bots
  There was a past post with the best robots.txt file?
11Googly 11:18 pm on Dec 12, 2002 (gmt 0)
newpost thread iconRobots.txt file
  disallow target_blank
3junior 9:06 am on Dec 5, 2002 (gmt 0)
newpost thread iconrobots.txt
  robots.txt
3darkroom 9:50 am on Dec 3, 2002 (gmt 0)
newpost thread iconrobots.txt and mod_rewrite, meta robots, etc.
 
4Trisha 4:17 am on Nov 30, 2002 (gmt 0)
newpost thread iconrobot.txt syntax
 
7jamesf4218 9:48 am on Nov 25, 2002 (gmt 0)
newpost thread iconAsk Jeeves robots.txt entry
  what UA should be used
3Kerrin 4:40 pm on Nov 7, 2002 (gmt 0)
newpost thread iconlogic check on robots.txt
  will robots.txt properly disallowing a links page prevent link pop penalty?
9wooden 6:47 pm on Nov 5, 2002 (gmt 0)
newpost thread iconWhich robots to exclude
 
8biggles 1:56 am on Nov 4, 2002 (gmt 0)
newpost thread iconrobots.txt validator bug
  The robots.txt validator accepts malformed files (missing User-agent).
4mortenf 11:02 pm on Nov 3, 2002 (gmt 0)
newpost thread iconWhat is a Robots.txt?
 
4Noximus 3:48 pm on Oct 28, 2002 (gmt 0)
newpost thread iconSpider Blocking
  still a bit unclear
3flexsez 7:43 pm on Oct 23, 2002 (gmt 0)
Displaying Page 68 of 71<< Back 1 ... 58 59 60 61 62 63 64 65 66 67 [ 68 ] 69 70 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved