homepage Welcome to WebmasterWorld Guest from 54.242.121.134
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Visit PubCon.com
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconRobots.txt
  How? Sub-Domains
2MarkHutch 8:37 pm on Feb 6, 2003 (gmt 0)
newpost thread iconframes and robots.txt
 
4snowfishin 9:15 pm on Feb 3, 2003 (gmt 0)
newpost thread iconDo proxies and Internet caches respect robots.txt?
 
9Scooter24 12:06 am on Feb 3, 2003 (gmt 0)
newpost thread iconrobots.txt file - what program to write it in?
  I use Frontpage 2002
6Web_Footed_Newbie 4:00 pm on Jan 29, 2003 (gmt 0)
newpost thread iconRobots.txt - What program to use to write?
  I use Frontpage 2002
4Web_Footed_Newbie 3:34 pm on Jan 29, 2003 (gmt 0)
newpost thread iconPotbot not respecting robots.txt
 
3Scooter24 1:59 pm on Jan 24, 2003 (gmt 0)
newpost thread iconrobots.txt and robots meta tag not mutually exclusive?
  robots.txt robots meta tag
4Check6 1:35 pm on Jan 24, 2003 (gmt 0)
newpost thread icon"User-Agent"?
  ...or, just User-Agent ?
13pendanticist 3:22 am on Jan 22, 2003 (gmt 0)
newpost thread iconrobots.txt command to disallow indexing
  I want to only disallows robots to index a specific directory
2wellnesscafe 4:50 am on Jan 18, 2003 (gmt 0)
newpost thread iconQuick question about robots.txt
  Excluding dynamic pages
4TheDave 4:09 am on Jan 15, 2003 (gmt 0)
newpost thread iconWhy would rogue spiders obey robots.txt?
 
2irock 5:29 am on Jan 9, 2003 (gmt 0)
newpost thread iconwhat is robots.txt file?
 
3antipodes 5:57 am on Jan 4, 2003 (gmt 0)
newpost thread iconGetting ERROR_DISALLOW with Inktomi after adding new robots.txt
 
7bebox 1:41 pm on Dec 29, 2002 (gmt 0)
newpost thread iconWebmasterworld's Robots.txt
  A couple of questiosn for Brett
4victor 5:00 pm on Dec 27, 2002 (gmt 0)
newpost thread iconRobots.txt Validator bug?
  Christian Storm from Turnitin Robot maintainers says my robots.txt is wrong
3scottspence 11:15 am on Dec 17, 2002 (gmt 0)
newpost thread iconHow do I create a robot.txt?
 
9palmpal 1:35 pm on Dec 15, 2002 (gmt 0)
newpost thread iconrobots.txt to stop the BAD bots
  There was a past post with the best robots.txt file?
11Googly 11:18 pm on Dec 12, 2002 (gmt 0)
newpost thread iconRobots.txt file
  disallow target_blank
3junior 9:06 am on Dec 5, 2002 (gmt 0)
newpost thread iconrobots.txt
  robots.txt
3darkroom 9:50 am on Dec 3, 2002 (gmt 0)
newpost thread iconrobots.txt and mod_rewrite, meta robots, etc.
 
4Trisha 4:17 am on Nov 30, 2002 (gmt 0)
newpost thread iconrobot.txt syntax
 
7jamesf4218 9:48 am on Nov 25, 2002 (gmt 0)
newpost thread iconAsk Jeeves robots.txt entry
  what UA should be used
3Kerrin 4:40 pm on Nov 7, 2002 (gmt 0)
newpost thread iconlogic check on robots.txt
  will robots.txt properly disallowing a links page prevent link pop penalty?
9wooden 6:47 pm on Nov 5, 2002 (gmt 0)
newpost thread iconWhich robots to exclude
 
8biggles 1:56 am on Nov 4, 2002 (gmt 0)
newpost thread iconrobots.txt validator bug
  The robots.txt validator accepts malformed files (missing User-agent).
4mortenf 11:02 pm on Nov 3, 2002 (gmt 0)
Displaying Page 68 of 71<< Back 1 ... 58 59 60 61 62 63 64 65 66 67 [ 68 ] 69 70 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved