homepage Welcome to WebmasterWorld Guest from 54.234.147.84
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconrobots.txt file
 
3ktd121 3:14 am on Jul 4, 2003 (gmt 0)
newpost thread iconmeta name = robot
  What does that do?
7aroye 5:57 pm on Jun 30, 2003 (gmt 0)
newpost thread iconScooter 3.3 ignores robots.txt?
  this is ridiculous
6berli 2:39 am on Jun 27, 2003 (gmt 0)
newpost thread iconRobots.txt
  Help please mine is missing?
3peteredmonds 8:37 pm on Jun 25, 2003 (gmt 0)
newpost thread iconRobots.txt
  Huh?
2zoobie 6:52 pm on Jun 19, 2003 (gmt 0)
newpost thread icon"robots"
  What should it be set at?
4kimderella86 6:22 pm on Jun 18, 2003 (gmt 0)
newpost thread iconRobots.txt exclusion for dynamic page
 
2rogerd 9:22 pm on Jun 17, 2003 (gmt 0)
newpost thread iconRobots.txt & How your site is spidered
  Positives of Robots.txt file aside from disallowing bots
4floridadesigns 11:31 pm on Jun 14, 2003 (gmt 0)
newpost thread iconWhy should I obey robots.txt
  Developer questions
22PWalker 1:01 am on Jun 13, 2003 (gmt 0)
newpost thread iconPhp robots.txt parser script
  does that exitst? if not regex help needed.
5ikbenhet1 2:57 pm on Jun 9, 2003 (gmt 0)
newpost thread iconfiles requested by robots.txt
  i would like to find out about the files requested by robots
8andrea_edwards 7:53 pm on Jun 8, 2003 (gmt 0)
newpost thread iconToolbar and Robots.txt
 
8rintrah 8:41 pm on Jun 5, 2003 (gmt 0)
newpost thread iconRobots.txt - files in folders
 
3rintrah 9:43 pm on Jun 4, 2003 (gmt 0)
newpost thread iconrobots.txt
  bad spider
6copongcopong 9:11 pm on Jun 4, 2003 (gmt 0)
newpost thread iconRobots.txt - What's the Point?
 
7rintrah 6:59 pm on May 30, 2003 (gmt 0)
newpost thread iconPragma & Cache-Control
  Are these Safe with Search Engines
9stuart 8:32 pm on May 27, 2003 (gmt 0)
newpost thread iconQuick robots.txt question
  Do I just need to put the file name
2bateman_ap 8:21 pm on May 24, 2003 (gmt 0)
newpost thread iconUsing Allow: / in robots.txt
  Google says to use it?
5werty 3:28 am on May 20, 2003 (gmt 0)
newpost thread iconHow often should a robot check the robots.txt file?
  Your opinions please
8jrobbio 2:25 am on May 18, 2003 (gmt 0)
newpost thread iconUsing Robot.txt to Block Spidering of Link pages
  Is this a common underhanded practice?
12jdancing 4:26 pm on May 8, 2003 (gmt 0)
newpost thread iconrobots.txt
  should i even bother if i want the whole site indexed?
9madeonmoon 4:37 pm on May 7, 2003 (gmt 0)
newpost thread iconrobots.txt
  used for site with 2 urls
3spinweb 2:11 am on May 7, 2003 (gmt 0)
newpost thread iconrobots.txt validator warning suggestion
  Missing newlines can cause problems, but in some cases can be detected.
2mortenf 4:42 pm on Apr 25, 2003 (gmt 0)
newpost thread iconDangers of the robots.txt file
  A warning to the amateurs like myself
3joeuz 6:37 am on Apr 23, 2003 (gmt 0)
newpost thread iconRobots.txt
  Stopping robots visiting single dynamic page.
2donovana 2:02 am on Apr 22, 2003 (gmt 0)
Displaying Page 66 of 71<< Back 1 ... 56 57 58 59 60 61 62 63 64 65 [ 66 ] 67 68 69 70 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved