homepage Welcome to WebmasterWorld Guest from 54.166.105.24
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconbasic robots.txt question
 
5kapow 4:31 pm on Aug 2, 2001 (gmt 0)
newpost thread iconI am new to all of this about Spiders...Robot.txt
  Help Message
8lightwriter 12:42 am on Jul 23, 2001 (gmt 0)
newpost thread iconrobots.txt
  Banning dodgy spiders - do they take any notice?
4themoff 7:35 pm on Jul 13, 2001 (gmt 0)
newpost thread iconRobots.txt
  Does Google honour it....
12Rusky 5:58 am on Jul 5, 2001 (gmt 0)
newpost thread iconFear and Loathing of Robots.txt
  Coming Clean - Confessions of a a robo'phobic
12Brett_Tabke 1:23 am on Jun 25, 2001 (gmt 0)
newpost thread iconBad style in robots.txt file. Why?
 
5billyblu 10:25 am on Jun 1, 2001 (gmt 0)
newpost thread iconRobots.txt Validator
 
4Froggyman 9:17 am on May 26, 2001 (gmt 0)
newpost thread iconRobots.txt
  Is there such a thing as Allow: *
4merlin78 2:38 am on May 24, 2001 (gmt 0)
newpost thread iconRobots.txt file
  How do I do this...
2lizzie 8:27 pm on May 19, 2001 (gmt 0)
newpost thread iconRobots.txt
  How effective is it anyway
6circuitjump 2:35 pm on May 11, 2001 (gmt 0)
newpost thread iconRobozilla and Robots.txt
 
4Brett_Tabke 8:11 am on May 5, 2001 (gmt 0)
newpost thread iconreplacing robots.txt with a CGI
 
7theperlyking 7:42 pm on Apr 25, 2001 (gmt 0)
newpost thread iconrobots.txt
  warning; robots.txt containing DOS or Mac line
5roma 10:35 am on Apr 25, 2001 (gmt 0)
newpost thread iconRobots.txt
  Do you use it if you want to all all robots?
7WebSeeker 8:30 am on Apr 16, 2001 (gmt 0)
newpost thread iconLooking for complicated borderline Robots.txt
 
4Brett_Tabke 10:14 am on Apr 9, 2001 (gmt 0)
newpost thread iconGoogle Link points and Robots.txt[2] ( 1 2 )
  Google points algo and side discussion of robots.txt format
34tmw 2:53 pm on Mar 27, 2001 (gmt 0)
newpost thread iconDoorway / Hallway pages Vs. Robots.txt
  Does one need to build "many" sites under one roof?
5Garyh 10:45 pm on Mar 20, 2001 (gmt 0)
newpost thread iconYahoo and Robots.txt
 
2Brett_Tabke 9:16 pm on Feb 28, 2001 (gmt 0)
newpost thread iconHow does robot.txt work?
 
4Acternaweb 2:59 am on Feb 28, 2001 (gmt 0)
newpost thread iconMr. Roboto
  How do I set up a Robots.txt file?
6CaveToad 3:57 pm on Jan 31, 2001 (gmt 0)
newpost thread iconRobots.txt and subdomains.
  How do spiders react on robots.txt on subdomains?
1Rumbas 11:05 am on Jan 29, 2001 (gmt 0)
newpost thread iconSpider.txt
 
10graemel 10:47 pm on Jan 24, 2001 (gmt 0)
newpost thread iconSpiders Ignoring Robots.txt
  Content Vs. Legal Link or Site Trespassing.
6Garyh 4:12 pm on Jan 22, 2001 (gmt 0)
newpost thread iconWhy would I use a Robots.txt file?
 
7BoneHeadicus 12:40 am on Jan 14, 2001 (gmt 0)
newpost thread iconShould I be using a Robots.txt file?\n
 
6BoneHeadicus 7:40 pm on Jan 10, 2001 (gmt 0)
Displaying Page 71 of 72<< Back 1 ... 61 62 63 64 65 66 67 68 69 70 [ 71 ] 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved