homepage Welcome to WebmasterWorld Guest from 54.146.190.193
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconIs it illegal to ignore a robots.txt?
  Could I sue google?
2transactiongeek 6:44 am on Feb 3, 2004 (gmt 0)
newpost thread iconrobots metatag
  I have noindex,follow but the page is indexed
2nevetS 4:41 am on Feb 3, 2004 (gmt 0)
newpost thread iconsave and upload robots.txt
 
4helenp 8:27 pm on Feb 2, 2004 (gmt 0)
newpost thread iconIs this robots.txt blocking anybody
 
8nalavanje 2:20 pm on Feb 2, 2004 (gmt 0)
newpost thread iconIs this enough
 
5dougie 11:20 am on Feb 2, 2004 (gmt 0)
newpost thread iconHow Do I Code My Robots.txt File To Disallow Subdomians?
  Is it Disallow: /apple.domain.com or Disallow: /apple/
2Christyl_Stevens 9:34 am on Feb 2, 2004 (gmt 0)
newpost thread iconasterias/2.0
  Looks for streaming media files...
6pendanticist 11:43 pm on Feb 1, 2004 (gmt 0)
newpost thread iconwill all robots crawl the sites without robot.txt?
  robot crawl, robot.txt
2johnlim 3:08 am on Feb 1, 2004 (gmt 0)
newpost thread iconWhat will Inktomi do in this scenario?
 
18flobaby 6:02 am on Jan 31, 2004 (gmt 0)
newpost thread iconGoogle spidering past secure server
  Robots.txt file to boot the bot?
2xbase234 12:52 am on Jan 31, 2004 (gmt 0)
newpost thread iconLogs keeping showing 404 error
  I have a robots file and it looks ok
3eesnard 12:03 am on Jan 30, 2004 (gmt 0)
newpost thread iconneed robots.txt for pages that are not linked?
  robots.txt
5rld3 5:09 pm on Jan 29, 2004 (gmt 0)
newpost thread iconSEO Robot.txt tag and robot meta tags... need advice
  Help about robot.txt and robot meta tags
3jamieabbott 2:09 pm on Jan 28, 2004 (gmt 0)
newpost thread iconRobots.txt returning "test"
  Host has really messed me up
4GodLikeLotus 10:37 am on Jan 28, 2004 (gmt 0)
newpost thread iconNeed quick help
  Prevent robots from clicking on PPC links
2hostlead 11:11 am on Jan 27, 2004 (gmt 0)
newpost thread iconhow to let robot crawl the links at a page but not index this page?
  index, robot
2johnlim 11:39 pm on Jan 26, 2004 (gmt 0)
newpost thread iconWildcard use in robots.txt
  Need to disallow spidering of graphics files
2Probably 4:58 am on Jan 26, 2004 (gmt 0)
newpost thread iconrobots.txt and htaccess
  .htaccess circumvents to site root
4stefan009 5:35 pm on Jan 24, 2004 (gmt 0)
newpost thread iconAltavista Scooter eats up too much trafic
 
2maxidrom11 5:28 am on Jan 24, 2004 (gmt 0)
newpost thread iconDoes every site need a robots.txt?
  I have one, but I'm not sure I need it
2MatthewHSE 4:00 pm on Jan 23, 2004 (gmt 0)
newpost thread iconcreating robots.txt
 
5artdog 12:21 pm on Jan 23, 2004 (gmt 0)
newpost thread iconMy page has never been indexed
 
2kknusa 6:45 am on Jan 23, 2004 (gmt 0)
newpost thread iconSubdomains
 
2BBB_Guy 6:38 am on Jan 23, 2004 (gmt 0)
newpost thread iconIndexed pages that are disalowed by robots.txt
  is this normal?
10WebJoe 7:03 pm on Jan 20, 2004 (gmt 0)
newpost thread iconGoogle ignoring robots.txt
  No real surprise, but a real pain
9voice220 4:12 pm on Jan 19, 2004 (gmt 0)
Displaying Page 62 of 72<< Back 1 ... 52 53 54 55 56 57 58 59 60 61 [ 62 ] 63 64 65 66 67 68 69 70 71 ... 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved