homepage Welcome to WebmasterWorld Guest from 54.196.189.229
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconLogs keeping showing 404 error
  I have a robots file and it looks ok
3eesnard 12:03 am on Jan 30, 2004 (gmt 0)
newpost thread iconneed robots.txt for pages that are not linked?
  robots.txt
5rld3 5:09 pm on Jan 29, 2004 (gmt 0)
newpost thread iconSEO Robot.txt tag and robot meta tags... need advice
  Help about robot.txt and robot meta tags
3jamieabbott 2:09 pm on Jan 28, 2004 (gmt 0)
newpost thread iconRobots.txt returning "test"
  Host has really messed me up
4GodLikeLotus 10:37 am on Jan 28, 2004 (gmt 0)
newpost thread iconNeed quick help
  Prevent robots from clicking on PPC links
2hostlead 11:11 am on Jan 27, 2004 (gmt 0)
newpost thread iconhow to let robot crawl the links at a page but not index this page?
  index, robot
2johnlim 11:39 pm on Jan 26, 2004 (gmt 0)
newpost thread iconWildcard use in robots.txt
  Need to disallow spidering of graphics files
2Probably 4:58 am on Jan 26, 2004 (gmt 0)
newpost thread iconrobots.txt and htaccess
  .htaccess circumvents to site root
4stefan009 5:35 pm on Jan 24, 2004 (gmt 0)
newpost thread iconAltavista Scooter eats up too much trafic
 
2maxidrom11 5:28 am on Jan 24, 2004 (gmt 0)
newpost thread iconDoes every site need a robots.txt?
  I have one, but I'm not sure I need it
2MatthewHSE 4:00 pm on Jan 23, 2004 (gmt 0)
newpost thread iconcreating robots.txt
 
5artdog 12:21 pm on Jan 23, 2004 (gmt 0)
newpost thread iconMy page has never been indexed
 
2kknusa 6:45 am on Jan 23, 2004 (gmt 0)
newpost thread iconSubdomains
 
2BBB_Guy 6:38 am on Jan 23, 2004 (gmt 0)
newpost thread iconIndexed pages that are disalowed by robots.txt
  is this normal?
10WebJoe 7:03 pm on Jan 20, 2004 (gmt 0)
newpost thread iconGoogle ignoring robots.txt
  No real surprise, but a real pain
9voice220 4:12 pm on Jan 19, 2004 (gmt 0)
newpost thread iconIs there any bots to block?
  Else than the infamous "Web Content International"?
3Daemon 11:56 pm on Jan 16, 2004 (gmt 0)
newpost thread iconRobots.txt help
 
5mikemcs 1:04 pm on Jan 16, 2004 (gmt 0)
newpost thread iconWho are these spiders and what do they want?
 
7obsos 4:51 pm on Jan 14, 2004 (gmt 0)
newpost thread iconRevisit instruction for robots
  I want to specify different revisit instruction different sections
3adityamodi 5:13 am on Jan 10, 2004 (gmt 0)
newpost thread iconHave I been blocking Altavista spider from spidering my site?
 
6irock 6:10 pm on Jan 9, 2004 (gmt 0)
newpost thread iconrobot.txt to remove an alias
  Afraid of duplicated content
2Ulkari 5:48 pm on Jan 8, 2004 (gmt 0)
newpost thread iconSpider Names
  I need the names of some SEs usernames
3The_Young_One 2:47 am on Jan 8, 2004 (gmt 0)
newpost thread iconBlocking Includes (SSI)
  blocking ssi
3direktor1 4:03 pm on Jan 7, 2004 (gmt 0)
newpost thread iconSubdomains
  How to exclude
4gosman 2:54 pm on Jan 6, 2004 (gmt 0)
newpost thread iconDoes the leading / make any difference?
  Sorry, I'm new to robots.txt
2AsleepATheWheel 1:07 am on Jan 6, 2004 (gmt 0)
Displaying Page 62 of 71<< Back 1 ... 52 53 54 55 56 57 58 59 60 61 [ 62 ] 63 64 65 66 67 68 69 70 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved