homepage Welcome to WebmasterWorld Guest from 54.204.168.212
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconRobots.txt and Query Strings
  Can they block non directory paths?
3soquinn 4:45 am on Oct 26, 2005 (gmt 0)
newpost thread iconSpider and registred access to web
 
5andreabg 8:07 am on Oct 19, 2005 (gmt 0)
newpost thread icongoogle accessing blocked image files
  why does it appear that google is not abiding robots.txt?
3latimer 8:01 am on Oct 19, 2005 (gmt 0)
newpost thread iconDisallow all Directories
 
2Omega1Megs 8:48 pm on Oct 14, 2005 (gmt 0)
newpost thread iconDisallow: line in robots.txt
 
2stefan009 5:33 pm on Oct 14, 2005 (gmt 0)
newpost thread iconCan a robots.txt be too long?
 
3gingerspice 3:32 am on Oct 13, 2005 (gmt 0)
newpost thread iconLog file is robot.txt file?
  Log file is robot.txt file?
5i_am_dhaval 9:20 pm on Oct 12, 2005 (gmt 0)
newpost thread iconExcluding domain in robots.txt
 
10triumph 7:25 pm on Oct 11, 2005 (gmt 0)
newpost thread iconDisallow dynamic content with mod_rewrite
  Disallow dynamic content with mod_rewrite
2crglmb 3:54 pm on Oct 10, 2005 (gmt 0)
newpost thread iconRobots.txt in Ascii
  How-to?
2funnycloud 6:20 pm on Oct 7, 2005 (gmt 0)
newpost thread iconis there a reason to use a robots.txt
 
5thorthedestroyer 2:50 am on Oct 7, 2005 (gmt 0)
newpost thread iconRobot.txt and Sitemap
 
3sierra11b 1:26 am on Oct 5, 2005 (gmt 0)
newpost thread iconNewbie Robots.txt file Q
 
5CoffeeMan 12:34 pm on Oct 3, 2005 (gmt 0)
newpost thread iconSygolBot, from sygol.com: anyone seen this before?
  just visited my site, and did NOT respect robots.txt
3stapel 12:37 am on Oct 3, 2005 (gmt 0)
newpost thread iconCan I get a page crawled daily?
  disallow commands all I can find
5innocbystr 11:33 pm on Oct 2, 2005 (gmt 0)
newpost thread iconNeed to double check my Entry
  Q - regarding ( / ) (Forward Slash) in Entry
2Tutti 3:59 pm on Oct 2, 2005 (gmt 0)
newpost thread iconTrying to add robots.txt to invisionfree forum
  can't get it to work
5Damaris 1:34 am on Sep 29, 2005 (gmt 0)
newpost thread iconWhat does robots.txt help to do?
 
3flashcination 10:40 pm on Sep 28, 2005 (gmt 0)
newpost thread iconHow to block URL's of other sites link on my site?
  How to block URL's of other sites link on my site?
3sweetu 9:09 pm on Sep 25, 2005 (gmt 0)
newpost thread iconHow to make a robots.txt file?
  What are the steps?
2steelcutman1975 8:37 pm on Sep 23, 2005 (gmt 0)
newpost thread iconGoogle "URL exclusion" not following our robots.txt!
  robots.txt in subdirectory is not getting followed
9joeduck 9:00 pm on Sep 21, 2005 (gmt 0)
newpost thread iconnewbie question
  About robots.txt
5dicken288 7:37 am on Sep 21, 2005 (gmt 0)
newpost thread iconKeeping engines away from some files.
  Keeping some image files away from the prying seahc engines....
3ScoobyFlew 6:05 am on Sep 20, 2005 (gmt 0)
newpost thread iconRobot.txt
  newbie question
2playadonna 2:51 pm on Sep 18, 2005 (gmt 0)
newpost thread iconhow long till search bot checks again?
 
2editordude 11:47 am on Sep 17, 2005 (gmt 0)
Displaying Page 43 of 72<< Back 1 ... 33 34 35 36 37 38 39 40 41 42 [ 43 ] 44 45 46 47 48 49 50 51 52 ... 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved