homepage Welcome to WebmasterWorld Guest from 54.204.141.129
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Visit PubCon.com
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconDisallow dynamic content with mod_rewrite
  Disallow dynamic content with mod_rewrite
2crglmb 3:54 pm on Oct 10, 2005 (gmt 0)
newpost thread iconRobots.txt in Ascii
  How-to?
2funnycloud 6:20 pm on Oct 7, 2005 (gmt 0)
newpost thread iconis there a reason to use a robots.txt
 
5thorthedestroyer 2:50 am on Oct 7, 2005 (gmt 0)
newpost thread iconRobot.txt and Sitemap
 
3sierra11b 1:26 am on Oct 5, 2005 (gmt 0)
newpost thread iconNewbie Robots.txt file Q
 
5CoffeeMan 12:34 pm on Oct 3, 2005 (gmt 0)
newpost thread iconSygolBot, from sygol.com: anyone seen this before?
  just visited my site, and did NOT respect robots.txt
3stapel 12:37 am on Oct 3, 2005 (gmt 0)
newpost thread iconCan I get a page crawled daily?
  disallow commands all I can find
5innocbystr 11:33 pm on Oct 2, 2005 (gmt 0)
newpost thread iconNeed to double check my Entry
  Q - regarding ( / ) (Forward Slash) in Entry
2Tutti 3:59 pm on Oct 2, 2005 (gmt 0)
newpost thread iconTrying to add robots.txt to invisionfree forum
  can't get it to work
5Damaris 1:34 am on Sep 29, 2005 (gmt 0)
newpost thread iconWhat does robots.txt help to do?
 
3flashcination 10:40 pm on Sep 28, 2005 (gmt 0)
newpost thread iconHow to block URL's of other sites link on my site?
  How to block URL's of other sites link on my site?
3sweetu 9:09 pm on Sep 25, 2005 (gmt 0)
newpost thread iconHow to make a robots.txt file?
  What are the steps?
2steelcutman1975 8:37 pm on Sep 23, 2005 (gmt 0)
newpost thread iconGoogle "URL exclusion" not following our robots.txt!
  robots.txt in subdirectory is not getting followed
9joeduck 9:00 pm on Sep 21, 2005 (gmt 0)
newpost thread iconnewbie question
  About robots.txt
5dicken288 7:37 am on Sep 21, 2005 (gmt 0)
newpost thread iconKeeping engines away from some files.
  Keeping some image files away from the prying seahc engines....
3ScoobyFlew 6:05 am on Sep 20, 2005 (gmt 0)
newpost thread iconRobot.txt
  newbie question
2playadonna 2:51 pm on Sep 18, 2005 (gmt 0)
newpost thread iconhow long till search bot checks again?
 
2editordude 11:47 am on Sep 17, 2005 (gmt 0)
newpost thread iconHow to prevent hotlinking but allow one site using htaccess?
 
2hersheyyu 11:38 pm on Sep 15, 2005 (gmt 0)
newpost thread iconHelp with disallow
  Help with disallow
4SlugKing 10:21 pm on Sep 14, 2005 (gmt 0)
newpost thread iconCan Spiders see my dynamic "includes" pages?
 
3zepper 11:57 pm on Sep 13, 2005 (gmt 0)
newpost thread iconpicking up
 
5stevelibby 7:04 am on Sep 13, 2005 (gmt 0)
newpost thread iconNeed Help
  Google spiders crawl 2 sites same content
1milindnaik 6:54 am on Sep 12, 2005 (gmt 0)
newpost thread iconAllow search spider - error 503 socket.gaierror: (7, 'getaddrinfo fail
  Allow search spider - error
5shahindastur 5:39 am on Sep 12, 2005 (gmt 0)
newpost thread iconQuick Question
  When something is disallowed, do bots ignore it completely?
5jlander 2:43 am on Sep 12, 2005 (gmt 0)
newpost thread iconAm I correctly blocking diamondbot?
 
2JAB_Creations 11:25 am on Sep 11, 2005 (gmt 0)
Displaying Page 43 of 71<< Back 1 ... 33 34 35 36 37 38 39 40 41 42 [ 43 ] 44 45 46 47 48 49 50 51 52 ... 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved