homepage Welcome to WebmasterWorld Guest from 54.163.72.86
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Visit PubCon.com
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconDisallow .php in root folder but allow in sub.
 
3mysticalsock 5:19 pm on Oct 21, 2008 (gmt 0)
newpost thread iconrobot.txt restricted but still showing !
 
9member22 2:05 pm on Oct 21, 2008 (gmt 0)
newpost thread iconrobot.txt blocked pages but still alive
 
3member22 1:27 pm on Oct 21, 2008 (gmt 0)
newpost thread iconDisallow: /*? - is it ok ?
 
6nex99 1:22 pm on Oct 21, 2008 (gmt 0)
newpost thread iconhow to block indexing of root domain, but still allow sub-domain
  how to block indexing of root domain, but still allow sub-domain
6chazeo 1:20 pm on Oct 21, 2008 (gmt 0)
newpost thread iconBlock directory but not lower level folder
  robots.txt block directory
2Clicknowdomains 1:04 pm on Oct 21, 2008 (gmt 0)
newpost thread iconrobots.txt
 
2kiransarv 1:00 pm on Oct 21, 2008 (gmt 0)
newpost thread iconSub-domain & crawl-delay
 
2foxfox 4:39 pm on Oct 7, 2008 (gmt 0)
newpost thread iconCrawl Rate: How do I know when you request slow versus fast
  Does a long maximum time spent crawling a page have adverse impact on targe
211364guy 7:50 pm on Oct 6, 2008 (gmt 0)
newpost thread iconNon-white list blocking
  With a sting
4phred 12:51 pm on Sep 30, 2008 (gmt 0)
newpost thread iconHow to attract Yahoo! Slurp Spider
 
2ouarsenis 1:28 pm on Sep 25, 2008 (gmt 0)
newpost thread iconMy website all pages are in cache but home page are not caching
  home page is not caching , what should i do for it?
3pervezalam 6:43 pm on Sep 24, 2008 (gmt 0)
newpost thread iconrobots , duplicate content and google
 
9fsmobilez 8:07 pm on Sep 23, 2008 (gmt 0)
newpost thread iconRobots.txt
  Ive never used robots.txt
4srobinson 3:21 am on Sep 23, 2008 (gmt 0)
newpost thread iconUnderstanding the structure of robots.txt
  subdirectories
10alahamdan 12:00 am on Sep 5, 2008 (gmt 0)
newpost thread iconBaiduspider - how do I keep it out.
  having some difficulty blocking this spider
3ChicagoFan67 1:44 pm on Sep 4, 2008 (gmt 0)
newpost thread iconrestrict special duplicate pages
 
4sixeleven 11:58 pm on Sep 3, 2008 (gmt 0)
newpost thread iconHow to set Slow GoogleBot Crawl Speed
  I need to fix Google to crawl my site as very slow or specifice times, how?
7Senthil 7:54 am on Aug 31, 2008 (gmt 0)
newpost thread iconErrors of robots.txt unreachable in google webmaster sitemap
  Please anyone help me
1pervezalam 9:35 am on Aug 29, 2008 (gmt 0)
newpost thread iconWhat is Robot.txt?
  What is robot.txt,
8deepk 7:16 am on Aug 21, 2008 (gmt 0)
newpost thread icondisallow & allow
 
3doumiao 6:14 am on Aug 15, 2008 (gmt 0)
newpost thread iconRegular Expressions, robots.txt, and robotstxt.org
 
4Funtick 7:24 pm on Aug 14, 2008 (gmt 0)
newpost thread iconHow to disallow https urls
  How to disallow https urls
6spiritualseo 8:39 pm on Aug 12, 2008 (gmt 0)
newpost thread iconURLs restricted by robots.txt
  How to remove that restricton or reinclude URL
2Raheel 2:28 pm on Aug 9, 2008 (gmt 0)
newpost thread icongoogle search box
 
1sixeleven 3:10 pm on Aug 8, 2008 (gmt 0)
Displaying Page 22 of 72<< Back 1 ... 12 13 14 15 16 17 18 19 20 21 [ 22 ] 23 24 25 26 27 28 29 30 31 ... 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved