homepage Welcome to WebmasterWorld Guest from 54.159.171.164
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconA bit of help if possible would be appreciated.
 
8glennk 7:52 pm on Dec 14, 2008 (gmt 0)
newpost thread iconBlocking Email and Print only URLS
 
2himalayaswater 12:14 am on Dec 7, 2008 (gmt 0)
newpost thread iconRobots.txt - Site in a Directory
 
10Sarao 10:14 pm on Dec 6, 2008 (gmt 0)
newpost thread iconnofollow noindex or disallow?
 
4Gemini23 12:09 am on Dec 6, 2008 (gmt 0)
newpost thread iconAllowing Adsence bot
  Googled emailed me asking to allow the adsence bot
5alahamdan 2:02 pm on Dec 5, 2008 (gmt 0)
newpost thread iconPossible to block all urls with ? in them?
 
2texasville 1:28 pm on Dec 5, 2008 (gmt 0)
newpost thread iconHow is correct code within robots.txt?
 
4toplisek 11:07 am on Nov 29, 2008 (gmt 0)
newpost thread icondisallowing a variable
 
4KevinC 10:37 pm on Nov 17, 2008 (gmt 0)
newpost thread iconQueries about Robots.txt
  Importance of Robots.txt
2Monalisa 5:24 pm on Nov 17, 2008 (gmt 0)
newpost thread iconDisallow Yahoo to block site explorer
 
9minnapple 2:23 am on Nov 14, 2008 (gmt 0)
newpost thread iconMatching patterns in robots.txt
 
3kiransarv 2:39 pm on Nov 9, 2008 (gmt 0)
newpost thread iconhow to block ALL bots?
 
18Makaveli2007 10:50 am on Nov 9, 2008 (gmt 0)
newpost thread iconGoogle has a new robots.txt for today only!
 
4travelin_cat 8:56 pm on Oct 31, 2008 (gmt 0)
newpost thread iconPreventing a page from being indexed but wanting links to be followed
 
3realmaverick 5:24 pm on Oct 21, 2008 (gmt 0)
newpost thread iconDisallow .php in root folder but allow in sub.
 
3mysticalsock 5:19 pm on Oct 21, 2008 (gmt 0)
newpost thread iconrobot.txt restricted but still showing !
 
9member22 2:05 pm on Oct 21, 2008 (gmt 0)
newpost thread iconrobot.txt blocked pages but still alive
 
3member22 1:27 pm on Oct 21, 2008 (gmt 0)
newpost thread iconDisallow: /*? - is it ok ?
 
6nex99 1:22 pm on Oct 21, 2008 (gmt 0)
newpost thread iconhow to block indexing of root domain, but still allow sub-domain
  how to block indexing of root domain, but still allow sub-domain
6chazeo 1:20 pm on Oct 21, 2008 (gmt 0)
newpost thread iconBlock directory but not lower level folder
  robots.txt block directory
2Clicknowdomains 1:04 pm on Oct 21, 2008 (gmt 0)
newpost thread iconrobots.txt
 
2kiransarv 1:00 pm on Oct 21, 2008 (gmt 0)
newpost thread iconSub-domain & crawl-delay
 
2foxfox 4:39 pm on Oct 7, 2008 (gmt 0)
newpost thread iconCrawl Rate: How do I know when you request slow versus fast
  Does a long maximum time spent crawling a page have adverse impact on targe
211364guy 7:50 pm on Oct 6, 2008 (gmt 0)
newpost thread iconNon-white list blocking
  With a sting
4phred 12:51 pm on Sep 30, 2008 (gmt 0)
newpost thread iconHow to attract Yahoo! Slurp Spider
 
2ouarsenis 1:28 pm on Sep 25, 2008 (gmt 0)
Displaying Page 22 of 72<< Back 1 ... 12 13 14 15 16 17 18 19 20 21 [ 22 ] 23 24 25 26 27 28 29 30 31 ... 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved