homepage Welcome to WebmasterWorld Guest from 54.234.60.133
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe and Support WebmasterWorld
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconHow to block a subdomian
  Where to put the file
5Musicarl 5:39 am on Dec 15, 2008 (gmt 0)
newpost thread iconA bit of help if possible would be appreciated.
 
8glennk 7:52 pm on Dec 14, 2008 (gmt 0)
newpost thread iconBlocking Email and Print only URLS
 
2himalayaswater 12:14 am on Dec 7, 2008 (gmt 0)
newpost thread iconRobots.txt - Site in a Directory
 
10Sarao 10:14 pm on Dec 6, 2008 (gmt 0)
newpost thread iconnofollow noindex or disallow?
 
4Gemini23 12:09 am on Dec 6, 2008 (gmt 0)
newpost thread iconAllowing Adsence bot
  Googled emailed me asking to allow the adsence bot
5alahamdan 2:02 pm on Dec 5, 2008 (gmt 0)
newpost thread iconPossible to block all urls with ? in them?
 
2texasville 1:28 pm on Dec 5, 2008 (gmt 0)
newpost thread iconHow is correct code within robots.txt?
 
4toplisek 11:07 am on Nov 29, 2008 (gmt 0)
newpost thread icondisallowing a variable
 
4KevinC 10:37 pm on Nov 17, 2008 (gmt 0)
newpost thread iconQueries about Robots.txt
  Importance of Robots.txt
2Monalisa 5:24 pm on Nov 17, 2008 (gmt 0)
newpost thread iconDisallow Yahoo to block site explorer
 
9minnapple 2:23 am on Nov 14, 2008 (gmt 0)
newpost thread iconMatching patterns in robots.txt
 
3kiransarv 2:39 pm on Nov 9, 2008 (gmt 0)
newpost thread iconhow to block ALL bots?
 
18Makaveli2007 10:50 am on Nov 9, 2008 (gmt 0)
newpost thread iconGoogle has a new robots.txt for today only!
 
4travelin_cat 8:56 pm on Oct 31, 2008 (gmt 0)
newpost thread iconPreventing a page from being indexed but wanting links to be followed
 
3realmaverick 5:24 pm on Oct 21, 2008 (gmt 0)
newpost thread iconDisallow .php in root folder but allow in sub.
 
3mysticalsock 5:19 pm on Oct 21, 2008 (gmt 0)
newpost thread iconrobot.txt restricted but still showing !
 
9member22 2:05 pm on Oct 21, 2008 (gmt 0)
newpost thread iconrobot.txt blocked pages but still alive
 
3member22 1:27 pm on Oct 21, 2008 (gmt 0)
newpost thread iconDisallow: /*? - is it ok ?
 
6nex99 1:22 pm on Oct 21, 2008 (gmt 0)
newpost thread iconhow to block indexing of root domain, but still allow sub-domain
  how to block indexing of root domain, but still allow sub-domain
6chazeo 1:20 pm on Oct 21, 2008 (gmt 0)
newpost thread iconBlock directory but not lower level folder
  robots.txt block directory
2Clicknowdomains 1:04 pm on Oct 21, 2008 (gmt 0)
newpost thread iconrobots.txt
 
2kiransarv 1:00 pm on Oct 21, 2008 (gmt 0)
newpost thread iconSub-domain & crawl-delay
 
2foxfox 4:39 pm on Oct 7, 2008 (gmt 0)
newpost thread iconCrawl Rate: How do I know when you request slow versus fast
  Does a long maximum time spent crawling a page have adverse impact on targe
211364guy 7:50 pm on Oct 6, 2008 (gmt 0)
newpost thread iconNon-white list blocking
  With a sting
4phred 12:51 pm on Sep 30, 2008 (gmt 0)
Displaying Page 21 of 71<< Back 1 ... 11 12 13 14 15 16 17 18 19 20 [ 21 ] 22 23 24 25 26 27 28 29 30 ... 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved