Welcome to WebmasterWorld Guest from 54.166.198.6

Forum Moderated by: goodroi

Sitemaps, Meta Data, and robots.txt


Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
Thread SubjectMessagesStarted byLast Message
robots.txt Disallow query
Disallow specific query in Robots.txt
4 frogz 10:35 pm Jan 9, 2009
Robots.txt - Working or not ?
10 glennk 12:15 pm Jan 9, 2009
Blocking bots
6 ag_47 4:58 am Jan 8, 2009
Indexing and Crawling to Dynamic pages
Would google Crawl Dynamic CMS
3 mick_simon 2:55 am Jan 7, 2009
Disallow in Robots against guidelines?
4 cgrant 4:28 pm Jan 6, 2009
How to block this type of URL's
9 iCyborg 9:32 pm Jan 4, 2009
Having Trouble With an Allow Instruction.
Trying to adapt an example given by jdMorgan.
11 Durnovaria 1:43 pm Jan 2, 2009
If a dir is disallowed, will a sitemap in a subdir be read?
Sitemaps located in non-content directories
7 badbadmonkey 8:45 pm Jan 1, 2009
Disallow pdf
13 FiRe 11:34 am Dec 22, 2008
I want entire site indexed
7 brnm98105 8:38 pm Dec 19, 2008
blocking variables
2 fishingfriend 5:45 pm Dec 15, 2008
How to block a subdomian
Where to put the file
5 Musicarl 5:39 am Dec 15, 2008
A bit of help if possible would be appreciated.
8 glennk 7:52 pm Dec 14, 2008
Blocking Email and Print only URLS
2 himalayaswater 12:14 am Dec 7, 2008
Robots.txt - Site in a Directory
10 Sarao 10:14 pm Dec 6, 2008
nofollow noindex or disallow?
4 Gemini23 12:09 am Dec 6, 2008
Allowing Adsence bot
Googled emailed me asking to allow the adsence bot
5 alahamdan 2:02 pm Dec 5, 2008
Possible to block all urls with ? in them?
2 texasville 1:28 pm Dec 5, 2008
How is correct code within robots.txt?
4 toplisek 11:07 am Nov 29, 2008
disallowing a variable
4 KevinC 10:37 pm Nov 17, 2008
Queries about Robots.txt
Importance of Robots.txt
2 Monalisa 5:24 pm Nov 17, 2008
Disallow Yahoo to block site explorer
9 minnapple 2:23 am Nov 14, 2008
Matching patterns in robots.txt
3 kiransarv 2:39 pm Nov 9, 2008
how to block ALL bots?
18 Makaveli2007 10:50 am Nov 9, 2008
Google has a new robots.txt for today only!
4 travelin_cat 8:56 pm Oct 31, 2008

Featured Threads

Hot Threads This Week

Hot Threads This Month