Welcome to WebmasterWorld Guest from 54.221.28.179

Forum Moderated by: goodroi

Sitemaps, Meta Data, and robots.txt


Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
Thread SubjectMessagesStarted byLast Message
disallowing url midway
2 defanjos 5:20 pm Jul 29, 2007
Block sub directory
5 maxbear 5:24 am Jul 28, 2007
Does this work?
disallow multiple bots at once...
3 Devilfear 6:48 am Jul 27, 2007
Using multiple hash marks
Does that conform to protocol?
5 Reno 7:17 pm Jul 25, 2007
Disallowing Iframed pages
2 Ozz13 1:29 pm Jul 23, 2007
Proper format for disallow of URL
4 Ozz13 1:14 pm Jul 23, 2007
What the dilly?
3 tonynoriega 5:39 pm Jul 18, 2007
Re-instate pages previously blocked
sorry for this noob question
4 dibbern2 11:23 pm Jul 16, 2007
Broken Links on my web site
I have too many broken links
2 erdsah88 12:47 pm Jul 16, 2007
"Disallow: //" - error or meaningful?
robots.txt crawler error
5 sitetruth 11:58 pm Jul 11, 2007
BLOX dynamic urls with robots.txt
BLOX dynamic urls with robots.txt
2 bo0oost 11:50 am Jul 11, 2007
Block affiliate tracking
but keep the main page in Google
5 chubba 2:26 pm Jul 9, 2007
We can put sitempp.html to robots.txt?
need some advise
2 etgsgroup 12:00 pm Jul 6, 2007
A question about blocking IP from Nigerians
2 rjr686 10:56 pm Jul 3, 2007
Google and cached, obsolete robots.txt
5 MamaDawg 10:59 am Jul 3, 2007
Idbot
IDBOT hammered our site, now banned
1 JohnCanyon 5:21 pm Jun 29, 2007
Disallow a single image?
Is it possible to block the search engines from indexing a single image?
4 PrattTA1 4:59 pm Jun 26, 2007
Blocking Site Server 3.0 Robot
How?
13 Matt_Probert 5:53 am Jun 23, 2007
A list of american search engines.
search engine list. American.
9 Michel_Samuel 12:49 am Jun 22, 2007
Does this trailing slash mean anything?
<meta name="robots" content="noindex" />
11 bouncybunny 7:10 pm Jun 20, 2007
Need help with a specific robots.txt question
4 Kurgano 9:18 pm Jun 12, 2007
about:blank >>>what is this
then the page loads?
4 twtnyc 11:44 pm Jun 8, 2007
Allow rather than Disallow
Is it possible?
5 Marshall 10:30 pm Jun 7, 2007
Why is msnbot ignoring this robot's.txt rule?
Wildcard disallow ignored
3 bouncybunny 12:17 pm Jun 6, 2007
Block all Nutch variants. Possible?
6 morags 9:02 pm Jun 5, 2007

Featured Threads

Hot Threads This Week

Hot Threads This Month