Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

How can I block these kinds of urls?

11:44 am on Aug 1, 2011 (gmt 0)

New User

10+ Year Member

joined:May 26, 2005
posts: 35
votes: 0


My forum based site is getting hit hard by google duplicate content filters. I have pages such as the two examples below being indexed.


How can I block pages such as /my-forum/discussion/*/?Focus*

Is that valid? If not how can I fix it please?
12:12 pm on Aug 1, 2011 (gmt 0)

Senior Member from NL 

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Sept 25, 2005
votes: 153

I would recommend implementing the canonical tag [googlewebmastercentral.blogspot.com], the effect being similar to a 301 redirect, over the blocking of pages via robots.txt.
1:45 am on Aug 2, 2011 (gmt 0)


WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 10, 2004
votes: 43

if those urls return the same content and assuming /my-forum/discussion/416/discussion-name/ is the canonical url the proper technical solution would be to respond to the request for /my-forum/discussion/416/?Focus=6222 with a 301 redirect to the canonical url.
if you use robots.txt to exclude crawling of those urls they may appear in search results as url-only snippets, possibly with a title based on the anchor text of the inbound link.
also note that a 301 redirect is incontrovertible while rel canonical is a "suggestion" not a directive.
About rel="canonical":
http://www.google.com/support/webmasters/bin/answer.py?answer=139394 [google.com]

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members