Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

Dissallowing Dynamic Content

Blocking "index.php?"

2:14 pm on Oct 25, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member trillianjedi is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Apr 15, 2003
votes: 0

I'm sure I have asked this before, a couple of years ago, but now can't find the thread ;)

I've been handed a site to fix which has some duplicate content issues. I need to block everything which starts with:-


The "?" is important as I don't want to block:-


I just want to block anything that looks like:-



The key question is can I use the "?" as a wildcard to block all dynamic content in this way?

I suppose a secondary question would be is there an adhered standard by the SE's in this respect? Or should I expect some weirdness from a. n. other engine (which I won't care about too much as long as it works in you-know-which-one).

I note that Google allows the use of an asterisk as a wildcard, although at first glance there appears to be a typo on their page about this which makes me a little nervous:-

To block access to all URLs that include a question mark (?), you could use the following entry:

User-agent: *
Allow: /*?*

From : [google.com...]

Have I mis-interpreted that, or is that a google typo and "allow" should be "disallow"?



6:58 pm on Oct 25, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:Nov 17, 2005
votes: 0

You are absolutely right about the typo. It's been fixed and the examples shown on the help page are now correct. I'm sorry about the confusion.