Welcome to WebmasterWorld Guest from 23.20.221.93

Forum Moderators: goodroi

Message Too Old, No Replies

Dissallowing Dynamic Content

Blocking "index.php?"

     
2:14 pm on Oct 25, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member trillianjedi is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Apr 15, 2003
posts:7242
votes: 0


I'm sure I have asked this before, a couple of years ago, but now can't find the thread ;)

I've been handed a site to fix which has some duplicate content issues. I need to block everything which starts with:-

/index.php?

The "?" is important as I don't want to block:-

www.example.com/index.php

I just want to block anything that looks like:-

www.example.com/index.php?myname=fred
www.example.com/index.php?hisname=jim

etc.

The key question is can I use the "?" as a wildcard to block all dynamic content in this way?

I suppose a secondary question would be is there an adhered standard by the SE's in this respect? Or should I expect some weirdness from a. n. other engine (which I won't care about too much as long as it works in you-know-which-one).

I note that Google allows the use of an asterisk as a wildcard, although at first glance there appears to be a typo on their page about this which makes me a little nervous:-

To block access to all URLs that include a question mark (?), you could use the following entry:

User-agent: *
Allow: /*?*

From : [google.com...]

Have I mis-interpreted that, or is that a google typo and "allow" should be "disallow"?

Thanks!

TJ

6:58 pm on Oct 25, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:Nov 17, 2005
posts:44
votes: 0


You are absolutely right about the typo. It's been fixed and the examples shown on the help page are now correct. I'm sorry about the confusion.