Forum Moderators: goodroi

Message Too Old, No Replies

BLOX dynamic urls with robots.txt

BLOX dynamic urls with robots.txt

         

bo0oost

6:01 am on Jul 10, 2007 (gmt 0)

10+ Year Member



Say for example I have a file that google indexes called:

[test.com...]

How can I block all of the dynamic possibilities (C=1, C=2, etc).

I have searched and found different answers:

Disallow: /widgets.php?

and also

Disallow: /widgets.php?C=*

Which would be correct in the robots.txt file, or would they both be correct?

Thanks in advance.

goodroi

11:50 am on Jul 11, 2007 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



Please note this is google specific robots.txt advice since wildcards are not officially supported by the robots.txt standard.

Since your variations all seem to have C=. You can use this line to block all google from all urls with C=.
Disallow: /*C=*