goodroi
msg:4163945 | 1:32 pm on Jul 3, 2010 (gmt 0) |
wildcards aka pattern matching is not officially part of the robots.txt protocol. this means most of the big search engines support it but most of the smaller one won't.
According to Google's page [google.com...]
To match a sequence of characters, use an asterisk (*). For instance, to block access to all subdirectories that begin with private:
User-agent: Googlebot
Disallow: /private*/
|
|
|
phranque
msg:4164270 | 10:45 am on Jul 4, 2010 (gmt 0) |
according to the robots exclusion protocol (which doesn't include any wildcard extensions as supported by google) the matching occurs left-to-right and the correct option would be:
/search/?t
|
chms
msg:4164334 | 3:14 pm on Jul 4, 2010 (gmt 0) |
Without * at the end?
|
Dijkgraaf
msg:4164435 | 9:20 pm on Jul 4, 2010 (gmt 0) |
Yes, without the * at the end.
The standard is that any URL which starts with the string you specified will be matched.
|
chms
msg:4164436 | 9:36 pm on Jul 4, 2010 (gmt 0) |
Ok, thank you
|
chms
msg:4165791 | 2:08 pm on Jul 7, 2010 (gmt 0) |
Hello,
Finally Google took the wildcards.
Thank you
|
|
