Can I exclude a parameter in a robots.txt? For example,
User-agent: * Disallow: *path=*
jimbeetle
2:23 pm on May 15, 2007 (gmt 0)
Both Google [google.com] and Yahoo [help.yahoo.com] support wildcards (and to a certain extent, for file formats, also Live). I would give very serious thought before implementing this because of the lack of support among other bots.
g1smd
9:22 pm on May 18, 2007 (gmt 0)
You need to put those instructions in a section of robots.txt especially for Googlebot.
Be aware that if the Googlebot section is present, Google does NOT read any other sections of the robots.txt file; especially the User-agent: * section of the file.
