Welcome to WebmasterWorld Guest from 188.8.131.52 , register , free tools , login , search , subscribe , help , library , announcements , recent posts , open posts Subscribe to WebmasterWorld
Robots.txt Can you exclude a parameter in robots.txt? North_Shore msg:3340043 1:49 pm on May 15, 2007 (gmt 0) Can I exclude a parameter in a robots.txt? For example,
jimbeetle msg:3340061 2:23 pm on May 15, 2007 (gmt 0)
Both Google [ google.com] and Yahoo [ help.yahoo.com] support wildcards (and to a certain extent, for file formats, also Live). I would give very serious thought before implementing this because of the lack of support among other bots. g1smd msg:3343472 9:22 pm on May 18, 2007 (gmt 0)
You need to put those instructions in a section of robots.txt especially for Googlebot.
Be aware that if the
Googlebot section is present, Google does NOT read any other sections of the robots.txt file; especially the User-agent: * section of the file.