Removing specific file type

[MisterT]

Google gives the below instructions for removing a specific file type. (.gif in this example)

User-agent: Googlebot
Disallow: /*.gif$

I'm guessing this would work with all engines? For example, to remove/prevent .gif files from all engines, I would use:

User-agent: *
Disallow: /*.gif$

Am I right?

[goodroi]

Using wildcards in the disallows was not part of the original 1994 robotx.txt protocol. Most robots will not handle these wildcards correctly. This is more of a special Google exception as opposed to an industry wide acceptance.

[MisterT]

Ok, thanks for the info.

Would Google understand this particular wildcard?

[goodroi]

Hi MisterT,

Google would understand this wildcard. It is better to use this type
User-agent: Googlebot
Disallow: /*.gif
to make sure googlebot sees it.

[MisterT]

isn't googlebot smart enough to understand the * wildcard?

i thought it was just other less sophisticated bots that might have trouble?

is there any harm in having both?

[goodroi]

googlebot will understand the wildcard, my preference is to always make it as easy as possible for the bots, that is why i would add a specific googlebot section. just my personal preference.

[MisterT]

any idea how long it might take googlebot to respond to something like this?

User-agent: Googlebot
Disallow: /*.gif

in this case, how long after the bot sees that for all .gif files to be removed from the google index? i'm sure this varies, but anyone have a specific experience? this is the first time i've tried doing this. thanks!

[MisterT]

anyone know how long google takes to respond to this sort of thing?