Welcome to WebmasterWorld Guest from 107.20.5.156

Forum Moderators: goodroi

Message Too Old, No Replies

Wildcard use in robots.txt

Need to disallow spidering of graphics files

     

Probably

9:57 pm on Jan 25, 2004 (gmt 0)



I want to disallow the spidering/caching of graphics formats such as *.gif, *.jpg etc., and wonder how to go about it. I tried entering "/*.gif" rules, but the robots.txt validator warns that this is non-standard, and one can only expect Google to get it. So what is the clever way?

mbauser2

4:58 am on Jan 26, 2004 (gmt 0)

10+ Year Member



There is no clever way.

The functional way is to put all your graphics in a graphics-only directory, then prohibit robot access to that directory.