Pfui - 6:31 pm on Sep 10, 2011 (gmt 0)

For years -- YEARS -- I've denied Slurp all graphics in robots.txt and I just presumed it was heeding the restriction.

Wrong.

Depending on the Host and UA, the official Yahoo! Slurp apparently does whatever it wants to. Note the subtle differences in the subdomains and UAs...

This morning, the only Host to read/heed robots.txt was:

b3091154.crawl.yahoo.net [67.195.112.189]
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)

These retrieved graphics by the pageful, over 60 total:

b5101137.yst.yahoo.net [98.137.72.218]
b5101139.yst.yahoo.net [98.137.72.228]
Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp)

I can't say if this is new and/or MSN-related. I can say I'm irked.