lucy24 - 7:01 am on Nov 26, 2012 (gmt 0)
They fetch complete packages. In my case this means that every request is followed by a fetch of errorstyles.css. Every single time. Which in turn means they actually go to the 403 page, rather than swallowing the numbers and moving on. Very un-robotlike. Rarely they also get the favicon.
Random hopping through raw logs brought up only one request for robots.txt, and that was in October. Of 2011. (I later found a day in March where all they asked for was robots.txt. Two redirects, four successful pickups. I guess they freeze them for later.)
Along the way I was staggered to discover that they've been steadily asking for the same handful of pages over and over again, several times a day. Maybe they lost their shopping list and these are the only titles they can remember.
I have also now remembered that it was YahooCacheSystem that originally offended me. Slurp is just along for the ride. Makes no difference to searches, since they don't do their own crawling.