dstiles - 8:43 pm on Mar 16, 2010 (gmt 0)
pageoneresults - yes, I noticed it. I put it down to just another google discourtesy: grabbing more pages to boost their publicised stats (we're the best!) whilst ignoring the robots.txt spec - or at least mis-interpreting it.
I get over it by adding a 405 to pages they ain't supposed to visit for detected non-browsers: ie bots. That also copes with quite a few scrapers and form-spammers.