Forum Moderators: open
Requested robots.txt then disobeyed it by taking 7 html files, stopping to get robots.txt again, and resuming with requests for another 7 webpages. Their crawler info page says they support robots.txt, so possibly there's a problem with their bot. I've sent them an email, and am now banning UA via mod_rewrite.