lucy24 - 4:39 pm on Sep 16, 2013 (gmt 0)
I found an unusual cluster of 403's recently. Turned out to be the bingbot-- random mix of 157.56 and 65.55 ranges-- requesting every .zip and .sit file I own. (Both are generically blocked to anything with "bot" or "crawler" in the name.) They're two sets of files in widely separated directories with completely unrelated content, all within a ten-minute period, so it was clearly a batch request. Further searching turns up a previous similar attempt in early August.
Ordinarily when bing does something, you expect to find google doing the same thing, only four times as much. But here there's absolutely nothing. The only other robot with a consistent interest in downloadable files is mail.ru.
Anyone know what they're looking for? pdf I can understand, because most browsers can read them. But .zip and .sit have no existence except as downloads. Does .sit even exist on That Other Platform? Is there a secret army of classic Mac OS emulators somewhere in Redmond?