New one on me, and didn't come up in Forums search.
88.198.48.46 - - [08/Apr/2015:01:33:48 -0700] "GET /robots.txt HTTP/1.1" 301 528 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +https://www.megaindex.ru/?tab=linkAnalyze)"
That's a Hetzner range, to save everyone the lookup, meaning that all other requests were blocked. I felt bad about locking out a seemingly compliant robot, until I looked closer and established that
-- the robots.txt request was
preceded by a pair of 403'd requests (presumably with and without www) for the front page
-- the redirected robots.txt request was never followed-up with a 200 request to the correct name
-- the very next request after robots.txt was for a page in a roboted-out directory
and, finally,
-- a visit to the URL in the UA leads to a redirect to a "create account" page.
Psst! Botrunners! Somewhere there's a list of Top Ten Ways To Get Yourself Blocked.
(In case anyone wondered: My 403 page is made for humans. Apparently this robot actually read it, instead of just noting the 403 response, because it proceeded to request all pages linked from it. No skin off my nose.)
