SearchBot

[keyplyr]

UA: SearchBot
robots.txt: yes

Took 1 page per second. Crawled entire site. No robot info page.

rDNS: ns1.rostelecom.ru
Electronic government of the Russian Federation
109.207.0.0 - 109.207.15.255
109.207.0.0/20

I'm going to deny it in robots.txt and see what happens.

[tangor]

If you are whitelisting, ie. listing allowed bots and disallowing all others, then most likely even if specifically denied in robots.txt it will not comply. Just my experience, YMMV!

Meanwhile, among my .htaccess rules is .ru which has been a big help over the years.

[keyplyr]

Thanks for your input tangor. I get increasing traffic from Russia mostly due to Yandex. Yes there is a significant amount of villainous activity from .ru, but I decided to take it on a case-by-case basis.