semrush/whorush

web marketing tools

[Dijkgraaf]

UA: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3

IP: 88.214.193.166
rDNS: [No reverse DNS entry per ns1.ipipe.net.]
Robots.txt: No

Referrers: (my domain replaced with example.com and space added)
http:// www.semrush.com/info/example.com
http:// www.whorush.com/search/?q=example.com

Behaviour: Only requests root page

A I haven't subscribed to their marketing tools, they don't request robots.txt and don't declare themselves in the UA I consider it a bot.

[jdMorgan]

Seen them too. Same IP address, same behavior.

Since it was the exact same IP address, this makes it unlikely that we are seeing a user of the service checking out our sites, as opposed to the service's own 'bot. The chances of someone being interested in your site and mine are pretty low, no matter what our sites are about...

The HTTP Connection, Accept, Accept-Language, Accept-Encoding, and Accept-Charset headers are incorrect for Firefox.

Jim

[Pfui]

Same thing here on at least two different days on two disparate sites -- same UA, same bot-runner IP, same log-spamming refs containing the sites' names, etc. Also no robots.txt, no images, no nothing but root.

FWIW, SEMRush is a competitor research-checker and its domain is registered to a Russia-based SEO endeavor. The bot-runner IP hails from a UK-based server farm (Hqhost), ditto kin SeoQuake, SeoDigger, SeoPivot, and SEMIsland. WHORush and PicsDigger are also connected to the same data mining/selling mix, and who knows how many more.

ANYway, all of the above earned 88.214.193.166 a spot in my firewall's killfile and 403s for the self-refs.

(I'm weary of mod_rewrite-403'ing bot-runners only to see them try and try again anyway.)