Clarabot

What's heppened to it?

[dstiles]

I've noticed an increase in visits from clarabot this week but can no longer find clarabot[.]info and its DNS is screwed. Nor can I find any mention online of clara as a search engine; I seem to recall it was crawling for something else?

I was looking initially to find a feedback for them - their bot is dropping into a robots.txt "blocked" page.

[lucy24]

It wouldn't be the first time some bad agent has glommed onto the name of a legitimate robot. Remember when everyone was calling themselves Googlebot?

:: detour to logs ::

Hm. Scattering of robots.txt immediately followed by (blocked) request for root. But I've only just started denying them, so can’t make any further conclusions.

My 403 page is intended for humans, and therefore includes links to the main directories, plus boilerplate such as the contact page. So far I haven't seen Clarabot requesting them, although it is in their power to learn of the URLs’ existence.

Are all of yours coming in from 37.17.172.128-159 (best guess about /27 based on IPs in logs)? Servers in Hungary, I think.

[dstiles]

This is within that clara range, yes. I'm concerned about the clarabot site being offline and with faulty dns. Wondering if they have a) been hijacked or b) taken over. Odd that I can no longer find any reference to the search engine itself, too, or at least to an engine using clarabot.

I'm inclined to close access to it for a while, either until the clarabot site comes back online or forever if it doen't.

[lammert]

Clarabot is/was a backlink tracker from Hungary. According to whois, the DNS is still registered to a Hungarian registrant and the mentioned IP addresses are also in Hungary. That would imply that the requests could be genuine. The only useful information I have found is on the Hungarian Wikipedia site [hu.wikipedia.org]. When translated with Google Translate it reads somewhat like an advertisement and the topic was mainly written by one contributor so I am not sure how unbiased that information is.

[dstiles]

Thanks, lammert. When I originally whitelisted it the bot seemed genuine enough but I have a nasty feeling it's been subverted somehow. Anyway, now blocked until I can find a reason not to.

[dstiles]

Just found clarabot[.]com seems to be back but with a bad certificate...

"www.clarabot.com uses an invalid security certificate.
"The certificate is not trusted because it is self-signed.
"The certificate is only valid for web01-dns2.szerverplex.hu"

I did not try it out!

clarabot[.]info is still dead.