Findxbot

new bot on the hunt

[dupres01]

was visited today by a new bot- Findxbot. the web site says the bot is "still in stealth mode"?

anyone know anything about Findxbot?

[trintragula]

They've been dressing as:

_{GigablastOpenSource/1.0}

until recently. Now apparently as:

_{Mozilla/5.0 (compatible; Findxbot/1.0; +http://www.findxbot.com)}


They seem to be from Denmark. Their address range is on their website:
_{77.66.121.235 - 77.66.121.244}

They apparently didn't have reverse DNS set up until very recently.

There are threads about gigablast/gigabot and GigablastOpenSource on this site, which may or may not be relevant.

Other than that, not a lot.

[blend27]

yes, blocked it at HELLO.

FIND I get, BOT I get, but wondering what X stands 4 in UA, left in a middle...

[keyplyr]

FIND I get, BOT I get, but wondering what X stands 4 in UA, left in a middle...

Typically X is used as a wild card, so Find anything_here Bot.

[Pfui]

Still "in stealth mode" -- and now just wasting my time and resources:

crawl-77.66.121.240.findxbot.com (77.66.121.240)
Mozilla/5.0 (compatible; Findxbot/1.0; +http://www.findxbot.com)

14:06:47 /robots.txt
14:06:53 /robots.txt
15:09:28 /robots.txt
15:41:04 /robots.txt

No, thanks.

[lucy24]

:: detour to raw logs ::

All it ever does is ask for robots.txt and then the front page. Honestly I don't think that's worth the trouble of blocking. Factor in the several seconds it would take to add a Deny From or BrowserMatch line (once) and the added nanoseconds for the server (on every request ever, in perpetuity) -- and for what?

[keyplyr]

All it ever does is ask for robots.txt and then the front page

Sounds like most bots :)

[lucy24]

ymmv, but if all they ever ask for is the front page, I don't even particularly care if they asked for robots.txt or not. That is, a previously unknown robotic visitor from a not-otherwise-blocked IP only warrants further action if (a) they asked for any interior page or (b) they show up with an ostentatiously fake* UA.


* Where "fake" includes giving an URL that leads to anything other than a currently valid informational page in some major European language.

[lucy24]

Oh yes and...
At the time I posted the above, I forgot that I'd had another visit from Findxbot just a couple of days ago. This time, over the course of twenty minutes, they asked for
-- robots.txt
-- one directory
-- other directory (this is my personal site, which only has two)
The funny part is that when I first looked at the record, I only considered the last part of the timestamp, and thought they had done all this in the course of ten seconds-- which would still have been perfectly acceptable for a three-request visit.

[keyplyr]

So FindXBot was formerly known as Gigabot, which at one time I was allowing then they were sold and re-purposed so I started blocking them.

Anyway, FindXBot says they are now coming from 77.66.121.235 - 77.66.121.244, which is part of a German server farm I have blocked:

netgroup.dk
77.66.0.0/17
77.66.0.0 - 77.66.127.255

[lucy24]

German
<snip>
.dk

Ahem, cough-cough.

[mrtonyg]

NetGroup AS has servers in different countries.

Below are all their full IPv4 and IPv6 ranges:

IPv4 List:
77.66.0.0/17
77.66.70.0/24
79.134.165.0/24
79.134.185.0/24
83.221.128.0/19
89.186.160.0/19
91.236.38.0/23
91.236.44.0/24
194.126.180.0/22
195.162.80.0/22
195.182.22.0/24
195.74.72.0/24
217.116.224.0/19




IPv6 List:
2001:067c:2618::/48
2001:1448:000a:8300::/64
2001:1448:0208::/48
2001:1448::/32

[keyplyr]

Ahem, cough-cough.

Surely you aren't assuming that because they use a Denmark TLD they are an exclusively Denmark owned company? I mean they they are NetGroup Data Centers :)

[edited by: keyplyr at 11:10 pm (utc) on May 9, 2015]

[brotherhood of LAN]

>but wondering what X stands 4 in UA, left in a middle..

Perhaps expired. Contextually relevant expired domains are somewhat popular and there's a number of services that crawl the web looking for references to them.

[dstiles]

mrtonyg - only three of those ranges are Danish Netgroup Datacenter. The others are Belgian and Ukraine and have different netnames. Thanks for the list, though. :)

[blend27]

Perhaps expired.

Well so far 5 hits since the middle of Feb on the domain that still has 5 years left on Reg.

...waiting when findYbot comes along.... :)
...does Z still counts for Zombie?...

[keyplyr]

Until they publish who exactly they are, what data they mine & what they do with it they will stay blocked from my sites.

...does Z still counts for Zombie?...

Yeah... but no Crawl-delay: necessary.