Welcome to WebmasterWorld Guest from 54.198.164.83

Forum Moderators: Ocean10000 & incrediBILL & keyplyr

Message Too Old, No Replies

Findxbot

new bot on the hunt

     
9:30 am on Feb 13, 2015 (gmt 0)

New User

5+ Year Member

joined:Apr 25, 2012
posts: 40
votes: 0


was visited today by a new bot- Findxbot. the web site says the bot is "still in stealth mode"?

anyone know anything about Findxbot?
6:58 pm on Feb 13, 2015 (gmt 0)

Junior Member

Top Contributors Of The Month

joined:Jan 26, 2014
posts:197
votes: 0


They've been dressing as:

GigablastOpenSource/1.0

until recently. Now apparently as:

Mozilla/5.0 (compatible; Findxbot/1.0; +http://www.findxbot.com)


They seem to be from Denmark. Their address range is on their website:
77.66.121.235 - 77.66.121.244

They apparently didn't have reverse DNS set up until very recently.

There are threads about gigablast/gigabot and GigablastOpenSource on this site, which may or may not be relevant.

Other than that, not a lot.
2:56 pm on Feb 14, 2015 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 27, 2004
posts:1949
votes: 60


yes, blocked it at HELLO.

FIND I get, BOT I get, but wondering what X stands 4 in UA, left in a middle...
7:43 pm on Feb 14, 2015 (gmt 0)

Moderator This Forum from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:11105
votes: 659



FIND I get, BOT I get, but wondering what X stands 4 in UA, left in a middle...

Typically X is used as a wild card, so Find anything_here Bot.
1:36 am on May 5, 2015 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Nov 5, 2005
posts: 2038
votes: 1


Still "in stealth mode" -- and now just wasting my time and resources:

crawl-77.66.121.240.findxbot.com (77.66.121.240)
Mozilla/5.0 (compatible; Findxbot/1.0; +http://www.findxbot.com)

14:06:47 /robots.txt
14:06:53 /robots.txt
15:09:28 /robots.txt
15:41:04 /robots.txt

No, thanks.
4:38 pm on May 5, 2015 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:14597
votes: 595


:: detour to raw logs ::

All it ever does is ask for robots.txt and then the front page. Honestly I don't think that's worth the trouble of blocking. Factor in the several seconds it would take to add a Deny From or BrowserMatch line (once) and the added nanoseconds for the server (on every request ever, in perpetuity) -- and for what?
8:16 pm on May 5, 2015 (gmt 0)

Moderator This Forum from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:11105
votes: 659


All it ever does is ask for robots.txt and then the front page

Sounds like most bots :)
11:15 pm on May 5, 2015 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:14597
votes: 595


ymmv, but if all they ever ask for is the front page, I don't even particularly care if they asked for robots.txt or not. That is, a previously unknown robotic visitor from a not-otherwise-blocked IP only warrants further action if (a) they asked for any interior page or (b) they show up with an ostentatiously fake* UA.


* Where "fake" includes giving an URL that leads to anything other than a currently valid informational page in some major European language.
3:20 am on May 6, 2015 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:14597
votes: 595


Oh yes and...
At the time I posted the above, I forgot that I'd had another visit from Findxbot just a couple of days ago. This time, over the course of twenty minutes, they asked for
-- robots.txt
-- one directory
-- other directory (this is my personal site, which only has two)
The funny part is that when I first looked at the record, I only considered the last part of the timestamp, and thought they had done all this in the course of ten seconds-- which would still have been perfectly acceptable for a three-request visit.
9:19 am on May 9, 2015 (gmt 0)

Moderator This Forum from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:11105
votes: 659


So FindXBot was formerly known as Gigabot, which at one time I was allowing then they were sold and re-purposed so I started blocking them.

Anyway, FindXBot says they are now coming from 77.66.121.235 - 77.66.121.244, which is part of a German server farm I have blocked:

netgroup.dk
77.66.0.0/17
77.66.0.0 - 77.66.127.255
6:17 pm on May 9, 2015 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:14597
votes: 595


German
<snip>
.dk
Ahem, cough-cough.
10:55 pm on May 9, 2015 (gmt 0)

Junior Member

5+ Year Member

joined:Jan 25, 2011
posts:51
votes: 0


NetGroup AS has servers in different countries.

Below are all their full IPv4 and IPv6 ranges:

IPv4 List:
77.66.0.0/17
77.66.70.0/24
79.134.165.0/24
79.134.185.0/24
83.221.128.0/19
89.186.160.0/19
91.236.38.0/23
91.236.44.0/24
194.126.180.0/22
195.162.80.0/22
195.182.22.0/24
195.74.72.0/24
217.116.224.0/19




IPv6 List:
2001:067c:2618::/48
2001:1448:000a:8300::/64
2001:1448:0208::/48
2001:1448::/32
11:02 pm on May 9, 2015 (gmt 0)

Moderator This Forum from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:11105
votes: 659


Ahem, cough-cough.

Surely you aren't assuming that because they use a Denmark TLD they are an exclusively Denmark owned company? I mean they they are NetGroup Data Centers :)

[edited by: keyplyr at 11:10 pm (utc) on May 9, 2015]

11:04 pm on May 9, 2015 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member brotherhood_of_lan is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Jan 30, 2002
posts:4912
votes: 22


>but wondering what X stands 4 in UA, left in a middle..

Perhaps expired. Contextually relevant expired domains are somewhat popular and there's a number of services that crawl the web looking for references to them.
6:36 pm on May 13, 2015 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:May 14, 2008
posts:3179
votes: 8


mrtonyg - only three of those ranges are Danish Netgroup Datacenter. The others are Belgian and Ukraine and have different netnames. Thanks for the list, though. :)
7:23 pm on May 13, 2015 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 27, 2004
posts:1949
votes: 60


Perhaps expired.

Well so far 5 hits since the middle of Feb on the domain that still has 5 years left on Reg.

...waiting when findYbot comes along.... :)
...does Z still counts for Zombie?...
8:59 pm on May 13, 2015 (gmt 0)

Moderator This Forum from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:11105
votes: 659


Until they publish who exactly they are, what data they mine & what they do with it they will stay blocked from my sites.

...does Z still counts for Zombie?...

Yeah... but no Crawl-delay: necessary.