homepage Welcome to WebmasterWorld Guest from 54.204.94.228
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

This 193 message thread spans 7 pages: < < 193 ( 1 2 [3] 4 5 6 7 > >     
Server Farms - April 2014
Tracking and Reporting Data Center IP Ranges
incrediBILL




msg:4660480
 6:51 pm on Apr 4, 2014 (gmt 0)

Continuation of the Server Farm threads.

This is where we report data center IP ranges as they are discovered or change in the rapidly evolving assigned IP landscape.

Past server farm threads:


 

wilderness




msg:4666241
 10:38 am on Apr 27, 2014 (gmt 0)

Many thanks easy.

webcentric




msg:4666267
 3:04 pm on Apr 27, 2014 (gmt 0)

Yes, thanks everyone for identifying and organizing that mess.

keyplyr




msg:4666304
 6:26 pm on Apr 27, 2014 (gmt 0)


Sakura
182.48.0.0 - 182.48.63.255
182.48.0.0/18

keyplyr




msg:4668520
 7:36 am on May 5, 2014 (gmt 0)

ServerDale, Ukraine
93.171.138.0 - 93.171.138.255
93.171.138.0/24


Accelerated, Germany
84.200.0.0 - 84.201.63.255
84.200.0.0/16, 84.201.0.0/18

keyplyr




msg:4669681
 8:02 am on May 9, 2014 (gmt 0)

HostingProd (Inktomi/Yahoo)

66.94.224.0 - 66.94.255.255
66.94.224.0/19

74.6.53.0 - 74.6.53.255
74.6.53.0/24

98.139.128.0 - 98.139.255.255
98.139.128.0/17

any more?

lucy24




msg:4670816
 8:29 pm on May 13, 2014 (gmt 0)

###, clicked Close instead of Post.

204.44.64.0/18
new QuadraNet range: free lookup says 19 March (it used to be someone else), and they've already been recruited into a botnet.*


* Interior page giving front page as referer (this is an automatic 403), followed by Contact page giving first page as referer.

keyplyr




msg:4670866
 12:59 am on May 14, 2014 (gmt 0)

I've had them blocked for a while... I assumed this range was listed here at WW at some point.

Quadranet
204.44.64.0/18
204.44.64.0 - 204.44.127.255

lucy24




msg:4670930
 4:16 am on May 14, 2014 (gmt 0)

I thought so too, but looked and didn't find it. There are long QuadraNet lists in both the current thread and the previous one.

dstiles




msg:4671284
 2:10 pm on May 15, 2014 (gmt 0)

New Google Cloud, announced today by a couple of dozen hits on 13 IPs from something called nerdybot. I'm sure the internet is doomed. :(

107.178.192.0 - 107.178.255.255

not2easy




msg:4671325
 5:29 pm on May 15, 2014 (gmt 0)

Yes, AmazonII. I saw that offer for cheap secure cloud hosting and thought the same thing :(
Build And Host Your Website On Google App Engine
https://cloud.google.com/products/app-engine/

lucy24




msg:4671463
 9:47 pm on May 15, 2014 (gmt 0)

Previously in this thread:
66.117.0.0/20 Corporate Colo
(I missed this one until I met a botnet from China at 66.117.9.210)
Previously in this thread:
66.117.32.0/19 Carpathia Hosting
(either they've got Rules or I've just been lucky, because I have never met these)

At this point the obvious response is to look up
66.117.16.0/20
This turns out to belong to Adobe (in Arcadia, not to be confused with Arcata). So if you don't have the kind of site where people might legitimately browse on their lunch breaks,* you could easily collapse the whole thing to

66.117.0.0/18


* Someone on these forums once suggested that every time a site demands an email address, you should create one specific to that entity. As a result, I now receive a fair amount of spam addressed to "adobe@example.com". Quick detour tells me that memory was faulty and Adobe does not make Fontographer, which would have been the only justification for poking a hole.

keyplyr




msg:4671486
 10:53 pm on May 15, 2014 (gmt 0)

66.117.0.0/18 :)

dstiles




msg:4671772
 8:22 pm on May 16, 2014 (gmt 0)

And another google cloud range - nerdybot again. Are they hoovering up short IP ranges from elsewhere?

199.223.232.0 - 199.223.239.255

not2easy




msg:4671774
 8:53 pm on May 16, 2014 (gmt 0)

The nerdybot/GoogleCloud list is building here too: [webmasterworld.com...]

keyplyr




msg:4671781
 9:56 pm on May 16, 2014 (gmt 0)

And another google cloud range - nerdybot again. Are they hoovering up short IP ranges from elsewhere?

199.223.232.0 - 199.223.239.255


For that one I have:
199.223.232.0 - 199.223.239.255
199.223.232.0/21

[added]
Whoops... I see now they are the same :)

dstiles




msg:4671938
 6:55 pm on May 17, 2014 (gmt 0)

223 and 232 always confuse me, too. :)

lucy24




msg:4672107
 9:40 pm on May 18, 2014 (gmt 0)

Didn't find this in site search:

5.199.172.0/22
within
5.199.160.0/20
Some tangle of Russian (LoyaltyServers) and Lithuanian (BalticServers) sublets that I don't feel like dealing with. Just met a robot from 5.199.175.174.

Everyone's got their own map. On mine, Russia is on the "one strike you're out" side of the line, while Lithuania is on the "let's at least look them up" side. I happened to have this one listed as Lithuania.

Requests for the front page generally get a free ride unless there are aggravating factors-- in this case a wholly gratuitous auto-referer.

keyplyr




msg:4673706
 11:49 pm on May 22, 2014 (gmt 0)

Zayo (formerly Abovenet)

Here's the deal... These ranges definitely house servers, I block bot traffic from Zayo ranges all the time but the company also provides a huge amount of connectivity for client employees which is also a significant part of my traffic. There lies the enigma; to continue to block, or open it up and block/filter the best I can by UA?



64.124.0.0 - 64.125.255.255
64.124.0.0/15

209.66.64.0 - 209.66.127.255
209.66.64.0/18

209.133.0.0 - 209.133.127.255
209.133.0.0/17

209.249.0.0 - 209.249.255.255
209.249.0.0/16

lucy24




msg:4673712
 12:51 am on May 23, 2014 (gmt 0)

UA seems the way to go if there are a lot of desirable humans around. Is it one of those companies where everyone's UA is identical down to the last .NET CLR ... or the opposite kind, where no two computers are alike because they're all hand-me-downs from some more affluent branch of the business?

You can also look at robotic behaviors like questionable referers.

keyplyr




msg:4673719
 1:53 am on May 23, 2014 (gmt 0)

Lots of different bots with different UAs from the hundreds of different companies that have web sites on the Zayo servers across those ranges.

wilderness




msg:4673761
 4:29 am on May 23, 2014 (gmt 0)

keyplr,
I've a lot going on and not exactly firing on all eight.

You should be able to use a multiple condition mod_rewrite focusing on the header that bots fail to use and those multiple IP's.

keyplyr




msg:4673832
 9:28 am on May 23, 2014 (gmt 0)

I already am blocking these (and all) bots using a filter of header fields, UA and IP, as well as blocking the usual suspect behavior typical with bots (admin, wp-, login, etc.) I use a php script for the header check.

However, many do *not* fail the header check and do a good job of spoofing a human. That's the grey area I'm concerned with.

I'm tempted to just block the ranges and accept the collateral damage, but there must be 10s of thousands of employees at all the companies using fiber from Zayo.

I'll write a bit of code to compile the server history from these ranges and watch for a while.

lucy24




msg:4674711
 11:35 pm on May 26, 2014 (gmt 0)

This was somehow missing from the latest DataShack list [webmasterworld.com] (first post on page) so I'm repeating it here:

63.141.224.0/19

keyplyr




msg:4674722
 1:51 am on May 27, 2014 (gmt 0)

Had that one but thanks for the reminder to give it another look.

lucy24




msg:4675881
 5:51 pm on May 29, 2014 (gmt 0)

Found this while looking up a new botnet (characterized by "PUT /nyet.gif"* if anyone cares).

37.128.0.0/17
Free lookup says "Internetia". Meaningless to me.
Free lookup says "Selena FM". Meaningless to me.
Free lookup says 11 websites use this address (that is, the exact IP I was looking up: 37.128.80.222)
Now you're speaking my language.

Same lookup led to
213.246.64.0/18
(UK, Namesco)
which also appears to be servers, at least as far as the specific offender at 213.246.110.188.

Edit:
Robot at 176.223.207.4 may be related. But 176.223 is Poland so I just blocked 'em without further investigation.

If you are Polish or Ukrainian, it must feel awful to read WebmasterWorld ;)


* My server responded with 405, which is a perfectly acceptable translation of "nyet".

[edited by: phranque at 6:03 am (utc) on Jun 4, 2014]
[edit reason] typofix [/edit]

dstiles




msg:4675918
 7:41 pm on May 29, 2014 (gmt 0)

I have Netia (Poland) as below, mostly set to DSL rather than servers, although I have one sub-range and one IP blocked. There was a flurry of bad accesses a couple of years ago but mostly clean since then.

5.226.64.0 - 5.226.127.255
37.128.0.0 - 37.128.127.255
46.227.104.0 - 46.227.111.255
62.233.128.0 - 62.233.255.255
62.244.128.0 - 62.244.159.255
77.252.0.0 - 77.255.255.255
81.210.0.0 - 81.210.127.255
81.219.0.0 - 81.219.255.255
81.219.16.0 - 81.219.23.255 (blocked)
83.238.0.0 - 83.238.255.255
83.238.167.113 - 83.238.167.113 (blocked)
87.204.0.0 - 87.205.255.255
159.205.0.0 - 159.205.255.255
178.36.0.0 - 178.37.255.255
213.238.64.0 - 213.238.127.255
213.241.0.0 - 213.241.127.255

I have namesco listed as DSL and servers - they do both. I may be wrong about the DSL one but no bad hits shown since I listed it in 2012. Your example IP could be just a botnet-infected machine.

85.233.160.0 - 85.233.167.255 (blocked)
195.7.224.0 - 195.7.255.255 (blocked)
213.246.64.0 - 213.246.127.255 (dsl)

I have 176.223.0.0 - 176.223.255.255 as DSL (Romania) with only a couple of bad hits last year.

I've had no activity at all on 165.223.0.0/16 since my list began in 2010.

lucy24




msg:4675931
 8:28 pm on May 29, 2014 (gmt 0)

That last was a typo (look at your keyboard); just missed the time cutoff so I'll have to find a moderator.

could be just a botnet-infected machine

Free lookup says 13 www sites use the exact IP. I'd call that a server, though of course it might not cover the whole /18.

keyplyr




msg:4675959
 10:19 pm on May 29, 2014 (gmt 0)

Lucy, I've had 213.246.64.0/18 (Namesco 213.246.64.0 - 213.246.127.255) blocked for a while now.

blend27




msg:4677283
 3:13 am on Jun 4, 2014 (gmt 0)

QuadraNet, Inc

[bgp.he.net...]

in this case:

NetRange: 69.12.64.0 - 69.12.95.255
CIDR: 69.12.64.0/19

some Anastasie's from there bypassing my filters following same UA(Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.63 Safari/537.36) from DigitalFyre Internet Solutions, LLC.

23.95.92.0/22

3 times so far

p.s.

re: quadranet: I have some ranges from the past associated with MZIMA, we all remember what that was, right?

not2easy




msg:4677284
 4:37 am on Jun 4, 2014 (gmt 0)

I have had Quadranet with nearly the same UA since around March this year, had not seen the 23.95.92.0/22 visits, but the same UA was on 107.6.0.0/16 which I believe is Peer-1 hosting.

Incidentally, the UA was coming in with self referrals that included the//RK=0 gibberish that was being discussed back then as in:
"http://www.example.com//RK=0/RS=WQY_ZDkoVpseFK_rRoMpi6KXeBQ-" where example.com is my site. There were additional requests on that site with the //RK=0 etc attached to the requests. Same UA: "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.72 Safari/537.36"
Note - The UA is not quite identical, you're seeing NT 6.1 and the one I keep seeing is NT 6.2

keyplyr




msg:4677369
 10:01 am on Jun 4, 2014 (gmt 0)



@blend

Yup, I've had 69.12.64.0/19 and all other Quadranet ranges blocked for a while now.

RE: DigitalFyre 23.95.92.0/22 is inside ColoCrossing:
23.94.0.0 - 23.95.255.255
23.94.0.0/15

This 193 message thread spans 7 pages: < < 193 ( 1 2 [3] 4 5 6 7 > >
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved