homepage Welcome to WebmasterWorld Guest from 54.197.215.146
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld

Visit PubCon.com
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

This 327 message thread spans 11 pages: < < 327 ( 1 2 [3] 4 5 6 7 8 9 10 11 > >     
Server Farms - Sept. 2013
Ongoing Hosting Data Center Discussion
incrediBILL




msg:4607413
 11:55 pm on Sep 4, 2013 (gmt 0)

Continuation of the May 2013 thread:
[webmasterworld.com...]

 

dstiles




msg:4615978
 7:36 pm on Oct 10, 2013 (gmt 0)

Hmm. Thanks. No mention of that in DNS, which is where I look. :)

However, against that record I have the note:

added to IIS firewall for php exploits 13-4-2012

so it must have been pretty bad at the time. :(

Angonasec




msg:4618023
 2:23 am on Oct 21, 2013 (gmt 0)

New data centre bot from ramnode.com

So far I've only located this range:
192.184.80.0 - 192.184.95.255
CIDR: 192.184.80.0/20

They have data centres in Seattle, Atlanta and NL

jmccormac




msg:4618052
 6:21 am on Oct 21, 2013 (gmt 0)



192.184.80.0 - 192.184.95.255
192.249.56.0 - 192.249.63.255
192.30.32.0 - 192.30.35.255
192.73.232.0 - 192.73.239.255
199.241.28.0 - 199.241.31.255
176.56.236.0 - 176.56.236.255
81.4.120.0 - 81.4.120.255
176.56.237.0 - 176.56.237.255
176.56.238.0 - 176.56.238.255
81.4.120.0 - 81.4.127.255
176.56.235.0 - 176.56.235.255

I wonder if it might be easier to publish a data centre map of the internet rather than doing this piecemeal.

Regards...jmcc

Angonasec




msg:4618077
 8:52 am on Oct 21, 2013 (gmt 0)

Thank you kindly Sir!

Both a map and a worldwide list of ranges and CIDRs would be a splendid help.

lucy24




msg:4618088
 9:21 am on Oct 21, 2013 (gmt 0)

192.184.80.0 - 192.184.95.255
192.249.56.0 - 192.249.63.255
192.30.32.0 - 192.30.35.255
192.73.232.0 - 192.73.239.255

Psst!

Before sorting:
\b(\d\d?)\b
>>
0$1
twice

Sort, then delete

\b0+(\d)

Is it
81.4.120 only, or
81.4.120-127 ?

Angonasec




msg:4618091
 9:46 am on Oct 21, 2013 (gmt 0)

The numerically arranged cidr list for ramnode.com (so far) is:

81.4.120.0/24
81.4.120.0/21
176.56.235.0/24
176.56.236.0/24
176.56.237.0/24
176.56.238.0/24
192.30.32.0/22
192.73.232.0/21
192.184.80.0/20
192.249.56.0/21
199.241.28.0/22

lucy24




msg:4618146
 4:22 pm on Oct 21, 2013 (gmt 0)

81.4.120.0/24
81.4.120.0/21

?

keyplyr




msg:4618183
 6:51 pm on Oct 21, 2013 (gmt 0)


176.56.236.0/24 + 176.56.237.0/24 = 176.56.236.0/23

dstiles




msg:4618195
 7:31 pm on Oct 21, 2013 (gmt 0)

Some of those ramnodes are sub-ranges of NL weservit and proserve and some can be combined.

proserve (including weservit and ramnode):
81.4.64.0 - 81.4.127.255

weservit (including some ramnode) (all NL)
176.56.224.0 - 176.56.239.255


ramnode (all USA):
192.30.32.0 - 192.30.35.255
192.73.232.0 - 192.73.239.255
192.184.80.0 - 192.184.95.255
192.249.56.0 - 192.249.63.255
199.241.28.0 - 199.241.31.255

jmccormac:
> I wonder if it might be easier to publish a data centre map of the internet rather than doing this piecemeal.

I would really love to see that! Not an easy map, given so many /24 ranges amongst the /14 and even /12 ranges. :(

And how would you delineate the ranges?

jmccormac




msg:4618211
 8:26 pm on Oct 21, 2013 (gmt 0)

I would really love to see that! Not an easy map, given so many /24 ranges amongst the /14 and even /12 ranges. :(
And how would you delineate the ranges?
Well most of the approaches that I've seen with online IP checkers are delegated range lists and working down to /24s. It is not the best way for dealing with data centres because some data centres are on ISP ranges. The ISP might have the greater allocation so a /16 might end up getting blocked when in reality it could be just a /24 or less that is causing the problem.

For the larger DCs, an inclusive block range might work as they would include various subnets of other hosters using their ranges. In terms of data, the granularity is good (far better than any of those online IP checker things). I spent a few weeks working on this for a project that did not happen so much of the data is actually just stitting on harddrives. In individual IP terms, most of the world's data centres are included in the data. (I'll hammer it into some statistical form later tonight.)

Regards...jmcc

wilderness




msg:4618562
 9:39 am on Oct 23, 2013 (gmt 0)

Amanah Tech Inc

AMANAH 162.219.176.0 - 162.219.179.255 162.219.176.0/22
AMS4-NTBLK2 184.75.208.0 - 184.75.223.255 184.75.208.0/20
AMANAH-BLOCK1 204.187.100.0 - 204.187.101.255 204.187.100.0/23

dstiles




msg:4618684
 8:26 pm on Oct 23, 2013 (gmt 0)

Thanks. Didn't have the first one and the last one was so old it hadn't been tagged. :)

Angonasec




msg:4618730
 1:27 am on Oct 24, 2013 (gmt 0)

Intergenia AG German hosting company using US data centre:
50.30.32.0/20
199.189.84.0/22

jmccormac




msg:4618807
 11:26 am on Oct 24, 2013 (gmt 0)

The whois for the 50.30.32.0/20 range shows as Hosting Solutions International Inc. (6 identified ranges.) Is that a US trading name for Intergenia?.

Regards...jmcc

Angonasec




msg:4618818
 1:54 pm on Oct 24, 2013 (gmt 0)

The Whois for hostingsolutionsinternational.com confirms it is owned by Intergenia AG

So let's have all the ranges and CIDRs you can find and let's block the rotters.

Angonasec




msg:4618819
 2:01 pm on Oct 24, 2013 (gmt 0)

Re: German Intergenia: Here's the dubious looking UA that alerted me to their bot today:

"ADmantX Platform Semantic Analyzer - ADmantX Inc. - www.admantx.com - support@admantx.com"

wilderness




msg:4618831
 3:31 pm on Oct 24, 2013 (gmt 0)

So let's have all the ranges and CIDRs you can find


HSI-1 69.64.32.0 - 69.64.63.255
HSI-2 (209.239.112.0 - 209.239.127.255
HSI-3 173.224.112.0 - 173.224.127.255
HSI-IPV6 2605:DE00:: - 2605:DE00:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF
HSI-4 50.30.32.0 - 50.30.47.255
HSI-5 199.189.84.0 - 199.189.87.255
HSI-6 199.217.112.0 - 199.217.119.255

Angonasec




msg:4618854
 5:47 pm on Oct 24, 2013 (gmt 0)

Thank you Sir.
The IPV6 has thrown me, but for those using CIDRs...
All HSI/Intergenia so far:

50.30.32.0/20
69.34.32.0/19
69.34.64.0/18
69.34.128.0/17
69.35.0.0/16
69.36.0.0/14
69.40.0.0/13
69.48.0.0/12
69.64.0.0/18
173.224.112.0/20
199.189.84.0/22
199.217.112.0/21
209.239.112.0/20
HSI-IPV6 2605:DE00:: - 2605:DE00:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF

dstiles




msg:4618878
 8:31 pm on Oct 24, 2013 (gmt 0)

Angonasec - hosting solutions: my current list is:

50.30.32.0 - 50.30.47.255
69.64.32.0 - 69.64.63.255
80.77.80.0 - 80.77.95.255
88.214.192.0 - 88.214.255.255
173.224.112.0 - 173.224.127.255
199.189.84.0 - 199.189.87.255
199.217.112.0 - 199.217.119.255
209.239.112.0 - 209.239.127.255

The others in there seem to be embarq, windstream and others which I do not have specifically listed - they seem to be DSL services. I did notice a webair rqange in there but not intergenia as far as I can tell.

Angonasec




msg:4618901
 12:25 am on Oct 25, 2013 (gmt 0)

My apologies Sir, here's what I should have posted:
(though I personally do block Embarq and other ISPs)

All HSI/Intergenia so far:
50.30.32.0/20
69.64.32.0/19
173.224.112.0/20
199.189.84.0/22
199.217.112.0/21
209.239.112.0/20
HSI-IPV6 2605:DE00:: - 2605:DE00:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF

Angonasec




msg:4618902
 12:32 am on Oct 25, 2013 (gmt 0)

All HSI/Intergenia so far:
50.30.32.0/20
69.64.32.0/19
80.77.80.0/20
88.214.192.0/18
173.224.112.0/20
199.189.84.0/22
199.217.112.0/21
209.239.112.0/20

lucy24




msg:4618932
 4:42 am on Oct 25, 2013 (gmt 0)

HSI-IPV6 2605:DE00:: - 2605:DE00:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF

Are these two forms of the same address? I haven't got IPv6 internalized yet.

bhukkel




msg:4618952
 6:28 am on Oct 25, 2013 (gmt 0)

@lucy24 :: stands for all zeros, so it is a range

2605:DE00:0000:0000:0000:0000:0000:0000 - 2605:DE00:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF

lucy24




msg:4618964
 8:03 am on Oct 25, 2013 (gmt 0)

So it's all of 2605:DE00? How would you say that in shorthand? (The way that in IPv4, 12.23 = 12.23.0.0 - 12.23.255.255)

Angonasec




msg:4618967
 8:16 am on Oct 25, 2013 (gmt 0)

"How would you say that in shorthand?"

Good question Ms. Lucy, and as a CIDR too please...

lucy24




msg:4618973
 8:57 am on Oct 25, 2013 (gmt 0)

OK, I looked it up. The syntax is essentially the same. In this case

2605:de00::/32

Since IPv6 uses hexadecimal rather than decimal notation, every 2 digits = one IPv4 block. 8 digits = 4 IPv4 blocks, i.e. all of them. Where /32 is the very end of IPv4 (2^8^4), in IPv6 it's just the beginning.

:: counting on fingers ::

Does it go up to /128?

Why is it called 6 when there are 8 pieces? I suspect I will be very embarrassed when someone explains this to me.

Edit: As far as I can make out, you can't truncate. 2605:de00:: without a trailing /number would just mean 2605:de00:0000:0000: et cetera. (As if 12.23 meant exactly 12.23.0.0 only.)

Angonasec




msg:4618976
 9:58 am on Oct 25, 2013 (gmt 0)

You're ahead of me Ms. Lucy :)

Wondering if mod_rewrite recognizes formatting like this?

2605:de00::/32

Wary of experimenting, my htaccess file is like a plate spinning on a pole.

lucy24




msg:4618981
 10:46 am on Oct 25, 2013 (gmt 0)

Ooh, interesting question. Didn't I read that mod_rewrite in 2.4 recognizes CIDR notation?

:: detour to apache docs ::

Yikes! Allow/Deny/Order (mod_authz-thingy) are on their way out, as of 2.4. New form is Require (not). But even 2.2 lets you use IPv6 (including ranges) in mod_authz-whatsit.

The <If> directive, added in 2.4, replaces many things that mod_rewrite has traditionally been used to do, and you should probably look there first before resorting to mod_rewrite.


:: looking irritably at host because I am mad with envy and want to start using <If> last week ::

Angonasec




msg:4619062
 4:04 pm on Oct 25, 2013 (gmt 0)

My host gets touchy if I ask about Apache 2 (still using 1.3.41) so I'm out of my depth here, already :)

I'll stick, but have fun.

lucy24




msg:4619177
 2:24 am on Oct 26, 2013 (gmt 0)

Czech robot, new to me:
31.133.13.232
from range
31.133.8.0/21 (8-15, a pretty typical size for 31)

Free lookup says Petr Kadanek, which as his name indicates is apparently a hosting company. Cursory googling suggests that there have been other offenders from the 31.133.13 neighborhood.

1.3? Ouch, ouch. My host won't say exactly what version they use, but there's a cut-and-paste function that shows which mods are installed, so I know it's 2.2.something.

:: wandering off to see if I can figure out why my brain wants to translate "Angonasec" as "He goes hunting". ::

Angonasec




msg:4619184
 4:32 am on Oct 26, 2013 (gmt 0)

Checked, and I already block all 31.
(I know, I know, but sometimes the Anne Bolyn method is useful.)

1.3.x I'm quite fond of "retro" in most things. (Gas boiler, car, computer, missus, etc) My host would have to redesign their custom panel, logs etc., so I'm content they are "still assessing the value of Apache 2."

This 327 message thread spans 11 pages: < < 327 ( 1 2 [3] 4 5 6 7 8 9 10 11 > >
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved