homepage Welcome to WebmasterWorld Guest from 54.237.99.131
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

This 327 message thread spans 11 pages: < < 327 ( 1 2 [3] 4 5 6 7 8 9 10 11 > >     
Server Farms - Sept. 2013
Ongoing Hosting Data Center Discussion
incrediBILL

WebmasterWorld Administrator incredibill us a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 11:55 pm on Sep 4, 2013 (gmt 0)

Continuation of the May 2013 thread:
[webmasterworld.com...]

 

dstiles

WebmasterWorld Senior Member dstiles us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 4607411 posted 7:36 pm on Oct 10, 2013 (gmt 0)

Hmm. Thanks. No mention of that in DNS, which is where I look. :)

However, against that record I have the note:

added to IIS firewall for php exploits 13-4-2012

so it must have been pretty bad at the time. :(

Angonasec

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 4607411 posted 2:23 am on Oct 21, 2013 (gmt 0)

New data centre bot from ramnode.com

So far I've only located this range:
192.184.80.0 - 192.184.95.255
CIDR: 192.184.80.0/20

They have data centres in Seattle, Atlanta and NL

jmccormac

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 6:21 am on Oct 21, 2013 (gmt 0)



192.184.80.0 - 192.184.95.255
192.249.56.0 - 192.249.63.255
192.30.32.0 - 192.30.35.255
192.73.232.0 - 192.73.239.255
199.241.28.0 - 199.241.31.255
176.56.236.0 - 176.56.236.255
81.4.120.0 - 81.4.120.255
176.56.237.0 - 176.56.237.255
176.56.238.0 - 176.56.238.255
81.4.120.0 - 81.4.127.255
176.56.235.0 - 176.56.235.255

I wonder if it might be easier to publish a data centre map of the internet rather than doing this piecemeal.

Regards...jmcc

Angonasec

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 4607411 posted 8:52 am on Oct 21, 2013 (gmt 0)

Thank you kindly Sir!

Both a map and a worldwide list of ranges and CIDRs would be a splendid help.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4607411 posted 9:21 am on Oct 21, 2013 (gmt 0)

192.184.80.0 - 192.184.95.255
192.249.56.0 - 192.249.63.255
192.30.32.0 - 192.30.35.255
192.73.232.0 - 192.73.239.255

Psst!

Before sorting:
\b(\d\d?)\b
>>
0$1
twice

Sort, then delete

\b0+(\d)

Is it
81.4.120 only, or
81.4.120-127 ?

Angonasec

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 4607411 posted 9:46 am on Oct 21, 2013 (gmt 0)

The numerically arranged cidr list for ramnode.com (so far) is:

81.4.120.0/24
81.4.120.0/21
176.56.235.0/24
176.56.236.0/24
176.56.237.0/24
176.56.238.0/24
192.30.32.0/22
192.73.232.0/21
192.184.80.0/20
192.249.56.0/21
199.241.28.0/22

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4607411 posted 4:22 pm on Oct 21, 2013 (gmt 0)

81.4.120.0/24
81.4.120.0/21

?

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 6:51 pm on Oct 21, 2013 (gmt 0)


176.56.236.0/24 + 176.56.237.0/24 = 176.56.236.0/23

dstiles

WebmasterWorld Senior Member dstiles us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 4607411 posted 7:31 pm on Oct 21, 2013 (gmt 0)

Some of those ramnodes are sub-ranges of NL weservit and proserve and some can be combined.

proserve (including weservit and ramnode):
81.4.64.0 - 81.4.127.255

weservit (including some ramnode) (all NL)
176.56.224.0 - 176.56.239.255


ramnode (all USA):
192.30.32.0 - 192.30.35.255
192.73.232.0 - 192.73.239.255
192.184.80.0 - 192.184.95.255
192.249.56.0 - 192.249.63.255
199.241.28.0 - 199.241.31.255

jmccormac:
> I wonder if it might be easier to publish a data centre map of the internet rather than doing this piecemeal.

I would really love to see that! Not an easy map, given so many /24 ranges amongst the /14 and even /12 ranges. :(

And how would you delineate the ranges?

jmccormac

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 8:26 pm on Oct 21, 2013 (gmt 0)

I would really love to see that! Not an easy map, given so many /24 ranges amongst the /14 and even /12 ranges. :(
And how would you delineate the ranges?
Well most of the approaches that I've seen with online IP checkers are delegated range lists and working down to /24s. It is not the best way for dealing with data centres because some data centres are on ISP ranges. The ISP might have the greater allocation so a /16 might end up getting blocked when in reality it could be just a /24 or less that is causing the problem.

For the larger DCs, an inclusive block range might work as they would include various subnets of other hosters using their ranges. In terms of data, the granularity is good (far better than any of those online IP checker things). I spent a few weeks working on this for a project that did not happen so much of the data is actually just stitting on harddrives. In individual IP terms, most of the world's data centres are included in the data. (I'll hammer it into some statistical form later tonight.)

Regards...jmcc

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 9:39 am on Oct 23, 2013 (gmt 0)

Amanah Tech Inc

AMANAH 162.219.176.0 - 162.219.179.255 162.219.176.0/22
AMS4-NTBLK2 184.75.208.0 - 184.75.223.255 184.75.208.0/20
AMANAH-BLOCK1 204.187.100.0 - 204.187.101.255 204.187.100.0/23

dstiles

WebmasterWorld Senior Member dstiles us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 4607411 posted 8:26 pm on Oct 23, 2013 (gmt 0)

Thanks. Didn't have the first one and the last one was so old it hadn't been tagged. :)

Angonasec

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 4607411 posted 1:27 am on Oct 24, 2013 (gmt 0)

Intergenia AG German hosting company using US data centre:
50.30.32.0/20
199.189.84.0/22

jmccormac

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 11:26 am on Oct 24, 2013 (gmt 0)

The whois for the 50.30.32.0/20 range shows as Hosting Solutions International Inc. (6 identified ranges.) Is that a US trading name for Intergenia?.

Regards...jmcc

Angonasec

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 4607411 posted 1:54 pm on Oct 24, 2013 (gmt 0)

The Whois for hostingsolutionsinternational.com confirms it is owned by Intergenia AG

So let's have all the ranges and CIDRs you can find and let's block the rotters.

Angonasec

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 4607411 posted 2:01 pm on Oct 24, 2013 (gmt 0)

Re: German Intergenia: Here's the dubious looking UA that alerted me to their bot today:

"ADmantX Platform Semantic Analyzer - ADmantX Inc. - www.admantx.com - support@admantx.com"

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 3:31 pm on Oct 24, 2013 (gmt 0)

So let's have all the ranges and CIDRs you can find


HSI-1 69.64.32.0 - 69.64.63.255
HSI-2 (209.239.112.0 - 209.239.127.255
HSI-3 173.224.112.0 - 173.224.127.255
HSI-IPV6 2605:DE00:: - 2605:DE00:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF
HSI-4 50.30.32.0 - 50.30.47.255
HSI-5 199.189.84.0 - 199.189.87.255
HSI-6 199.217.112.0 - 199.217.119.255

Angonasec

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 4607411 posted 5:47 pm on Oct 24, 2013 (gmt 0)

Thank you Sir.
The IPV6 has thrown me, but for those using CIDRs...
All HSI/Intergenia so far:

50.30.32.0/20
69.34.32.0/19
69.34.64.0/18
69.34.128.0/17
69.35.0.0/16
69.36.0.0/14
69.40.0.0/13
69.48.0.0/12
69.64.0.0/18
173.224.112.0/20
199.189.84.0/22
199.217.112.0/21
209.239.112.0/20
HSI-IPV6 2605:DE00:: - 2605:DE00:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF

dstiles

WebmasterWorld Senior Member dstiles us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 4607411 posted 8:31 pm on Oct 24, 2013 (gmt 0)

Angonasec - hosting solutions: my current list is:

50.30.32.0 - 50.30.47.255
69.64.32.0 - 69.64.63.255
80.77.80.0 - 80.77.95.255
88.214.192.0 - 88.214.255.255
173.224.112.0 - 173.224.127.255
199.189.84.0 - 199.189.87.255
199.217.112.0 - 199.217.119.255
209.239.112.0 - 209.239.127.255

The others in there seem to be embarq, windstream and others which I do not have specifically listed - they seem to be DSL services. I did notice a webair rqange in there but not intergenia as far as I can tell.

Angonasec

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 4607411 posted 12:25 am on Oct 25, 2013 (gmt 0)

My apologies Sir, here's what I should have posted:
(though I personally do block Embarq and other ISPs)

All HSI/Intergenia so far:
50.30.32.0/20
69.64.32.0/19
173.224.112.0/20
199.189.84.0/22
199.217.112.0/21
209.239.112.0/20
HSI-IPV6 2605:DE00:: - 2605:DE00:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF

Angonasec

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 4607411 posted 12:32 am on Oct 25, 2013 (gmt 0)

All HSI/Intergenia so far:
50.30.32.0/20
69.64.32.0/19
80.77.80.0/20
88.214.192.0/18
173.224.112.0/20
199.189.84.0/22
199.217.112.0/21
209.239.112.0/20

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4607411 posted 4:42 am on Oct 25, 2013 (gmt 0)

HSI-IPV6 2605:DE00:: - 2605:DE00:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF

Are these two forms of the same address? I haven't got IPv6 internalized yet.

bhukkel



 
Msg#: 4607411 posted 6:28 am on Oct 25, 2013 (gmt 0)

@lucy24 :: stands for all zeros, so it is a range

2605:DE00:0000:0000:0000:0000:0000:0000 - 2605:DE00:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4607411 posted 8:03 am on Oct 25, 2013 (gmt 0)

So it's all of 2605:DE00? How would you say that in shorthand? (The way that in IPv4, 12.23 = 12.23.0.0 - 12.23.255.255)

Angonasec

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 4607411 posted 8:16 am on Oct 25, 2013 (gmt 0)

"How would you say that in shorthand?"

Good question Ms. Lucy, and as a CIDR too please...

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4607411 posted 8:57 am on Oct 25, 2013 (gmt 0)

OK, I looked it up. The syntax is essentially the same. In this case

2605:de00::/32

Since IPv6 uses hexadecimal rather than decimal notation, every 2 digits = one IPv4 block. 8 digits = 4 IPv4 blocks, i.e. all of them. Where /32 is the very end of IPv4 (2^8^4), in IPv6 it's just the beginning.

:: counting on fingers ::

Does it go up to /128?

Why is it called 6 when there are 8 pieces? I suspect I will be very embarrassed when someone explains this to me.

Edit: As far as I can make out, you can't truncate. 2605:de00:: without a trailing /number would just mean 2605:de00:0000:0000: et cetera. (As if 12.23 meant exactly 12.23.0.0 only.)

Angonasec

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 4607411 posted 9:58 am on Oct 25, 2013 (gmt 0)

You're ahead of me Ms. Lucy :)

Wondering if mod_rewrite recognizes formatting like this?

2605:de00::/32

Wary of experimenting, my htaccess file is like a plate spinning on a pole.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4607411 posted 10:46 am on Oct 25, 2013 (gmt 0)

Ooh, interesting question. Didn't I read that mod_rewrite in 2.4 recognizes CIDR notation?

:: detour to apache docs ::

Yikes! Allow/Deny/Order (mod_authz-thingy) are on their way out, as of 2.4. New form is Require (not). But even 2.2 lets you use IPv6 (including ranges) in mod_authz-whatsit.

The <If> directive, added in 2.4, replaces many things that mod_rewrite has traditionally been used to do, and you should probably look there first before resorting to mod_rewrite.


:: looking irritably at host because I am mad with envy and want to start using <If> last week ::

Angonasec

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 4607411 posted 4:04 pm on Oct 25, 2013 (gmt 0)

My host gets touchy if I ask about Apache 2 (still using 1.3.41) so I'm out of my depth here, already :)

I'll stick, but have fun.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4607411 posted 2:24 am on Oct 26, 2013 (gmt 0)

Czech robot, new to me:
31.133.13.232
from range
31.133.8.0/21 (8-15, a pretty typical size for 31)

Free lookup says Petr Kadanek, which as his name indicates is apparently a hosting company. Cursory googling suggests that there have been other offenders from the 31.133.13 neighborhood.

1.3? Ouch, ouch. My host won't say exactly what version they use, but there's a cut-and-paste function that shows which mods are installed, so I know it's 2.2.something.

:: wandering off to see if I can figure out why my brain wants to translate "Angonasec" as "He goes hunting". ::

Angonasec

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 4607411 posted 4:32 am on Oct 26, 2013 (gmt 0)

Checked, and I already block all 31.
(I know, I know, but sometimes the Anne Bolyn method is useful.)

1.3.x I'm quite fond of "retro" in most things. (Gas boiler, car, computer, missus, etc) My host would have to redesign their custom panel, logs etc., so I'm content they are "still assessing the value of Apache 2."

This 327 message thread spans 11 pages: < < 327 ( 1 2 [3] 4 5 6 7 8 9 10 11 > >
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved