homepage Welcome to WebmasterWorld Guest from 54.167.179.48
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Level 3 - Block 4./8 and 8./8 or not?
How to handle?
keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4662263 posted 8:09 am on Apr 11, 2014 (gmt 0)


4.0.0.0 - 4.255.255.255
4.0.0.0/8

8.0.0.0 - 8.255.255.255
8.0.0.0/8

For years I've been going back and forth blocking the entire /8's, then fearing that I'm loosing the office workers, I go back to blocking just the usual suspects that lease space (OVH, Scalematrix, Google Aps, Chinanetcenter, etc.)

I'd like to get a census from a few others. Block it all and accept the collateral damage, or be more surgical?

 

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4662263 posted 2:19 pm on Apr 11, 2014 (gmt 0)

4 & 8 denied

jmccormac

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



 
Msg#: 4662263 posted 3:14 pm on Apr 11, 2014 (gmt 0)

Best to go surgical on a /8. Unless you are blocking at a country level, there's too much of a chance of collateral blockage.

Regards...jmcc

Samizdata

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 4662263 posted 4:03 pm on Apr 11, 2014 (gmt 0)

Block it all and accept the collateral damage, or be more surgical?

Be as surgical as time allows.

Collateral damage is never a good outcome.

...

webcentric

WebmasterWorld Senior Member Top Contributors Of The Month



 
Msg#: 4662263 posted 4:51 pm on Apr 11, 2014 (gmt 0)

I've been wondering about these two /8 blocks as well. I don't see a lot of traffic from them but when I do it's often from referrers like

/url or /search

dstiles

WebmasterWorld Senior Member dstiles us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 4662263 posted 7:16 pm on Apr 11, 2014 (gmt 0)

These are by no means the only /8 ranges.

My method is to leave alone until something nasty this way comes and then investigate, blocking as large a range as relevant or setting it to DSL if that is indicated (ie the hit was probably from an infected m/c).

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4662263 posted 8:06 pm on Apr 11, 2014 (gmt 0)


These are by no means the only /8 ranges.

I agree, but what's the point?


My method is to leave alone until something nasty this way comes and then investigate, blocking as large a range as relevant or setting it to DSL if that is indicated (ie the hit was probably from an infected m/c).

Yes, that's what I always do.

Point is, besides the server farms within the /8, I periodically get various trouble (scrapes, hack attempts, probes, refer spam, et al) from other areas with the /8 that have no specific assignment other than Level 3, at least no info I can dig up at the usual places.

Hence my temptation to just block the entire /8 (especially with 8.0.0.0/8 )

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4662263 posted 11:27 pm on Apr 11, 2014 (gmt 0)

I've a 2003 subnet search saved from the 4.0 range that is a rather large text file (213kb), however incomplete. Arin use to cut-off the search at a specific number and it was impossible to retrieve the complete subnets. If your interested?

I've no subnet on the 8.0 range.

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4662263 posted 11:34 pm on Apr 11, 2014 (gmt 0)

Thanks Don, however not interested as it's old data.

webcentric

WebmasterWorld Senior Member Top Contributors Of The Month



 
Msg#: 4662263 posted 4:38 pm on Apr 27, 2014 (gmt 0)

OK, I find this interesting if not quite the mess in it's own right.

[whois.arin.net ]

When running certain whois queries for stuff in the 4. range, sometimes, all I get for an answer is

LVLT-ORG-4-8 4.0.0.0 - 4.255.255.255

Other times a smaller subnet is returned and the list in the link above pretty much identifies what ranges are actually identified as specific subnets. Everything not accounted for as a named network in the 4 block (for example) appears to be ether. There are a bunch of /16s and at least one /15 listed for 4. but I find it interesting that when I look for a range to identify a request with a referrer like this (yes a referrer)

/_ylt=A0LEVxyZZFhT6hsA.o9XNyoA;_ylu=X3oDMTEzbHM3ZjVxBHNlYwNzcgRwb3MDOARjb2xvA2JmMQR2dGlkA1ZJUDQwM18x/RV=2/RE=1398330650/RO=10/RU=http%3a//... (edited for brevity)

coming from 4.34.68.x

All I get for my trouble is that it's in 4/8.

So, it's very tempting to block that /8 as there doesn't appear to be any explanation for what actually lives there.

Added: At the very least, I'm tempted to block 4.34/16 and get some satisfaction for my trouble. The question is, will that just eventually lead to blocking the entire 4/8 anyway? Just taking more time and effort in the process.

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4662263 posted 6:12 pm on Apr 27, 2014 (gmt 0)

Until a few years ago, ARIN used to allow "sub net" searches (subdelegations)

A search in the the following manner (greater than character) would provide all the sub-net assignments.

> 4.1.

Unfortuately, it no longer works.
Whether the method (subdelegations) is possible using another search option is unknown to me.

bhukkel



 
Msg#: 4662263 posted 8:05 pm on Apr 27, 2014 (gmt 0)

in larger ip blocks i also look at the routing information.

public routing info for 4 is:

cidr;provider;country
4.0.0.0/8;Level 3 Communications, Inc.;US
4.0.0.0/9;Level 3 Communications, Inc.;US
4.17.19.0/24;TSYS;US
4.23.88.0/23;AT&T Mobility Labs;US
4.23.88.0/24;AT&T Mobility Labs;US
4.23.89.0/24;AT&T Mobility Labs;US
4.23.92.0/22;AT&T Mobility Labs;US
4.23.92.0/23;AT&T Mobility Labs;US
4.23.94.0/23;AT&T Mobility Labs;US
4.23.112.0/22;Rapid Systems Corporation;US
4.23.112.0/24;Rapid Systems Corporation;US
4.23.113.0/24;Rapid Systems Corporation;US
4.36.112.0/22;Rapid Systems Corporation;US
4.36.112.0/24;Rapid Systems Corporation;US
4.36.113.0/24;Rapid Systems Corporation;US
4.36.114.0/24;Rapid Systems Corporation;US
4.36.115.0/24;Rapid Systems Corporation;US
4.36.116.0/23;Rapid Systems Corporation;US
4.36.116.0/24;Rapid Systems Corporation;US
4.36.117.0/24;Rapid Systems Corporation;US
4.36.118.0/24;Rapid Systems Corporation;US
4.38.0.0/20;AT&T Mobility Labs;US
4.38.0.0/21;AT&T Mobility Labs;US
4.38.8.0/21;AT&T Mobility Labs;US
4.43.50.0/23;AT&T Mobility Labs;US
4.43.50.0/24;AT&T Mobility Labs;US
4.43.51.0/24;AT&T Mobility Labs;US
4.53.201.0/24;Rebel Hosting;US
4.55.0.0/16;Level 3 Communications, Inc.;US
4.67.96.0/20;AT&T Mobility Labs;US
4.67.96.0/21;AT&T Mobility Labs;US
4.67.104.0/21;AT&T Mobility Labs;US

webcentric

WebmasterWorld Senior Member Top Contributors Of The Month



 
Msg#: 4662263 posted 11:44 pm on Apr 27, 2014 (gmt 0)

Well, this barely scratches the surface but it is revealing. Not sure how to get more of this (and not sure I actually do want more to be frank) when the search limits results to 256 records but it does provide a look under the surface of 4/8.

[whois.arin.net ]

My guess is that there are certainly some humans in here (probably office workers as keyplyr mentioned). Looking at my logs for the past month, I found one example of a visit that was probably human from Level 3 but it was from 8. Beyond that, lots of bots from 4. and nothing that looks remotely human. Blocking that whole /8 looks extreme but from my logs, it seems like I'd just be blocking bots. The original quandary posed in this thread is murkier than ever in my mind.

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4662263 posted 4:18 am on Apr 28, 2014 (gmt 0)

webcentric,
What method did you use to get those results?

In the old days and using the ">" every search provided that it was limited to 256 lines, however more often than not, the results contained more than 250 lines.
The unfortunate side was that when it did return 256 lines, there was not any way to retrieve the next 256 lines and/or any lines afterward.

webcentric

WebmasterWorld Senior Member Top Contributors Of The Month



 
Msg#: 4662263 posted 1:01 pm on Apr 28, 2014 (gmt 0)

Wilderness, the answer is in the url e.g. /children

whois.arin.net/rest/net/NET-4-0-0-0-1/children

Added: BTW, your mention of the ">" operator got me looking at the documentation to see if there was still a mention of it (and there is).
The documentation page is here...

[url]https://www.arin.net/resources/whoisrws/whois_api.html[/url]

The ">" operator is mentioned in the Port 43 service section and the /children parameter, elsewhere.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved