Welcome to WebmasterWorld Guest from 54.144.77.26

Forum Moderators: Ocean10000 & incrediBILL

Message Too Old, No Replies

Definitive Sources of IP Lists

     

incrediBILL

8:03 am on Nov 28, 2012 (gmt 0)

WebmasterWorld Administrator incredibill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Been looking for some definitive IP list sources maintained by the owner and not guessed at by 3rd parties by observation and a couple I came across are:

Microsoft's Azure Cloud Services provides an XML file:
[microsoft.com...]

Google provides a simple list in DNS from the command line:
nslookup -type=TXT _netblocks.google.com


Which returns:
_netblocks.google.com text = "v=spf1 ip4:216.239.32.0/19 ip4:64.233.160.0/19 ip4:66.249.80.0/20 ip4:72.14.192.0/18 ip4:209.85.128.0/17 ip4:66.102.0.0/20 ip4:74.125.0.0/16 ip4:64.18.0.0/20 ip4:207.126.144.0/20 ip4:173.194.0.0/16 ?all"


Anyone else got any direct IP list sources like these for Yahoo, Bing, etc.?

bigtoga

12:39 pm on Nov 28, 2012 (gmt 0)

10+ Year Member



That's a great list - thank you for sharing.

incrediBILL

7:29 pm on Nov 28, 2012 (gmt 0)

WebmasterWorld Administrator incredibill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Thought I'd add the IP assignments per country for those new to access control that don't know about these:

ftp://ftp.apnic.net/pub/apnic/stats/apnic/delegated-apnic-latest
ftp://ftp.arin.net/pub/stats/arin/delegated-arin-latest
ftp://ftp.lacnic.net/pub/stats/lacnic/delegated-lacnic-latest
ftp://ftp.ripe.net/ripe/stats/delegated-ripencc-latest
ftp://ftp.afrinic.net/pub/stats/afrinic/delegated-afrinic-latest

Anyone got anything else good, anyone?

dstiles

9:48 pm on Nov 28, 2012 (gmt 0)

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member



The MS lists are (mostly) very short ranges! A real pain to add to blocklists. Probably worth doing, though, if they are clouds.

The v=spf returns are for mail (part of DKIM encoding/checking) - they specify IPs that can send mail so are probably not a threat to web servers.

Update:

Looking more closely at the MS ranges: some can be concatenated; others I already have listed as bot IPs. Does that mean they are changing usage?

incrediBILL

11:40 pm on Nov 28, 2012 (gmt 0)

WebmasterWorld Administrator incredibill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



From what I read the Google SPF lists are pretty all encompassing because the change IPs for stuff all the time and that's pretty much the list I had too. Says right on their website that it's where to get the definitive list of IPs and that comment was in the Google Developer section I think and wasn't an email discussion.

MS IP usage is all over the map so who knows. I think any IP is fair game at MS to be used for anything.

I just want sources that I can download and process automatically and keep updated without going stale and these look good so far but you're right, the MS list needs some concatenation.

dstiles

8:38 pm on Nov 29, 2012 (gmt 0)

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member



It figures that G would abuse the spf system. :(

I've been wanting to ban G mail for a long time but my customers and their customers still insist on using it.

A while ago I began blocking all "bot company" IP ranges except for known bot IPs. This seems to block some "mobile" proxies but since Y and G seem to use these IPs indiscriminately, tough.

thetrasher

1:44 pm on Dec 1, 2012 (gmt 0)

10+ Year Member



From the Azure XML file:
<network>65.55.192.0/19</network>

Now I see "msnbot-media/1.1 (+http://search.msn.com/msnbot.htm)" coming from 65.55.211.186 <=> msnbot-65-55-211-186.search.msn.com. This Azure IP list is garbage.

blend27

2:38 pm on Dec 1, 2012 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I started using BGP Toolkit from Hurricane Electric site for finding IP Ranges that belong to specific organization/colo/datacenters a while back. These are on more granular level. Seems to work OK.

Goog IPs for example: bgp.he.net/search?search[search]=google .
---------------------------------------------------------------

The RIR lists from FTP Sites that incrediBILL mentioned are also pulled on monthly bases and processed into a DB. Also have a subscription for ip2location DB.

Bewenched

8:42 pm on Dec 1, 2012 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



Interesting stuff for sure. I use ip2location as well, but maxmind has a really great one as well that I use on an ad need basis. You can look up 25 per hour free on their site.
GeoIP City
GeoIP Country
GeoIP Region
GeoIP Organization (I especially like this one)
GeoIP ISP
GeoIP Netspeed
GeoIP Domain Name
 

Featured Threads

Hot Threads This Week

Hot Threads This Month