My Latest Google, MSN, Yahoo Spider IP List

IP list of GoogleBot, MSNBot and Yahoo Slurp spiders. CIDR formatted.

[hdogan]

Google

8.6.48.0/21
64.68.80.0/21
64.68.88.0/21
64.233.160.0/19
66.249.64.0/19
72.14.192.0/18
209.85.128.0/17
216.200.251.112/28
216.239.32.0/19

MSN

64.4.0.0/18
65.52.0.0/14
131.107.0.0/16
202.96.51.128/25
207.46.0.0/16
207.68.128.0/18
207.68.192.0/20

Yahoo

66.196.64.0/18
66.228.160.0/19
67.195.0.0/16
68.142.192.0/18
68.180.128.0/17
72.30.0.0/16
74.6.0.0/16
202.160.176.0/20

Please let me know if I forgot to add some IP block.

Thanks.

[wilderness]

This a bit more extensive.

http ://www.iplists.com/

[hdogan]

I've checked that site and updated my list.
And I've found that some of IP blocks are not registered by the company (ie, Google, Yahoo, MSN). Especially Korean and Chinese IP ranges. So I didn't include them.

Here is the updated list in PHP array format and PHP function to check:

<?php
$_SPIDER = array();
# Google
$_SPIDER[] = "8.6.48.0/21";
$_SPIDER[] = "64.68.80.0/21";
$_SPIDER[] = "64.68.88.0/21";
$_SPIDER[] = "64.233.160.0/19";
$_SPIDER[] = "66.249.64.0/19";
$_SPIDER[] = "72.14.192.0/18";
$_SPIDER[] = "209.85.128.0/17";
$_SPIDER[] = "216.33.229.160/29";
$_SPIDER[] = "216.200.251.112/28";
$_SPIDER[] = "216.239.32.0/19";

# Yahoo
$_SPIDER[] = "63.163.102.0/24";
$_SPIDER[] = "66.163.160.0/19";
$_SPIDER[] = "66.94.224.0/19";
$_SPIDER[] = "66.196.64.0/18";
$_SPIDER[] = "66.218.64.0/19";
$_SPIDER[] = "66.228.160.0/19";
$_SPIDER[] = "67.195.0.0/16";
$_SPIDER[] = "68.142.192.0/18";
$_SPIDER[] = "68.180.128.0/17";
$_SPIDER[] = "69.147.64.0/18";
$_SPIDER[] = "72.30.0.0/16";
$_SPIDER[] = "74.6.0.0/16";
$_SPIDER[] = "202.46.19.0/24";
$_SPIDER[] = "202.160.176.0/20";
$_SPIDER[] = "202.165.96.0/19";
$_SPIDER[] = "203.141.52.0/26";
$_SPIDER[] = "206.190.32.0/19";
$_SPIDER[] = "207.126.224.0/20";
$_SPIDER[] = "209.73.160.0/19";
$_SPIDER[] = "209.131.32.0/19";
$_SPIDER[] = "209.191.64.0/18";
$_SPIDER[] = "211.14.8.0/24";
$_SPIDER[] = "216.109.112.0/20";
$_SPIDER[] = "216.136.232.0/21";
$_SPIDER[] = "216.145.48.0/20";
$_SPIDER[] = "216.155.192.0/20";

# MSN
$_SPIDER[] = "63.194.155.144/29";
$_SPIDER[] = "64.4.0.0/18";
$_SPIDER[] = "65.52.0.0/14";
$_SPIDER[] = "131.107.0.0/16";
$_SPIDER[] = "202.96.51.128/25";
$_SPIDER[] = "204.95.96.0/19";
$_SPIDER[] = "207.46.0.0/16";
$_SPIDER[] = "207.68.128.0/18";
$_SPIDER[] = "207.68.192.0/20";
$_SPIDER[] = "213.199.128.0/20";

# found this little function at php.net manual pages.
function net_match($cidr, $ip) {
list($net, $mask) = explode('/', $cidr);
return (ip2long($ip) & ~((1 << (32 - $mask)) - 1)) == ip2long($net);
}

function is_spider() {
global $_SPIDER;

$count = count($_SPIDER);

for ($i = 0; $i < $count; $i++) {
if (net_match($_SPIDER[$i], $_SERVER['REMOTE_ADDR'])) {
return true;
}
}

return false;
}

# Example code
if (is_spider()) {
echo "Hello spider :)\n";
}
else {
echo "Umm, you're not a spider...\n";
}
?>