Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- IP Bunch Analysis


incrediBILL - 4:40 pm on Apr 6, 2012 (gmt 0)


I'm up to 330 IPs so far. Here is a little sample:


You may want to consider moving that list of IPs to a database or a flat file and get it out of code.

At a minimum, break up the list by the first digit in the IP address and use a switch() statement if you intend to keep it in code to minimize the processing time, or drop the data into a flat file organized by the first IP digit (ips_001.txt, ips_002.txt, etc), OR... assuming you retain your list in code, even better and more efficient than a switch() statement, chop up the list of IPs into include files (ips_001.php, ips_002.php, etc.) and just include the shorter IP code you need on request.

Short, fast, minimal processing.

Other techniques which work even faster is to move an IP list out to Apache as a RewriteMap [httpd.apache.org] starting with a plain text file and then change it to a DBM hash file for sheer speed once you get it working. Where Apache wins big time here is it also caches the RewriteMap so it only loads it on the first request and then it's faster than heck.


Thread source:: http://www.webmasterworld.com/search_engine_spiders/4437829.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com