I'm trying to do a tiny code that gets an IP "route" through our site. Like, gets an IP from a field, opens a log file into an array, compares the IP and creates a new array with only the matches(composed only of .htm or .php files), and prints it. Thing is, I'm having some issues with the code(with all the code, lol), mostly with the "open log" part. Our logs are very big(7k visits daily) and it takes a LOT of time to even open it in the first place. Isn't there a way to open it in "pieces"? (like, 20MB at a time) or do I need to manually split the log file? Do you know an already finished script that does this?
I'd advise you to look into logging to a database for this kind of thing. Or, take a look at PhpOpenTracker (www.phpopentracker.de), an excellent Open Source project that will do exactly what you are after and a lot more.
opening it with fopen shouldn't be an issue. The real problem with log parsers is the sheer volume of data you need to read through and compare.
I have one and it runs in seperate parts. I run a clean up script first which dumps a fair amount of data and stores the rest into another file. I then do all of the comparisons/searches/counts on that. It saved a lot of processing time.
You could have apache rotate the logs more often so that it gives you smaller pieces to deal with.