Access log compression

[rafabarcelos]

Does anyone know if there is compressor programs for access logs? For example, gzip is a good compressor but it is for binaries and we can't make searches in the compressed file...

I'd like to work on it, but before i am looking for to know what has been done...

thank you..

Rafael

PS: i am sorry for the english.. :(

[sugarkane]

Welcome to WebmasterWorld rafabarcelos

Gzip is commonly used for compressing logs, and although as you say you can't search directly in the compressed file, most stats programs can cope just fine by decompressing on-the-fly before processing.

An alternative would be interesting though - did you have anything in mind?

[rafabarcelos]

thank you for replying.....

Well i am finisinhg my computer science degree..and i've read a lot about text compression and search in compressed files.
With search in comresssed files techniques, we don't need decompressing the log, so the search is faster because the compressed files is much more smaller...
I don't know yet how gzip works, but i think that for searching, it decompress the entire line from the compressed log and then it makes the search which with big logs is very slow.

But before i have to know if someone has already done something like thah... do you know sugarkane?

thanks
Rafael