Forum Moderators: DixonJones

Message Too Old, No Replies

The daily access log nightmare! Monthly access log files save t&m!

We admins unwilling to make a sale how can I automate what the host should?

         

JAB Creations

9:29 pm on May 2, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Finding a competent web host is one of the least desirable things ever as I've already called two dozen web hosts today alone without any success. My main issue is the policy to split Apache access logs in to daily access logs instead of the much more manageable monthly access logs. I know that the virtualhost setting can override this for individual hosts though no web host is willing to take the twenty seconds to do this in order to make a sale.

So I considered the what if scenario...what if I swallow this poor industry-wide plague and work to merge uh, 365 log files?

Well first off I did find a program that will merge files, even letting me drag and drop them from Windows Explorer, great!

The main problem in that scenario is having to manually open 28~31 logs per month and having to do that every month because web admins are too lazy to change a single virtualhost setting.

Oh and to those who say I should stick to daily logs, then you can go ahead and merge them for me for free. Time is money and I don't feel I should be forced to waste my time because people are too lazy to make clients happy. Sure log files may get large (1GB per 100GB of traffic) though these are still compressed and I don't have nearly that amount of traffic...and again the time and thus money to blow doing something manually that should have stayed automated in the first place! Also I do not use third parties for statistical analysis nor am willing to reduce the quality of my analysis to outsource to a third party.

So right now I'm trying to figure out if there is a way I can extract upwards of 31 log files from 31 gzip files as a single file or automate the extraction process that or find a host willing to make a sale by getting off their butt for twenty seconds.

- John

Staffa

10:11 pm on May 2, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I can't help with unzipping gzip files - mine are .cab, which are all unzipped in one go with 7-zip. Once unzipped I run a .bat file (remember good old DOS days ? ) powered by uumerge exe (97 Kb) which combines all my Win log files residing in the same directory into one big .log file in the blink of an eye.

Maybe something to consider doing ?

JAB Creations

10:36 pm on May 2, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Linux servers from my experience only produce gzip files however 7Zip will open gzip files so if they open cab files too then could you please explain the steps needed to unzip all the cab files in one go the way you do it please?

- John

JAB Creations

11:31 pm on May 2, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Actually I had a minute and was able to extract all the files from all the gzip files by selecting multiple files and hitting extract, pretty nifty!

There is a program called "Merge your files" that allows the merging of files.

Even with this level of "automation" it is still a waste of my time and thus money to manually do something that can clearly be automated.

- John

Staffa

7:57 am on May 3, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Glad you found the multi select with 7zip, sorry I didn't come back here until this morning.

As an after thought, could the reluctance of the web hosts be due to the space these huge log files would take up on the server, space they could otherwise sell to another customer.

I have a close look through my log files twice a month when I run the steps above which take no time at all. However, when something peculiar catches my eye I just download the log file for that day to check.
Imagine having to download a huge monthly log file for a quick check on day 27 and having to do it all over again on day 31 to commplete the month. Personally, I would find that to be frustrating ;)