homepage Welcome to WebmasterWorld Guest from 54.205.247.203
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Visit PubCon.com
Home / Forums Index / WebmasterWorld / Website Analytics - Tracking and Logging
Forum Library, Charter, Moderators: Receptional & mademetop

Website Analytics - Tracking and Logging Forum

    
Very high Traffic Analyzers
What are the top 100 sites using for log analysis?
moehits




msg:897397
 12:40 am on Aug 18, 2004 (gmt 0)


What do sites that get 30-50 million hits per day use to analyze log files? Sites like Amazon, Yahoo, AOL, or even high traffic adult sites?

What are the best log analyzers out there that can handle extemely high volume and still let you get detailed reports?

Is this something where you'd be better off importing log files into Oracle, or any high end RDMS, and have a high end DBA make custom reports? Or is there something out there that large sites use?

 

edit_g




msg:897398
 12:49 am on Aug 18, 2004 (gmt 0)

Most of them use their own custom systems to do their log analysis. When you get to that sort of size each business unit will have wildly different requirements and it just becomes easier in the long term to do something yourself.

I can't speak with 100% certaintly for all big sites of course but I know that one of the sites which you mentioned do their own custom log analysis.

drbrain




msg:897399
 11:12 pm on Aug 31, 2004 (gmt 0)

Mozilla.org is using The Webalizer: [mozilla.org...]

The results for September through June were caused by log file truncation (the logs grew to larger than 2G).

XMLMania




msg:897400
 3:13 pm on Sep 1, 2004 (gmt 0)

Wow, moilla.org is really a mega site...

But whats with all the referrels from porn sites? Spamming I'm guessing...

drbrain




msg:897401
 3:15 pm on Sep 1, 2004 (gmt 0)

xmlmania: Nope! Those sites all promote Firefox for "safe browsing". Check them out (from home, preferably).

They did have some mega-casino sites earlier, but they seem to have been flushed out now.

bcolflesh




msg:897402
 3:15 pm on Sep 1, 2004 (gmt 0)

tail & grep ;)

Lord Majestic




msg:897403
 3:20 pm on Sep 1, 2004 (gmt 0)

tail & grep

rotate (more often than once a day) and bzip2 (x2 gain over gzip).

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / WebmasterWorld / Website Analytics - Tracking and Logging
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved