Welcome to WebmasterWorld Guest from 3.214.184.196

Forum Moderators: DixonJones & mademetop

Message Too Old, No Replies

Microsoft Log Parser 2.0

Free new supercool Microsoft tool to query logfiles

     
3:38 am on Jan 6, 2003 (gmt 0)

Full Member

10+ Year Member

joined:Dec 6, 2002
posts:279
votes: 0


[microsoft.com...]

This is tool is VERY COOL.

2:52 pm on Jan 6, 2003 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38256
votes: 115


Log Parser is available as a command-line tool

long live DOS ;-)

Very interesting product, but this means I have to brush up on my sql query syntax - ugh.

7:17 pm on Jan 7, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 17, 2002
posts:2251
votes: 0


I agree with you about DOS, Brett. FYI the package includes a .dll that can be used in Windows programming so you can build your own GUI for it if you wanted to.
7:39 pm on Jan 7, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Apr 25, 2002
posts:872
votes: 0


If you've played with this how would you say it compares speed-wise to say an unoptimised db query or perhaps something a little closer to itself such searching though the files using perl?

- Tony

9:08 pm on Jan 7, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 17, 2002
posts:2251
votes: 0


"c:\program files\log parser\logparser.exe" "SELECT TOP 20 cs-uri-stem, COUNT(*) AS Total, MAX(time-taken) AS MaxTime, AVG(time-taken) AS AvgTime, AVG(sc-bytes) AS AvgBytesSent FROM ex*.log GROUP BY cs-uri-stem ORDER BY Total DESC"

The above query was run on a 46mb log file to extract the top 20 URIs. I won't post the individual results but below are the stats Log Parser gave me about the overall job:

Elements processed: 293109
Elements output: 20
Execution time: 11.15 seconds

I'm not sure what you mean about how it compares to an un-optimized db query. I mean, to do that I'd first have to parse the log to get it into SQL Server and then do my query, right? As for searching through the files using something that supports regular expressions, I'll never do it again. This is so much easier than that if you're familiar with SQL queries.

6:30 pm on Jan 17, 2003 (gmt 0)

New User

10+ Year Member

joined:Jan 16, 2003
posts:18
votes: 0


Hmm... this could be an interesting project...no GUI-based solutions utilizing the DLL yet as far as I can find.

I'm thinking along the lines of inheriting the component into a .NET application; and utilizing the .NET regular expressions namespace. You could really do some interesting non-traditional inquiry and data-mining log operations with this combination. ;)

I think I'll give it a go this weekend... anyone have any ideas for some test cases they'd like to see it run/benchmark it on?

<grins>

6:42 pm on Jan 17, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:July 16, 2001
posts:2024
votes: 6


Neat! It reads all sorts of input formats.

Here is the Log Parser documentation [microsoft.com] in Word document

Some other SQL examples:

  • sum the number requests and bytes sent per hour
  • number of rejections by a client IP address [microsoft.com]
  • 6:25 pm on Jan 20, 2003 (gmt 0)

    New User

    10+ Year Member

    joined:Jan 16, 2003
    posts:18
    votes: 0


    So far so good in the test arena of just being able to inherit the component/run some queries through it. The readme doc actually has fairly good info on what each method of the COM component does.

    Performance-wise it seems to be pretty decent so far... I parsed through two years of logs for data on the top 10,000 by request - sorted by return code (200, 404, 500, etc.) and # of requests; and kicked out an XML file of the results all in under 30 seconds. I know there will be an improvement in performance in a regular environment as well(I was running it in debug mode - watching it very carefully for anything out of the ordinary).

    I had the thought that I might go the route of an ASP.NET application - have it read the logs/parse/display the results to a secure webpage; though this is a bit more of an undertaking than I'm currently looking for. We'll see though.

    2:19 pm on Mar 6, 2003 (gmt 0)

    New User

    10+ Year Member

    joined:Mar 6, 2003
    posts:7
    votes: 0


    Sorry for my ignorance everyone but I downloaded the Log PArser and even registered the .DLL on my WIN2000 Prof machine but I can not figure out how to use it from my VB6.0 IDE.

    How do I get started with it?