Microsoft Log Parser 2.0

Free new supercool Microsoft tool to query logfiles

[webdevsf]

[microsoft.com...]

This is tool is VERY COOL.

[Brett_Tabke]

Log Parser is available as a command-line tool

long live DOS ;-)

Very interesting product, but this means I have to brush up on my sql query syntax - ugh.

[GaryK]

I agree with you about DOS, Brett. FYI the package includes a .dll that can be used in Windows programming so you can build your own GUI for it if you wanted to.

[Dreamquick]

If you've played with this how would you say it compares speed-wise to say an unoptimised db query or perhaps something a little closer to itself such searching though the files using perl?

- Tony

[GaryK]

"c:\program files\log parser\logparser.exe" "SELECT TOP 20 cs-uri-stem, COUNT(*) AS Total, MAX(time-taken) AS MaxTime, AVG(time-taken) AS AvgTime, AVG(sc-bytes) AS AvgBytesSent FROM ex*.log GROUP BY cs-uri-stem ORDER BY Total DESC"

The above query was run on a 46mb log file to extract the top 20 URIs. I won't post the individual results but below are the stats Log Parser gave me about the overall job:

Elements processed: 293109
Elements output: 20
Execution time: 11.15 seconds

I'm not sure what you mean about how it compares to an un-optimized db query. I mean, to do that I'd first have to parse the log to get it into SQL Server and then do my query, right? As for searching through the files using something that supports regular expressions, I'll never do it again. This is so much easier than that if you're familiar with SQL queries.

[edicius]

Hmm... this could be an interesting project...no GUI-based solutions utilizing the DLL yet as far as I can find.

I'm thinking along the lines of inheriting the component into a .NET application; and utilizing the .NET regular expressions namespace. You could really do some interesting non-traditional inquiry and data-mining log operations with this combination. ;)

I think I'll give it a go this weekend... anyone have any ideas for some test cases they'd like to see it run/benchmark it on?

<grins>

[sun818]

Neat! It reads all sorts of input formats.

Here is the Log Parser documentation [microsoft.com] in Word document

Some other SQL examples:

sum the number requests and bytes sent per hour

number of rejections by a client IP address [microsoft.com]

[edicius]

So far so good in the test arena of just being able to inherit the component/run some queries through it. The readme doc actually has fairly good info on what each method of the COM component does.

Performance-wise it seems to be pretty decent so far... I parsed through two years of logs for data on the top 10,000 by request - sorted by return code (200, 404, 500, etc.) and # of requests; and kicked out an XML file of the results all in under 30 seconds. I know there will be an improvement in performance in a regular environment as well(I was running it in debug mode - watching it very carefully for anything out of the ordinary).

I had the thought that I might go the route of an ASP.NET application - have it read the logs/parse/display the results to a secure webpage; though this is a bit more of an undertaking than I'm currently looking for. We'll see though.

[jesus4u]

Sorry for my ignorance everyone but I downloaded the Log PArser and even registered the .DLL on my WIN2000 Prof machine but I can not figure out how to use it from my VB6.0 IDE.

How do I get started with it?