Steve Trefethen
Contact me
About Me View my LinkedIn profile

Powered by discountASP.NET
referal ID: sdtref
Why recommend discountASP.NET?
Need consulting?
Need Consulting?

Disclaimer

The posts on this weblog are provided AS IS with no warranties, and confer no rights. The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.

Publishing IIS log file analysis using Microsoft Log Parser

September 11 2007 8:20AM

Previously, I wrote about setting up IIS logging to allow for better performance analysis using Microsoft Log Parser. Now, that we’re collecting the necessary data it’s time to start querying the logs and see what we can find as well as prepare stats that can be viewed in a browser. Since I’m fairly new to Log Parser and considering it has been rather widely covered I’ve decided to touch on making reports available from the server.

I’m interested in setting up an automated process where log file stats get generated to html files and published from the server. Fortunately, Log Parser supports just such a thing using a template mechanism though it seems to be only lightly documented. Basically, you can create a template HTML file with specific tags which Log Parser uses in a mail merge-like fashion. Here is an example template and its resulting output/screenshot:

MostHit.tpl:

<lpheader>
<html>
<head><title>most hit urls</title>
</head>
<body>
<h1>most hit urls</h1>
<table>
   
<th>%fieldname_1%</th>
   
<th>%fieldname_2%</th>
</tr>
</lpheader>

<lpbody>
<tr>
   
<td><tt>%field_1%</tt></td>
    
<td><tt>%field_2%</tt></td>
</tr>
</lpbody>

<lpfooter>
</table>
</body>
</html>
</lpfooter>

To use this template with Log Parser the command line looks like this:

"C:\Program Files\Log Parser 2.2\LogParser.exe" "SELECT TOP 10 cs-uri-stem as Url, 
COUNT(cs-uri-stem) AS Hits FROM ex0709*.log GROUP BY cs-uri-stem ORDER BY
Hits DESC" -o:tpl -tpl mosthit.tpl > mosthit.htm  

Note, this particular query is from the Jeff Atwater’s excellent blog post on Log Parser which you should definitely check out since he’s posted a number of other useful queries.

You can see I’ve narrowed the range of files to the month of September (ex0709*.log) and I need to come up with a scheduled task to produce rolling 30 day reports. Btw, the report could look substantially better and given Log Parser’s support for many different output format’s including a fun but rather useless "Matrix" style there are lots of possibilities. If you’ve created some useful/interesting templates I’d love to hear about them.

Anyway, with log parser in my toolbox I can start watching trends over time to learn more about usage and performance of the application. On to more immediate methods of analyzing perf!

Digg It!DZone It!StumbleUponTechnoratiRedditDel.icio.usNewsVineFurlBlinkList

Tags:

Comments

9/22/2007 11:46:16 PM #

I absolutely enjoy Log Parser.  There are so many things that you can do with it.  www.bpsoftware.com/.../event-log-part-1.html">Event Log www.bpsoftware.com/.../...ems-do-exist-part-1.html">Free Gems 1 and www.bpsoftware.com/.../...ems-do-exist-part-2.html">Free Gems 2.  The Free Gems 2 post discusses using it with Delphi.  I may pick your brain on dasBlog, the site looks great.

Brad P

9/24/2007 9:44:01 PM #

Steve Trefethen

Hi Brad,
  Thanks for the comment. Feel free to shoot dasblog questions my way using the contact me link here on my blog (right next to my photo).

Steve Trefethen

10/5/2007 2:16:22 PM #

Daniel Wischnewski

Hi Steve,

just in case you might like to know Wink I have setup my DasBlog using Log Parser and have published the scripts used for download counting and entries read.
www.gumpi.com/.../...ParserForRoughStatistics.aspx">Post on Log Parser

Daniel Wischnewski

10/6/2007 2:40:19 AM #

oldmonk

Thanks for your recommendation. Have you tested http://www.loganalyzer.net">nihuo analyzer? I've used it for years, good solid package.

oldmonk

10/7/2007 8:22:21 PM #

Steve Trefethen

oldmonk,
No, I haven't used that product but thanks for the link. For what I was looking for the Log Parser price tag matched nicely.

Steve Trefethen

Add comment


(Will show your Gravatar icon)

  Country flag

biuquote
  • Comment
  • Preview
Loading



Spam filtering provided by: Spam Counter
336 comments approved, 1479 spam caught since October 28, 2009
Powered by Commentor