Monthly Archives: September 2009

dasBlog provider for use with BlogEngine.NET

I’ve used dasBlog since I first began hosting my own blog and previously considered a move to BlogEngine.NET (BE) but the URL’s aren’t compatible so there remains some investigation to figure out the best mechanism prior to switching. Unfortunately, dasblog seems to be on it’s final legs so spending time there at this point doesn’t seem wise.

To that end, I’ve been investigating various issues involved in a switch and in an attempt to make a transition easier I’ve created a BE provider which reads/writes blog posts and comments using dasBlog’s IBlogDataService rather than BE’s own XML provider. Basically, I’ve replace one XML provider with another, actually that’s not entirely true as this new provide is a descendant of XmlBlogProvider included with BE. Using this approach I avoid having to constantly re-import my blog data as I look into solving the URL issues.

I’ve made the code available via SVN on Google Code here so feel free to grab it and give it a go. I chose Google Code over Codeplex because the latter doesn’t want abandoned projects and I don’t see this as an ongoing project as it’s scope is quite narrow. What I’m interested in is any feedback regarding issues using the code and taking this sort of approach. In my initial playing around with BE it seems to function properly and allows me to use the “slug” feature of BE to “fix” at least part of the URL problem by simply removing spaces from Titles rather than replacing then with dashes.

Things I’ve tested (manually)

  • Adding/Editing/Deleting posts
  • Adding/Editing/Deleting comments

I haven’t played with images or attachments though my blog data seems to render fairly well. I can see some limitations such as no support for threaded comments though I don’t view this as a long term solution but I think it’s a step in the right direction.

Things that need testing/implementation:

  • DateTime conversions from dasBlog to BE
  • Image uploads
  • Windows Live support

Btw, I’m looking for help with the problem of retaining my existing URLs so if you’re a dasblog user and potentially interested in switching ping me and perhaps we can collaborate.

Using TestComplete Stores and Checkpoints

image I’ve recorded a video for Falafel TV illustrating how to use TestComplete’s Stores and Checkpoints features including v7’s new Checkpoint Wizard dialog. In the video I demonstrate recording a Keyword test and leveraging various checkpoints to validate the test has performed as expected. The checkpoints illustrated in this video include:

  • Object
  • Table
  • File

If you have questions or have suggestions for future videos leave a comment.

Other videos: 

Distributed Testing Using TestComplete
HTTP Load Testing Using TestComplete

Text file processing with LINQ

After working on this problem the other day I started Googling looking for posts written about using LINQ for text file processing. I found the post Parsing textfiles with LINQ (or LINQ-to-TextReader) by Arjan Einbu.

LINQ shows us alternate ways to write code, introducing a more declarative coding paradigm. To use LINQ over the lines of a file, we can read all the lines in the file into a collection, and use LINQ over that collection. There’s some overhead to this; the need to read the entire file upfront and to fit the entire file in memory at once.

The solution was to create an extension method on TextReader for IEnumerable<string>. That post was followed up by another post, rather unfortunately titled, improving upon the solution using  TextFieldParser class in the Microsoft.VisualBasic.FileIO namespace, something I wasn’t aware existed and now find it odd this class is stuck well off in left field.

One of the reasons this subject interests me is I’ve been working with EDI files for awhile now and querying data directly from this file format would be really nice. For example, given a PO with line item segments like this:

PO1*1*36*CA*11.15*PE***VP*RRSKRC85*PI*0001111091127~
PID*F****PRSL ROMNE BBY TRAY ORGNC~
PO1*2*84*CA*11.15*PE***VP*RSMKRC85*PI*0001111091131~
PID*F****PRSL SPRG BBY TRAY ORGNC~
PO1*3*84*CA*11.15*PE***VP*RBSKRC85*PI*0001111091128~
PID*F****PRSL SPNCH BBY TRAY ORGNC~
PO1*4*72*CA*11.15*PE***VP*RHEKRC85*PI*0001111091126~
PID*F****PRSL SPRG W/HRB CLM ORGNC~

You can calculate the total quantity, highlighted in yellow, of all line items using LINQ like this:

using (var reader = new StreamReader("c:\\edi\\inbound\\850_09022009_1311_89.txt"))
{
    var query = (from line in reader.GetSplittedLines("*")
                            where line[0].Equals("PO1") && line[2].Length > 0
                            select Convert.ToInt32(line[2])).Sum();
... }

Using Arjan’s implementation of GetSpittedLines, that’s his name not mine for the extension method he wrote, you can apply logic to any of the columns from the file which is pretty cool.

Of course, there are a myriad of ways of doing the same thing but it’s interesting to have access to the columns allowing for calculations and querying. For my EDI work I’m using FileHelpers which works well though I really like this LINQ option. That said, I haven’t done any benchmarking so I’m not sure about the performance but most of the PO’s I’m working with are less than 4KB and the volume isn’t so great that this would be a major factor. At any rate, I hope you find useful for you too.

Btw, if you’re looking for custom EDI implementations feel free to contact me.

Distributed Testing Using TestComplete

imageI’ve recorded a video for Falafel TV illustrating how to use TestComplete’s distributed testing features. In the video I demonstrate recording a Keyword test and adding the necessary support to allow it to be executed in a distributed fashion. If you’re just getting started this is a good introduction and should get you headed in the right direction. I’ve got some notes for an advanced video on the subject but if you have suggestions let me know.

Mail Server Log MIME Header Parsing

imageThe other day a friend, whose not a developer, approached me looking for help with a problem they were having dealing a few large server log files. The file contained 100’s of $g(MIME headers) listed one after the other each with a starting comment and separated by a blank line. I was given a sample containing 1400 MIME headers which at first I opened in Windows Notepad and looked like what you see to the right. They had been trying to work with this file using Excel but not having much luck because the headers were inconsistent sizes and I imagine extracting the right fields was clearly a problem. The desired result was a format from which they could perform some analysis of the data and particularly of the X-Originating-IP field.

After about 10 seconds of staring at the data in Notepad I opened it in Notepad++ and things looked a bit more sane and it sort of dawned on me what my friend’s first thoughts probably were when they first glanced at this file. Excel looked better but didn’t make the process any easier.

image

A few choices entered my mind:

  • Write a simple parser, regex etc.
  • Look for an existing MIME parser
  • Use VS.NET editor Macros to extract the content
  • grep/findstr

My focus desire was to:

  • not spend much time
  • produce a CSV file
  • do something my friend could duplicate (lessen “support”)

I opted to search for a MIME parser largely because I figured one written in C# had to exist then write a tool to spit out a CVS file. My first Google search was “parse email header C#” which gave me a few interesting links but nothing that really caught my eye. The next attempt was “parse MIME header C#”:

image

Bingo.

The CodeProject article is largely code snippets and the first one looked interesting.

Mime m = Mime.Parse("message.eml");
// Do your stuff with mime

I thought, if there is .eml message parsing then I’m good regardless of the fact that it was expecting a file. I downloaded the source and it compiled without error, always encouraging. Next, I looked into the parsing support:

// Summary:
//     Parses mime message from byte[] data.
//
// Parameters:
//   data:
//     Mime message data.
public static Mime Parse(byte[] data);
//
// Summary:
//     Parses mime message from stream.
//
// Parameters:
//   stream:
//     Mime message stream.
public static Mime Parse(Stream stream);
//
// Summary:
//     Parses mime message from file.
//
// Parameters:
//   fileName:
//     Mime message file.
public static Mime Parse(string fileName);

Sweet.

For testing I saved off a single MIME header and created a simple console application to try and parse a fake .eml file which worked like a charm. All that was left to do was write some code to read the log file one header at a time and spit out a .CSV file.

I made one minor change to the MIME parsing code which was to change it’s HeaderFieldCollection from an IEnumerable to IEnumerable<HeaderField> so as to leverage LINQ to search for the “X-Originating-IP”. Of course, I later found out that the code attached to the article is outdated.

At any rate, I quickly had the file parsed, output to .CVS using a simple console application with input and output filename params which I mailed off. So, if you’re looking for MIME header parsing this library worked well for the 1400 headers I tried and I’m glad I could offer this tiny bit of help in a situation that sounds very serious for the folks involved.

Btw, kudos to Ivar Lumi for making this available, heck I think writing this post took longer than developing the solution.

Upgrading to Windows 7 my list of required software

I’m contemplating upgrading to Windows 7 as a number of people here in the office have already done. In fact, at least one person had to make several attempts before it worked which has me feeling like I should be planning this through before diving in. To that end, I figured I’d make a list of the software I need installed to help gauge exactly how long I can expect the move to take. I’m currently running Vista on a MacBook Pro using $g(BootCamp) v1.3 beta which has me thinking that to ensure things go smoothly that I’ll probably need to consider upgrading OSX as well. My current thinking is that I’ll pave (and gladly I might add) my Vista partition and install “fresh”.

So here is my list of “required” software which I’ll update as I think of things I missed:

Ouch. Ok, so this is a good start and has me thinking I need to see if there are ways to run portable versions of at least some of this stuff to avoid having to install though the USB key isn’t always fun/easy to have attached. I wish there was an option to embed a USB key into the machine so it didn’t have to hang off the side.

This has me wondering if there are any alternatives? I can already see this is going to take some serious prioritization and time.

HTTP Load Testing Using TestComplete

image I just posted a new video to Falafel TV  illustrating how to create an HTTP Load Test using TestComplete v7.1. The video covers the basics of getting an HTTP Load Test recorded and running against a website including scaling the test out to four virtual users running on two different machines.

In the interest of keeping the video to a reasonable time length other features related to HTTP Load Testing will be covered in upcoming videos.