Automated testing of ASP.NET web applications using Selenium

Lately, I've been focused on Web Application testing frameworks and I've been looking at a number of different options and having varying degrees of success. That is, of course, until I started looking at Selenium after Lino mentioned it to me and I'm very impressed. Selenium is an open source project with multiple tools designed for testing web applications. Selenium includes:

• Selenium IDE an add-on to Firefox use for recording test scripts
• Selenium RC (Remote Control) a Java server used for execution of test scripts
• Selenium Core client side testing support for web applications added directly to your application

All of these are interesting in their own right so be sure to look closely at each piece.

Selenium IDE

The Selenium IDE is a non-modal dialog add-on for Firefox that supports Selenium test development. In fact, it's more than fair to call this single dialog an IDE because it fully supports recording, development and debugging of test scripts. Additionally, it can format test scripts in any of the following for use with Selenium RC:

• HTML
• Java
• C#
• Perl
• PHP
• Python
• Ruby

A Selenium test script consists of a series of Commands which have a Target and optionally a Value. For example, browsing to a web page using Selenium consists of the following:

Command: open
Target: http://localhost/myapp

To click on a link titled "About" on a page the Command might look like this:

Command: clickAndWait
Target: link=About

This will initiate the click and wait until the new page is done loading. There are literally hundreds of commands to choose from covering input (both keyboard and mouse), control manipulation, drag/drop, evaluation, verification, waiting, browser manipulation and just about anything else you'll need.

If you've done any UI testing at all you're familiar with the challenge of manipulating a UI programmatically in a manner independent from of the size, position or location of the control you're trying to manipulate. Selenium solves this problem using XPATH and providing the ability to locate controls based on XPATH expressions alleviating the need to hard code HTML tag structure into a test script. This is particularly crucial for ASP.NET testing since the runtime mangles the ID attributes of rendered tags. For example, the ASP.NET runtime may render ID attributes that look like:

id="ctl00_cphContents_gridMaint_DataGrid"

Finding this control using an XPATH expression can be simplified to something like this:

//table[contains(@id, "gridMaint")]

In the event the nesting of the DataGrid changes the script will continue to function properly as long as table's ID contains the text "gridMaint".

Example of using the Find button on the Selenium IDE

Notice the link is highlighted in the browser

Playing back tests using Selenium TestRunner

The Selenium IDE has a toolbar button that will launch a feature called TestRunner which allows you to playback your tests using controls hosted in an IFRAME inside your browser. Here's what it looks like:

You can playback your entire test from right within your browser. It's like having the IDE built right into your application.

Selenium RC

Next, is Selenium RC which is a Java server application used execute Selenium tests and drive a browser instance through AJAX communication with the browser. To start Selenium RC simply execute the following from a command prompt:

java -jar selenium-server.jar -interactive

Since I'm focused on .NET I'll discuss the C# approach using NUnit. Once you've recorded and debugged your test you can capture it as C# and compile it into an NUnit compatible assembly which you can run using the NUnit GUI or console runner where upon execution it will drive Selenium RC to manipulate the web application through a browser instance running on your desktop. Here is an example of the above recorded test in C#:

 1 using System;
 2 using System.Text;
 3 using System.Text.RegularExpressions;
 4 using System.Threading;
 5 using NUnit.Framework;
 6 using Selenium;
 7 
 8 namespace SeleniumTests
 9 {
10     [TestFixture]
11     public class NewTest
12     {
13         private ISelenium selenium;
14         private StringBuilder verificationErrors;
15         
16         [SetUp]
17         public void SetupTest()
18         {
19             selenium = new DefaultSelenium("localhost", 4444, "*firefox",
20                               "http://localhost:4444");
21             selenium.Start();
22             verificationErrors = new StringBuilder();
23         }
24         
25         [TearDown]
26         public void TeardownTest()
27         {
28             try
29             {
30                 selenium.Stop();
31             }
32             catch (Exception)
33             {
34                 // Ignore errors if unable to close the browser
35             }
36             Assert.AreEqual("", verificationErrors.ToString());
37         }
38         
39         [Test]
40         public void TheNewTest()
41         {
42             selenium.Open("/selenium-ide/");
43             selenium.Click("link=About");
44             selenium.WaitForPageToLoad("30000");
45         }
46     }
47 }
48 

Notice the "Using Selenium" statement referencing the necessary Selenium bits to have your NUnit test call Selenium RC and run your test. Now, when you run this test from NUnit it will call out to Selenium RC and drive the browser and run your tests.

Of course, generation of NUnit compatible tests is great because it makes it really easy to incorporate Selenium test suites into a continuous integration environment like CruiseControl.NET.

Conclusion

If you're in need of a web application testing framework you owe it to yourself to check out Selenium. I'd go so far as to say if you're doing doing any web related development and you're not already using Selenium you should stop right now and go download it. Here are there RSS feeds.

After playing with Selenium for about an hour I was easily able to:

• Record tests against a RIA
• Build a C# NUnit compatible assembly to drive Selenium RC
• Integrate the tests into an automated build using CruiseControl.NET

Since then I've been able to really dig in and accomplish a lot of work in a very short period of time. The next step is to build up a nice battery of tests and dig into NCover to help figure out where the holes are.

Lastly, I just want to tip my hat to the ThoughtWorks and volunteers and supporters of the Selenium project. Kudos for such a great framework!

[Update: May 1, 2009] Made clarifications as to how to run Selenium tests using NUnit based on a comment from a reader.