Handling unexpected Windows in GUI test automation

Since I conduct training for AutomatedQA’s TestComplete I like to follow the support forum to see the types of questions people are asking. Recently, Michael Groves posted the following question:

We do testing on our applications on production boxes that are setup as our production builds.  This being the case, we get applications that try to update i.e. virus, Adobe etc… and mess up test runs.  Does anyone know of a way have the test script ignore these items as they come up and continue with testing our app?

Michael’s talking about a class of unexpected windows which aren’t part of the application being tested. This includes things like Microsoft’s own Windows Update dialogs that popup, sometimes persistently, asking to reboot the machine. Unfortunately, there isn’t a simple answer to the question which is perhaps best illustrated by the lack of response he got (aside from me). Clearly, turning off such update prompts would eliminate the problem but they come from all kinds of different sources and may not be easy to disable particularly in environments where software configuration is strictly controlled.

While at Borland working on Delphi’s automation framework I solved, or perhaps a better way to put it would be mitigated, the problem at the framework level meaning test developers could write tests without ever thinking this particular issue. The solution used a secondary thread that constantly probed the active window and under certain conditions would dismiss the window allowing the test to proceed unimpeded. Additionally, the thread handled various exceptions dialogs raised by the application under test (AUT) and aborted the current test allowing subsequent tests to run. In most cases the AUT was terminated and the framework simply re-launched the AUT for the next test. The approach afforded test writer’s freedom to focus on their tests and write much less “defensive” code.

Those pesky #32770 windows!

If you’ve ever been knee deep in Windows GUI automation you’re likely to be familiar with #32770 windows. #32770 refers to a commonly used window class that you’ll find in all sorts of places throughout Windows. For example, the Windows Update dialog, mentioned above, can be particularly problematic not only because it keeps popping up but because it too is a #32770 window. The #32770 class is also used by the .NET Frameworks Assertion Failed dialog as well as other standard error dialogs not to mention Windows Task Manager and Windows common dialogs, ok have I made my point yet?

Clearly, identifying windows by window class alone is insufficient. To that end, the solution I developed cached a list of #32770 window handles that existed prior to test execution to help distinguish them from instances created by the AUT.

WinSpy UtilityAs Michael points out, there may be numerous other applications that display UI during test execution that the test may need to dismiss to prevent false failures. Years ago I wrote a tool (pictured right) to help figure out the specifics of such windows available here. It’s a simple utility application displaying a hierarchy of windows on your system listing Window Handle, {Classname}, Caption, thread ID, and dimensions. I found this tool to be very useful in developing solutions to deal with errant popup windows.

TestComplete’s Approach to Unexpected Windows

If you’re using TestComplete (TC) you can take advantage of it’s OnUnexpectedWindow event, fired when a keyboard or mouse command is unable to be directed to the intended window.

TestComplete onunexpectedwindow event

The event calls a function you write in script giving you the opportunity to manually handle the unexpected window which is passed in as a parameter. TC has a built-in mechanism to handle to try and dismiss the window but in certain cases that can lead to unintended consequences, an issue we explore in the TC training class that Falafel offers. Even if you don’t use this event TestComplete uses a fairly elaborate strategy to deal with unexpected windows and it’s important to clearly understand what it’s doing so you can understand what happens when things fail.

What’s the answer?

I don’t have the perfect answer as it’s often dependent on the environment where the tests are run but developing a flexible system to deal with unexpected windows the rule. In this particular case I suggested to Michael that it might be a good idea to write a console application that handles these unwanted dialogs and simply execute that app from TC and leave it running throughout the test. Using a separate application would help avoid TC’s unexpected window logic which can take time to trigger.

What’s you’re approach to dealing with unexpected windows?