GUI models allow for compile time error detection of changes that break test automation

When I was working at CodeGear one of the tools I wrote generated models of .DFM’s (Delphi’s form file format) for use with GUI automation. A model is a class that mirrors the control hierarchy found on a form with classes that can perform automation tasks (clicking, typing, determining location etc.) against the controls. For more information on models refer to this post.

Modeling a Form, an Example

Below is an example of a simple Delphi form and it’s corresponding model class:

Delphi Form Class Corresponding Model
type
TMyForm = class(TForm)
MessageText: TLabel;
Yes: TButton;
No: TButton;
Cancel: TButton;
...

end;
type
TMyFormModel = class(TBaseDlg)
MessageText: TLabelGem;
Yes: TButtonGem;
No: TButtonGem;
Cancel: TButtonGem;
...

end;

The model is generated off of the .DFM which looks like this:

object Form1: TForm1
  Left = 217
  Top = 88
  Width = 1082
  Height = 749
  Caption = 'Form1'
  object MessageText: TLabel
    Left = 19
    Top = 13
    Width = 32
    Height = 13
    Caption = 'Label1'
  end
  object Yes: TButton
    Left = 15
    Top = 43
    Width = 75
    Height = 25
    Caption = 'Button1'
    TabOrder = 0
  end
  object No: TButton
    Left = 101
    Top = 43
    Width = 75
    Height = 25
    Caption = 'Button2'
    TabOrder = 1
  end
end

Delphi’s VCL component library supports Visual Form Inheritance and since there isn’t enough contextual information to reconstruct the form’s class hierarchy without compiling the code the model generator is provided a file that contains these details so the generated models exactly mirrors the application’s form inheritance hierarchy not just the form itself.

So Why is a Model Generator Important?

At first, you might think sure, the model generator is going to save you lots of time coding which is entirely true. But, that’s not the only nor the largest benefit of using generated models. An additional benefit is that the models are compiled using a statically typed language providing for compile time error detection. For example let’s say a developer renames the “Yes” button to “YesBtn” or deletes the MessageText label?

If the model had been hand written or statically generated, which is the situation CodeGear was in, the error would only surface once the test suite had executed. At that point, log file analysis would have to be performed to distinguish between a real bug and a automation error, not a good situation to have QA people in day after day. With generated models these kinds of errors can be detected at compile time allowing for R&D to assess the impact of the change on QA’s automation and not vise versa. Btw, this also underscores the fact that the model generator can provide alias functionality thus preventing simple name changes from impacting test automation.

The faster developers can find out they’ve broken existing test suites the more likely the problem can be corrected even in the event that not all test suites are executed every build. Yet another benefit of continuous integration.

There are several other benefits like:

  • Give R&D the ability to determine the impact a given change will have on the existing test automation
  • Provide insight into the depth of testing through static code analysis and evaluating which models as well as which parts of models are being exercised
  • Allow R&D/QA to quickly automate new UI

Conclusion

Backing your GUI testing with compile time error checking allows you to leverage GUI test automation in several ways. In addition, a development team will have much more visibility into the impact a given change will have on the existing test automation as well as provide insight into areas that need additional testing.

It’s a shame the test framework developed at Borland starting back in 1994 has never made it into the hands of developers outside the company. Perhaps I should look to start an Open Source project for model driven testing using C# based on  IAccessible. Btw, I just Googled on IAccessible and the post I linked to is in the top ten, meaning this advice really does work.

See also: Automation

5 thoughts on “GUI models allow for compile time error detection of changes that break test automation

  1. Steve,
    It sounds interesting, but I am clearly missing something. I read this post, as well as the one to which you linked, and am still scratching my head. For the more obtuse among us, is there a more introductory article to be found anywhere?

  2. Hi Bill,
    You’re right and I need to do a better job writing about this subject. In fact, now that I’ve sat on this for a day this post could have been much different. What started it is that lately, I’ve been working on an automated test using TestComplete which has very nice Aliasing functionality that provides a mapping to a GUI element using a short name but it has no support for modeling nor does it support compile time error checking since it’s based on a javascript like scripting engine. IMO, these are two areas were TC could improve dramatically.
    As I mentioned in the last paragraph of this post I’m beginning to think that the real way to express this would be to start an Open Source testing framework, written in C# since that’s my focus, and use it to truly illustrate how useful models can be. The down side is that it’s a lot of work and I’m not sure I have enough time to dedicate to it.

  3. Steve,
    I’m glad it wasn’t just me. 😉 I know I have been mostly busy with coding at a lower level, interfacing 3rd party boards and so on, but I read and re-read your article, and no light went on. Of course, as I’m still getting over a bout of bronchitis (very nasty — even with antibiotics) I might just be too groggy to get it….

  4. Bill,
    Funny you should mention medication. I had taken some the night before I wrote this post at 4AM. That might explain things! I got about 90 minutes of sleep that night and just couldn’t fall asleep so I fired up Windows Live Writer.

  5. Steve,
    Ah, I see it all now…. 😉
    I’ve finished my antibiotics, but still have congestion, and am still coughing. My brother was down with this for two weeks, and I’ve heard of some folks locally who went through 2 or 3 courses of meds before getting well….

Comments are closed.