The Delphi Development Process: Handling Bugs

In this post I’ll talk about how we handle manage bugs in the product. I think the best way to discuss this would be to simply follow the lifecycle of a bug and try to illustrate how the process works along the way.

A Bug is Born

Well, not really, but you get the idea a QA engineer finds, reproduces, narrows down, prioritizes and finally logs a bug. Our internal bug tracking system (called RAID) which btw, is a Delphi application we’ve maintained/upgraded/improved since before D1 shipped, is the cornerstone of our bug tracking efforts and not only works internally but is also used for logging bugs by Field Testers. RAID is incredibly robust and I’d argue that it has probably been the single most reliable application that our entire team has used since Delphi 1. The server has roughly 200,000 bugs (covering several different products not just Delphi) and runs on a dual processor PIII 500MHz with 512MB of RAM with over 1 million records in the audit table running on Interbase 6.5. RAID would be a great topic to blog on but it’s not something I’ll focus on here. One last remark before I move on, on more than one occasion an R&D engineer has left the Delphi team only to call back later asking if there was any way they could get their hands on RAID for their new company/project. Ok, back on topic, QA fills in the following fields for each bug:

  • Product
  • Version
  • Area
  • Bug Type
  • Bug Serverity
  • Recommendation
  • Description
  • Steps
  • Found by
  • Tester
  • Developer
  • Locale

(I’ll see if I can post a screenshot of RAID)

Many of these fields are automatically filled in as the bug is entered so the QA engineer really just needs to focus on the Type, Severity, Recommendation, Description and Steps. Once the bug is logged the QA engineer may contact the assigned R&D engineer (depending on Severity) and mention the new bug. One of R&D’s core responsibilities is to review their open bugs and set an “Action” field which indicates what the impact of that bug is from an R&D perspective. The Action field corresponds to QA’s Recommendation field giving a final assessment from both sides as to the impact of the bug in question.

Bug Council

Once the bug has been prioritized by QA and R&D it’s up to the R&D engineer to schedule time to work on the bug unless it requires immediate attention. Throughout the development cycle the R&D, QA and Program managers review the bugs logged against the product during “bug councils” where they’ll contact the R&D/QA engineers to discuss (typically) the highest prioritized bugs. Bug Council is where the decisions are made to defer bugs, raise their priority or otherwise draw attention to a specific issue that the team feels should be addressed.

As R&D fixes bugs they are marked as fixed in RAID and checked into the the current branch using StarTeam. Change Requests in StarTeam are created to track which checkins were made to address specific bugs and allows us to better track which checkins were related to bug fixing versus feature work.

Back to QA

Once a bug is marked as fixed the “ball” lands back in QA’s court to verify the fix. If the bug is verified as fixed then the bug is marked as “verified” in RAID. At this point, the bug is considered to be resolved but not yet closed which will occur when QA makes a final regression test pass which usually occurs prior to any major milestone. If the bug doesn’t verify and isn’t truly fixed then it is reopend and the process starts over. Since our team is fairly small QA is usually in close contact with R&D and typically keep each other in sync regarding fixing/verifying bugs.

I’m thinking that I could break up these posts so that they’re smaller and therefore (perhaps) more frequent, thoughts?