In Retrospect: About Bugs

This is the third of several posts in which I’d like to share some of the things we learned throughout more than 14 sprints of Agile development using Scrum. Some of them might appear as open doors, but I wish I knew or thought about those before I started that project. Just by looking back at the mistakes a team of 9 developers and one tester made in a period of 12 months, they apparently aren’t that obvious. So after having discussed the way we handle the sprint planning meeting elaborately, let’s briefly talk about bugs.

Strive for a zero-bug policy

In other words, try to keep your list of bugs empty and don’t start a new story until all bugs are solved. The reason for this is the longer you wait with fixing that bug, chances are that it will take a whole lot of more work to fix it. And be reluctant to moving a bug to the product backlog for reprioritizing. If you’re doing that, then you’re probably not dealing with a bug after all and should have been filed as a user story instead. A side-effect of this practice is that it will make it very visible when your team is suffering from a lot of bugs. It should give you a natural tendency to change your way of working so that bugs are less likely to occur.

Reduce the focus factor to deal with regressions.

We never assign story points to bugs that are being solved within the same sprint. The estimated focus factor of your sprint should accommodate for the average number of bugs that are typically found in a sprint. And beware that if you’re not applying automated testing to your entire codebase, the larger the codebase, the bigger the chance that regression issues will occur.

Schedule left-overs in the sprint planning

When, at the end of the sprint not all bugs have been solved, move them onto the agenda of the next sprint planning meeting and estimate them just like you do with stories. This makes it clear that the quality level of the work in the last sprint was not at the expected level and protects you from a sprint with a false start.

Make sure a bug is really a bug

I lost count of how many times somebody in the team started working on a bug and found out that it really was a disguised change request. Especially HTML/CSS-related improvements have the tendency of ending up as bugs and require a lot of time to fix. Just move them to the product backlog as a user story storyotyped as UI Enhanchement. Misinterpretations and oversights in business rules have popped up as bugs once in the while as well, especially if the product owner didn’t spend enough time evaluating the story in the previous sprint. If that happens, we simply change it into a user story and ask the PO to reprioritize it for the next sprint. If you do this often enough, it might encourage the PO to allocate a bit more time to the evaluation next time.

Use a tool to keep track of issues

In the beginning of the project, we simply created a Microsoft OneNote page to keep track of small issues that our tester found throughout the sprint. We thought the low threshold of OneNote would keep the administration to a minimum. But we found that some descriptions were a bit vague, especially if the test professional that created them was out of the office. We introduced some specific requirements for each issue such as what he or she did to trigger the problem, what the expected behavior was, etc. Then we discovered that occasionally, two developers were working on the same issue. So we agreed to highlight the issue in the issue list as soon as a developer started to work on it and to add his name in a dedicated column. The next problem was the length of that list, in particular because our lack of automated UI tests. We tried to solve that by moving the fixed issues to another OneNote page, all to keep away from too much administration.

But you can guess how this story ends. We finally started to file real bugs in our Team Foundation Server environment. Our test professional are now using the Microsoft Test Manager to do structured and exploratory testing, so for them it’s little trouble to create a bug. In fact, we now get a lot more details about the circumstances under which the test was executed. Those really help tracking down a particular problem.

Remember that these are the experiences of me and my team, so they might not work as well for you as they did for us. Nevertheless, I’m really interested in learning about your own experiences, so let me know by commenting on this post or tweeting me at ddoomen. Next time, I’ll be discussing the things we did to get through the sprints as efficiently as possible.