It's been more than a year now since I've set up our bug tracking in JIRA. In there, I've set up an Issue Root Cause custom field where it had the following options:
- Incorrect coding
- Unclear / Missing / Incorrect Requirements
- Unclear / Missing / Incorrect Design
- Insufficient / Duplicated / Incorrect Testing
- Deployment Issue
- Environment
- Third-party issue
My thoughts when I listed these as options is so that it would be easy to identify which team is responsible for the cause why we had that bug. They're pretty straightforward - Incorrect coding is of course when the developers didn't follow the expectations, Unclear / Missing / Incorrect Requirements is because there's a gap in the requirement, and so on. And also, it's because that was the way it's done in my previous company so that's also my initial knowledge source.
Recently, I've been reading a few articles regarding Shift Left, reducing silos, and generally how quality is a team activity instead of it being a responsibility of one department only. Now that I'm looking at the list I realize that:
When I pull data from the bugs we've raised, the number of bugs that show for each root cause item didn't really tell me anything apart from which team might have caused the bug.
The keyword is "might". Because Incorrect coding isn't simply because the developers deviated fro the requirement. What I'd really want to know is the why? Why did they deviate from the requirement? It's not like our developers are rebels and disregard input from our product and design team right? The possible reasons why they made it incorrectly could be:
- Dev and QA had a different understanding of the requirement, and the QA was correct.
- There was a change in the requirement and they weren't aware of it.
- The scenario doesn't happen in the common flows (could be due to different data, or something that we didn't foresee could happen)
- Maybe yes, they simply skipped over a requirement and missed a detail which caused the deviation.
So now I wanna try out a new list of bug root causes:
- Deviation from requirement/design
- Mismatched expectations due to lack of clear communication
- Unexpected/unforeseen scenario
- Deployment Issue
- Environment
- Third-party issue
First of all, this dissolved the silo that the previous list promotes. In this list, it doesn't matter if the root cause comes from Dev, Product, Design, or even QA. This list focuses on the "why".
Deviation from requirement/design. This is the straightforward one. The requirement and design are clear, just that the implementation isn't the same as what's expected.
- Possible outcome from this root cause: We can see how much are really from simple mistakes like these then we can try to find ways to reduce these.
Mismatched expectations due to lack of clear communication. Requirements and designs changed, but it wasn't communicated clearly to everyone. An example is when the developers sent a DM to either product or design to ask about the feature they're working on and they concluded that something needs to change so the developer implemented it that way. When the QA comes to check on the feature when they're testing, it doesn't behave the way they expected it. But then it turns out to be expected, just that they weren't informed.
- Possible outcome from this root cause: We can start looking into the gaps in how we communicate, and think of how we can improve it.
Unexpected/unforeseen scenario. This would be when everyone on the team is aligned on the specifications, but when doing exploratory testing we could find a bug when we use specific test data. A recent example from what I'm testing is for our hotel booking where apparently some attributes are not always provided, which makes the display of the hotel details a bit weird. We didn't expect that there could be such a case because we thought it would always be available.
- Possible outcome from this root cause: The next question is another "why"? Why was it unforeseen? Could it have been avoided if we had taken more time to analyze the supplier data? Could it have been avoided if we asked someone who actually knew this could happen? And if it was really unforeseen, then so be it, and let's take it as a learning experience.
Hopefully, the next time I pull data from our bugs it will give a better view of what we need to work on and start to take action on how we can minimize development bugs.