Hands up if you’ve ever read a Test Strategy/Plan/Approach etc. that says the project will be following a “risk-based approach” to testing?
Hands up if you’ve ever sat in a planning meeting and heard the same thing?
Last time I promise – keep your hand up if that statement was regularly followed up with anything more than a knowing nod by the audience before moving to to the next topic of conversation? An awful lot of hands have just gone down, haven’t they?
When reviewing these documents or in walkthroughs I always find myself asking the same two questions:
- As opposed to?
- What does this actually mean?
Let’s look at the first one first.
As opposed to?
Has anyone ever worked on a project that didn’t utilise some sort of risk-based testing approach? Even in the unlikely event you are working on a project where every possible combination of test you could come up with must be covered, that in itself must be based on some sort of risk assessment.
The statement “we will be following a risk-based approach” means nothing unless you explain how you are measuring risk and what that means to the approach. Which leads us nicely on to…
What does this actually mean?
Risk means different things to different people. For example:
- The risk or impact to the business should this particular feature be delayed hitting Production
- The risk or impact to the business should this particular feature go wrong in Production
- The likelihood or impact of this feature failing in test
- The complexity of the feature under test
- The integration impacts of the feature under test
You could continue that list a fair while longer but the point is simple – if you do not agree and define how you will measure risk – and how that drives your approach – different stakeholders will apply their own definition, and that will never end well.
A phrase I hear often is “the risk appetite is too low”. More often than not that is either the result of stakeholders measuring risk in a different way and coming up with different answers, or an inability to communicate risk in an effective enough way to enable a decision to be made.
How best to appropriately measure risk will vary depending on the organisation, project or even individual feature, which is exactly why it is crucial for all involved that you agree it up front.
