Back in 2022 I wrote a test approach for my team and it was terrible… just boilerplate process that didn’t speak to what the team needed.

I’d been talking for years about asking the WHAT of something before jumping to the HOW… it’s how I’d said to define acceptance criteria in Triforce, it’s the whole shape of my 5 Ws and an H approach and it’s what I say about exploratory testing when I’m teaching others to look for risks first and then work out how to test them. But when I sat down to write a test approach that day I produced something like:

• Helping drive out ACs in Triforce.
• Using exploratory testing and [these tools] to uncover information.
• Capturing test notes in [this tool].
• Raising bugs and managing them via Jira.

Every line of that is something I’d probably do, but none of it is useful to tell the team (beyond for auditability). What I’d actually just said here was “I’ll do some testing at you” in a long document that said nothing about the project, what might go wrong on it and wasn’t something my team could use to understand what risks mattered to mitigate for this project. I could have handed that same approach to any team at any other company and it would have been equally as “correct” and equally as useless.

I saw that what I’d created wasn’t something that useful at all so I spoke to my manager and she advised rewriting it starting from the risks instead, exactly the thing I know that I should have been doing but hadn’t. The second version was a completely different document that was actually really useful and covered way more than the original boilerplate approach did; so let’s talk about why method first test documents just end up as generic slop and why being risk based is way more helpful.

Why we default to the method (the HOW)

Because it’s EASY!

The method (HOW we test) is comfortable and super easy to knock out, we know what we do when we’re testing, what tools to use and when to apply techniques… we can write that stuff in our sleep without leaving our desk or thinking and it feels like progress because words are appearing on a page. Thinking about actual risk is the uncomfortable part because it speaks to uncertainty, it needs you to go understand what’s happening, talk to people who know things that you don’t and admit where you have gaps and it means you have to think (which takes effort).

Because that’s what the internet said to do!

This is the template problem, if you search for a test approach template you’ll get a dozen documents with the same headings all focused around the same use of methodology. But a generic boilerplate template that says “do some testing for your project” is the same as taking a generic pregenerated Dungeons and Dragons character sheet from the starter set and just throwing your character name on the top, technically valid but it tells you nothing about what you’d actually be doing in roleplay.

Alternatively: Because that’s what the auditors want!

Having worked in a number of regulated industries I know full well that auditors want to see the aforementioned testing approach templates focused on methods and what you’re going to do. They’re less interested in the WHAT of your project but want to make sure you’ve used a valid template of HOW. Which is valid and is something that needs to happen sometimes.

But in this instance you’re creating an approach for the auditors… not to help your team / project.

What if we write an approach starting with the risks (the WHAT)?

In order to help my team and drive out a more meaningful test approach I used the same approach that I use in exploratory testing, defining the risks and then looking for ways to find out more information about that risk. Identifying and documenting any areas of uncertainty related to quality on the project helps us to widens our thinking of what should be in a test approach.

1. What could go wrong here? Specific to this product, this architecture, this team, this deadline rather than “bugs might exist”.
2. How much do we care? What’s the risk to the business framed as likelihood and impact, weighed against what this organisation is actually willing to live with (is this good enough).
3. So what will we do about it? What are the techniques, the tool, the person, the timing.

This isn’t new, it’s just the same kind of process that something like risk storming uses. I try to have my focus for risks be anything related to quality on the project, I don’t just limit this to testing or functionality. Instead I’m holistic in my thinking and not afraid to point out things that other people in the team might need to own or do.

The benefits of starting with the risks

More people can join in. Nobody outside engineering or testing probably has an opinion about whether you use Postman but I’m sure everybody has an opinion about whether it’s acceptable for the checkout to fall over during a launch. Framing your work as a risk puts it in the same language that the rest of the organisation already uses for decisions, which is how you get a meaningful decision rather than being told to go and test more.

You start shifting left early. Risks are available before code exists, defects aren’t. If you can name the risk at the beginning of the project then you can design ways to fix for it earlier (which is much cheaper); this is something I’ve written before in not waiting to be given a scope before you start testing.

You see the gaps. This is the one that’s really useful for a strategy or approach! Looking at the risks without filtering them through what you already know how to do means that you’re more likely to identify things that you have no idea how to test for. That’s not a failure, that’s the most valuable line in the document… it’s the gaps that you wouldn’t have covered otherwise. Maybe that’s a training need, a tooling spike or a hire into the project, which is made visible early enough to do something about. This beats out a method first approach document which probably wouldn’t surface this, because you only ever wrote down the things you were already planning to do.

You can surface new things. This approach allows us to actually address new things that we might not have heard of before or have tested. You might not have a tool or technique to hand for writing about AI and AI guardrails, but I bet you can raise the risks.

It sets context for AI. I’ve spoken a lot about AI and how it needs context to be meaningful, for example giving a basic prompt of “how do I test this” will give you a shallow and maybe meaningless set of tests that isn’t useful for the project. But if there’s meaningful risks associated to the prompt or a meaningful approach built from risks, then that sets the context of what you want to look for / what quality you want to build in. Setting the risks you want to work on (rather than just a method to follow) is a crucial guardrail and interface into AI.

What i’d done

So back at 2022 I hadn’t forgotten putting WHAT before the HOW, I knew to do that, I’d written about that and told other people to do that so many times. What happened was that I’d gotten busy and defaulted to using an easy template to go faster; I optimised for producing a document instead of producing an understanding because that felt like an achievement.

Thanks for reading. If this resonated (or if you think that I’ve got it wrong) there’s more to read on the blog… or get in touch via LinkedIn. I’ll talk quality to anyone who wants to listen!