QA teams often say they have reached 80% or 85% automation coverage and are now targeting 90%.
It’s easy to see why this number is important. Automation coverage is easy to measure, easy to communicate to leadership, and it shows that the company has made a serious investment in automation.
But is automation coverage really the best measure of success? I don’t think so; below, I discuss why test confidence is a much better measure of success.
The automation converage/confidence gap
A team can have 90% automation coverage and still not feel confident about releasing its product. Some tests may not have been updated when the application changed. Some may be flaky. Some may cover areas that are easy to automate but not particularly important to the customer.
Meanwhile, the most critical business journeys may still require someone experienced to manually confirm whether everything is working.
For me, the amount of an application that has an automated test is not so relevant. The true question is whether the team trusts the release it is about to ship.
Trusting releases becomes even more important as engineering teams adopt AI more heavily. Developers can now create code, make changes, debug problems, and build internal tools much faster than before. We are already seeing engineers make several AI-assisted changes in a day across different parts of a product.
That sounds like a productivity improvement, and in many ways it is. But faster code creation does not automatically mean faster software delivery. Somebody still must understand what changed, what could have been affected, whether the important flows still work, and whether the release is safe for customers.
This is where QA can unintentionally become a bottleneck, only because the amount and frequency of change are increasing faster than traditional testing processes can comfortably absorb.
If development output increases significantly and QA’s answer is simply to write and maintain more automated tests, the gap will eventually grow. More people and more scripts may help for some time, but they do not solve the underlying problem. The application is changing continuously, while many test suites are still being updated periodically.
Trust > automation coverage
Automation coverage is still a useful signal. But it should not become the final objective.
The objective should be to understand whether a particular change can be trusted, to obtain answers to questions such as:
- What changed?
- Which customer journeys could be affected?
- What was tested?
- What failed?
- Was the failure caused by the application, the environment, the data, or the test itself
- Is there enough evidence to make a release decision?
These questions need to be answered each time the product changes, not only at the end of a release cycle.
This is also how we think about continuous validation at Leapwork. Testing cannot remain something that happens after development has finished. It needs to stay close to the change and keep providing evidence as the application evolves.
Leapwork’s Continuous Validation Platform helps teams move the conversation beyond how many tests they have automated. It brings Enterprise Playwright via Leapwork Play, visual and business-process validation via Leapwork Flow, and performance, load, and API testing via Leapwork Go together under shared governance. This gives teams a broader view of what has been validated across the applications and workflows that matter. Execution history, audit trails, and results provide the evidence needed to investigate failures and make more informed release decisions.
The real aim is to reduce uncertainty around the release. That is why the conversation should gradually move from how much we have automated to how confidently we can release. Coverage can support that discussion, but it should not define it.
