Anna Thorsen
Exactly how mature is your testing strategy? This can be difficult to determine, especially if you are in the early stages of your journey.
Whether you’re only dabbling in testing, or further on in your journey and already using test automation, this post is for you.
Related reading: What is test maturity?
We’ve compiled a short overview of the key characteristics of each level of maturity.
By taking the Test Maturity Model and our homegrown version, you’ll be able to categorize where you’re at, and how you can progress.
What is your testing strategy maturity?
There are lots of ways to define your test maturity level. The most common is the Test Maturity Model (TMM), created by the Illinois Institute of Technology (see below).
You can determine the maturity of your testing by following TMM. It lets you set a benchmark for where you need to be.
|
Stage |
Objective |
Current state |
|
Initial |
No structured processes |
Tests will follow no clear structure. It's likely there are no dedicated testers. Few or no processes and standards are in place. The quality of software will be dependent on the person carrying it out. How to improve: a manager should be assigned to create common testing processes and standards. |
|
Managed |
Test policy and strategy |
Testing will be rudimentary. However, basic structures for testing will be in place. Quality assurance will start to be worked into the everyday. However, new practices will often be abandoned because of high workloads. How to improve: management needs a firm commitment to project management processes. |
|
Defined |
Test organization |
Testing becomes more organized. Those carrying it out are becoming committed to following processes. The right skills to get the job done are being identified. Planning and quality assurance will be thought into all testing initiatives. The main focus becomes standardizing tests, integrations with the development pipeline, and documenting results. How to improve: management needs to commit to running quality testing, and project management, even under time pressure. |
|
Measured |
Test measurement |
The testing performance is starting to be measured. Processes are adapting based on the outcome of measurements. Approaches to testing may become more sophisticated, and external tools to aid testing may be considered. How to improve: understanding and committing to increased productivity, cost reduction, and customer satisfaction. |
|
Optimized |
Defect prevention |
Here, it is about continuously improving processes. Businesses may begin looking into innovating their practices with the help of new tools and technologies. This level can be difficult to sustain and requires a strong toolset to maintain. How to improve: benchmark the business against competition and continuously innovating your processes. |
Digging deeper into test automation maturity
The maturity of your testing can be simplified even more. By taking a closer look at the stages above, we can categorize the type of testing a user is carrying out into four buckets.
These are:
|
No testing |
Quality status: bad quality |
By not testing your software (whether for internal or external software), you will have excessive costs, delayed product launches, a bad reputation, and dissatisfied users. How to improve: improve quality with testing |
|
Manual testing |
Quality status: improving quality |
Only carrying out manual testing takes more time and resources. It’s a less accurate method for testing that gives poor coverage. How to improve: continue to increase quality with more efficient processes (e.g. automation) |
|
Script-based automation |
Quality status: improving resource constraints |
At this stage, resource constraints have improved. The risk of costly bugs has gone down. However, code-based automation may open up new resource problems. It requires specialist resources and heavy maintenance both to use and to keep the automation running. This makes it difficult to scale. How to improve: continue to improve resource constraints by reconsidering tooling, and becoming less reliant on tools that require technical resources. |
|
Scalable and maintainable automation |
Quality status: reconsider tooling |
Cost savings become visible and risk is significantly reduced. Productivity increases with the toolset and technologies you’re using. How to improve: continue to adopt agile practices that reduce the maintenance burden. Adopt tooling that breaks down the barrier between machines and people. |
So what do these stages mean for you and how can you build scalable and maintainable testing practices? We’ve put together a detailed guide on building scalable and maintainable test automation to help you on your journey.
