Maria Homann
Regression testing is an essential part of software testing and quality assurance. In this blog, we define regression testing and help you gainer a deeper understanding of this type of testing.
Skip ahead to:
Examples of regression testing
Learn more about regression testing
What is regression testing?
When developers write and submit new code for an application, those pieces of code sometimes have unexpected effects on existing code, causing it to break.
It’s difficult to avoid this altogether, but it is possible to detect that broken code before it is released.
This is done with regression testing.
Regression testing is a series of tests that check that an application runs as intended. The idea is to run this series of tests every time new code is submitted.
It’s called regression testing because the verb regress means to return to a former state or condition, which in software, isn’t considered a good thing.
Regression testing is often confused with retesting, but the two differ on one key aspect: Regression testing is frequently run on the entire product, whereas retesting is used to check a specific feature, after you’ve fixed a bug in it.
In other words, regression testing is designed to test for bugs you don't expect to be there, whereas retesting is designed to test for bugs you do expect to be there.
Here's a breakdown of key points regarding regression testing:
- Objective: To identify bugs that may have been introduced into existing areas of a software application during development of new features, bug fixes, or any change in the existing code.
- Scope: It can cover a wide range of software functionality, but typically focuses on core features that are critical for the application's operation. Test cases are selected based on the areas of the software that have been changed.
- Execution: Regression tests can be performed manually but are often automated to save time and ensure consistency. Automated regression testing is crucial for continuous testing in CI/CD environments, allowing for rapid iterations of software development.
- Frequency: The frequency of regression testing can vary from project to project. It may be conducted after every major code change, at the end of a development cycle, or before a product release. In agile and DevOps practices, regression testing might be performed even more frequently.
- Benefits: The primary benefit of regression testing is the early detection of defects introduced into previously tested code, leading to higher quality and stability of the software product. It helps maintain confidence in the software as new changes are made.
Let's dive deeper into some of these key points.
Why regression test?
At its core, regression testing is not just about identifying bugs introduced by recent changes; it's a strategic approach to maintain and enhance software quality over time.
By systematically retesting the application after each change, developers can quickly detect and fix any issues, thereby preventing potential defects from progressing to later stages of development or into production. This is important because bugs become so much more expensive to fix once they reach production.
Regression testing is crucial for maintaining software stability, performance, and reliability, especially in agile and continuous integration environments where changes are frequent and incremental.
In the era of rapid digital transformation, where software updates are constant and expected to be seamless, the importance of regression testing cannot be overstated.
It's a safeguard against the risks associated with continuous development and deployment, ensuring that software enhancements lead to progress rather than setbacks - and ensuring happy users and customers.
When to do regression testing
Regression testing is used to search for bugs to ensure that new updates or features that are added to software (involving new or changed code) don’t break any previously released updates or features (existing code), and to ensure that the tested software still performs as expected after a change.
Regression testing isn’t just a safety net; it's like a routine checkup for your system under test. The frequency of these tests will depend on the set-up of the business. Normally, it’s done before a software update or customization is introduced, or to check that integrations between applications work.
Here's a simple overview of the stages where regression testing should be considered:
-
After every major code change
-
Before releases
-
After bug fixes
-
When integrating third-party tools or APIs
-
After environment changes
-
During Continuous Integration/Continuous Deployment (CI/CD) processes
-
Periodically for legacy systems
You can bucket regression testing into two approaches:
Manual testing is the process of letting people, often business experts from other departments who are closest to the business processes, run through the same regression tests. This process is very tedious, time intensive and is prone to error.
Automated testing is the process of building automated test cases once - and then adding them to your regression suite. Often this is done using a self-built framework or a commercial vendor.
Many testers consider manual regression testing particularly tedious; it’s a bit like being given the task of searching through a haystack for a needle that probably isn’t there. And you don’t just have to do it once, you have to do it every time new code is created.
What’s more, the suite of regression tests grows with the product, meaning that over time, it becomes practically impossible to execute and maintain manually.
For this reason, many QA teams choose to automate their regression testing for agility.
Examples of regression testing
If a customer application stops working, a business can lose revenue and their reputation is damaged. If an internal application like Dynamics 365 or Salesforce stops working, the business is at risk; revenue streams, customer experience and business reputation are just some of the things that are at stake when ERP and CRM processes break. Regression testing helps businesses minimize this risk.
In the bullets below, we have listed examples of what regression testing can look like across industries.
- San Diego Superior Court uses a system called Odyssey for their Court Management. This system goes through 3-4 updates per year, each of which have to be tested. Before they introduced Leapwork, they would spend between 2-3 weeks on regression testing Odyssey. Today, they spend 40% less time on each release, and have a suite of 200 automated regression tests.
- AdminContol uses test automation for their regression testing suite of 2000+ test cases. It’s integrated into their CI/CD pipeline, and they run end-to-end regression tests across web and mobile.
- Dutch insurance company Voogd & Voogd regression tests their business applications. 10,000 daily users rely on this application to complete their work. Their regression suite consists of testing each version release across a minimum of two browsers. Done manually, this would take eight hours and ≈115 tests. By introducing automation, they have been able to reduce this time to 20 minutes.
Learn more about regression testing
There's much more to learn about regression testing. Make sure to download our whitepaper on regression testing in agile teams to find out how you can develop a scalable regression testing strategy.
