I am sure most of you have been on a holiday with a travel agent. What made it good? Some of...
What is regression testing and how to do it?
In this article you’ll learn about:
- What regression testing is
- When and how to do regression testing
- Techniques used by tech teams
- Best practices that you may follow to safe-keep the proper execution of regression tests
As a matter of fact, adding changes to existing code doesn’t have to be a leap of faith, provided that there’s an effective process in place!
Here’s where regression testing comes into play – both as a process and a quality assurance strategy.
For starters, let’s take a look at how regression testing is defined.
Regression testing – a definition
If you pop the term into Google, you’ll come across the Wikipedia definition which aptly describes regression testing as “re-running functional and non-functional tests to ensure that previously developed and tested software still performs after a change”.
In practical terms, regression tests allow the development team to verify if the code is intact after deployed changes and is fully operational. This can be quite challenging, especially if you keep in mind all the historical tweaks, bug fixes, optimizations, and/or deletion of parts of code.
As a critical process in any software team, regardless of its size, it also helps verify if the problems uncovered during regression testing were there before, or resulted from the latest changes.
We’ll discuss the recommended types of regression testing based on team size further on in the post, but for now, let’s discuss when and how regression testing is applied.
When and how are regression tests used
There are, roughly, two types of situations when regression testing is highly advised. These are:
- when a new piece of code is added (so, the scenario we mentioned earlier in the post)
- when existing code is modified because it needs to align to new requirements (think, for instance, of any implications that may arise after implementing GDPR compliance).
An effectively-executed regression test can result in any of the following:
- detecting bugs that need to be fixed
- checking if bugs detected during a previous regression test have been properly fixed
- understanding the overall performance of the software (so, it’s not detecting mistakes/bugs in the code, per se, but rather checking the software’s general output quality).
As you can see, these are some powerful use cases software teams could not live without!
How to do regression testing
Now that we’ve covered the purpose and scenarios, let’s review the most popular techniques employed by teams based on their size and project type.
Guru99, one of the world’s leading software education websites, categorizes regression testing techniques as follows:
As you can likely guess from the name, “retest all” means that all the types of tests that have been previously run against a software will be launched again. While this is a bullet-proof method that won’t leave any areas unchecked, it can also be the most expensive and time-consuming. Especially, if many developers are engaged in the process.
Regression test selection
Contrary to the technique above, only part of the tests are chosen to be re-run. There are two types of tests to consider here – reusable test cases, so, such that can be re-applied in the future as many times as necessary, and obsolete test cases, which will only work once (for ex. until a specific bug is deleted).
Prioritization of test cases
The third technique allows the team to decide which tests should be run first based on their business importance. So, for instance, you may only focus on tests that are critical for the functioning of a feature that is most important to end-users.
As you can see, both regression test selection and prioritization of test cases are much more cost and time-effective and tend to be the techniques of choice for teams with limited human and financial resources.
Regression testing can also be distinguished on the basis of who executes them. These are either manual or automated.
Manual vs automated regression testing
Automated regression testing
While the benefits of running automated regression tests are a topic for a whole other post, it’s worth noting that its the choice of many startups and SME businesses.
The most important reason is that running frequent tests is much more affordable in this mode.
Many companies that use automated testing software hire Automation Engineers or Regression Test Engineers whose job it is to oversee the entire process. This allows the remaining engineers on the team to focus solely on developing the software.
Manual regression testing
Respectively, manual regression testing takes place mostly among teams where several people can dedicate their time to regularly scrutinizing the software.
As mentioned previously, engaging the team in frequent regression tests makes it an expensive and lengthy process for the company.
The pros and cons of manual regression testing also come down to human engagement.
On the one hand, people can, for instance, go off the beaten path and do a random test if they see something a tool wouldn’t notice. On the other, they feel exhaustion, which impacts their performance and extends the time to completion. Hence, that’s something that isn’t the case for an automatic tool, which is ready to work 24/7.
Best practices to follow
As we near the end of this post, we’d like to leave you with some takeaways as to how you yourself can plan software regression testing.
Here is something we like to think of as the 101 of any effective regression testing:
- Do not add any new code during the test.
- The database you are using for regression tests should be isolated; no changes should be applied during the testing process.
- Discuss and establish a regression testing schedule that works for you – think of the criteria that determine when tests will need to be made (these can be set cyclically, for example, once a week, or after a specific type of event such as adding a new feature).
- Track the results of all regression tests (either in a management tool or your regression tool’s analytics dashboard).
All of these points can be universally applied, regardless of the type of software you and your team have taken on to build.