Regression Testing in Software Development

Regression Testing in Software Development

One of the most popular techniques used in development is the iterative – incremental software process model [1] which is a combination of iterative design or iterative method and incremental build model. In this process models’ new functionalities are introduced repeatedly in each of the short development cycles. Adding new functionalities each time brings the risk of adding new errors. Regression testing (RT) [2] is effective in this software process model since it will check to see if previously developed and tested software still performing without an error. If the newly added functionality is creating errors for the previously developed one, this is called a regression. As you may have noticed this will add up to the number of tests needed each time the size of the software increases. This means the bigger the software size, the bigger the test cases, so consequently it will introduce new and additional costs for doing these tests. One of the most popular software development methodologies is agile software development which consists of very short life cycles and limited resources in each cycle. In this methodology adding regression testing at the end of each cycle can introduce unnecessary overhead and costs. In order to reduce these costs, many researchers have paid attention to optimize the execution of these tests. Their main focus has been on identifying the correct and appropriate test cases (TC) for regression tests. The writers have categorized regression testing techniques into four categories: minimization, selection, prioritization, and optimization. There are some other works in literature that focus on this topic and discuss RT techniques. For example in [4], the authors discuss RT techniques based on the prioritization approach. Another work [5] considers agile methodology and its requirements.

We categorize the techniques used for Regression testing into four main categories depicted in the figure below. We wouldn’t discuss Retest All approach here since it performs all possible test cases and the cost of such operation is tremendously high. Here in this work, we will try to briefly explain Regression testing techniques and introduce some automated regression testing tools.

What Is Regression Testing and Why Is It Important?

Regression Testing is a type of software testing which is used to make sure that the newly added or changed piece of the program, hasn’t inversely affected the existing program. We perform this type of testing to ensure that our program or application functionality hasn’t changed adding the new code. This type of testing empowers testers to identify any errors or bugs which may be caused by the changes to the code. But the question which comes to our mind is that when should we apply regression testing (RT). As a general rule of thumb, these tests should be taken on every new build, or in other words when something new is added up. There are various types of regression testings such as Unit RT, Partial RT, Complete RT, and Build Level RT. With the introduction of Agile methodology into software development, regression testing has also become a part of it under the continuous testing schema.
There are many advantages in Regression testing, but they may introduce excessive costs to the software development process. That’s why it has been well studied by many researchers.

Functional testing doesn’t test the compatibility of new features added with the existing application, they only focus on the behavior of the new features and their capabilities. Therefore without Regression testing, it is difficult to find the root cause of the problem of your product. With rapid and automated regression testing your product team can get immediate feedback about bugs or defects early in the deployment cycle. This will prevent excessive cost and maintenance efforts, and therefore it will create a huge advantage in your deployment cycle.

Difference Between Re-Testing and Regression Testing

Re-testing as its name suggests means to “test again”. We perform re-testing when a bug or defect in the source code of our application is modified or fixed, or when the test cases fail or break in the final execution.

In contrast, regression testing is performed to figure out either a change or improvement recently introduced has caused any defects in the existing functionality of your application. You can consider Regression testing as discovering the side-effects of the new changes. Regression testing ensures the unification and integrity of the software.

In a typical software development pipeline, you first perform re-testing and then regression testing. While re-testing, as we mentioned earlier, focuses on broken or failed test cases, regression testing is applied to those who pass the re-testing, and its aim is so to find unexpected new bugs or malfunctioning caused by a new piece of code.

Automated Regression Testing

Companies use automation to benefit from the advantages regression testing brings for them. They use automated regression testing in their software development pipeline. In the long run, automated regression testing will bring up many benefits such as iteration simplification, unified standards across all projects, and saving time.

In automated regression testing, we automate most of our testing efforts and run all the tests we performed previously on new builds automatically. We create a test case set and we know the expected results, therefore automating these test cases would save us time and effort.

Finally, we would like to introduce some automated regression testing tools here, talking about the pros and cons of each of these tools is out of the scope of this article.

Selenium

Katalon Studio

IBM Rational Functional Tester

Silk Test

Watir

These automating tools are using record and playback logic. You will have to define and record the test cases. Definition of expected results also should be provided. Then you will have an insight either the desired results were derived or not.

References
[1] Craig Larman and Victor R Basili. Iterative and incremental developments.a brief history.Computer, 36(6):47–56, 2003.

[2] Wikipedia.Regression Testing, 2020 (accessed December 1, 2020) .https://en.wikipedia.org/wiki/Regression_testing

[3] Shin Yoo and Mark Harman. Regression testing minimization, selectionand prioritization: a survey.Software testing, verification and reliability,22(2):67–120, 2012.

[4] Emelie Engstr ̈om, Per Runeson, and Mats Skoglund. A systematic reviewon regression test selection techniques.Information and Software Technol-ogy, 52(1):14–30, 2010.

[5] Akira K Onoma, Wei-Tek Tsai, Mustafa Poonawala, and Hiroshi Sug-anuma. Regression testing in an industrial environment.Communicationsof the ACM, 41(5):81–86, 1998.