Coders develop test scenarios and exercises that test new bits of code after they are written. These tests are part of the ‘test bucket’. Before new product version release, old test cases are run against new version to ensure the old functionality is intact. The reason for this is that sometimes, adding new code can introduce errors into the code.
Regression Testing often means partial or full selection of tests that have been already executed, to run again.
When is it required?
Regression Testing is necessary when
- Code is modified due to changing requirements
- New features are added to software
- After defect fixing
- Performance issue fix
- Patch fix
- Major changes
- Configuration changes
Considering the large volume of test cases and the need for speed, regression testing is very often automated.
Take for example, there is an app that records details of all employees in an office. The app has four key buttons: Add, Save, Delete and Refresh. All buttons function as expected. So, now we need to add a new button ‘Update’ to this application. The Update button functionality has been added to the code and tested to be fine. But we need to test if the introduction of this new functionality doesn’t break the existing buttons and how they functioned.
It becomes necessary to test all the functionality to ensure that other buttons/features are still working as expected. If not, the new issues in the code are fixed.
This is how the regression testing process typically looks:
Unit level regressions tests are executed to validate code that is modified, and other new tests that have been written to cover new functionality.
Merge and integrate the changed code to create a new build
Execute smoke tests to assure that build is good to go, before performing other tests. The tests may be automated or could be executed automatically by using CI servers like Jenkins or Bamboo.
Once the build is confirmed as green, sanity testing is performed and you can solve known defects before carrying out integration testing.
Next, integration regression tests are carried out to ensure interaction between components of the application and with back-end services.
Finally, system and UAT tests are conducted to make sure that the application’s functionality and performance are as expected. Based on the scope and volume of code, a full or partial regression will be required at this point. Next, defects are reported to the development team and might need a few iterations of regression testing before resolution.
This completes the process.
Types and aspects of Regression Testing
This is done during the unit testing phase. The code is tested independently without any dependencies on the unit.
Partial regression is carried out to ensure that the code works even with the changes, and that unit is integrated with the code. Tests are chosen based on priority of test cases, or the modules that they cover.
Complete Regression testing
Full regression testing is done when a code change is done on a number of modules and in cases where there is no certainty of the impact of code change on modules. Complete Regression testing is done when a change in code is done on a number of modules and also if the change impact of a change in any other module is uncertain. But it is expensive and time consuming. Also known as retest all technique, this is only used for major releases with significant code changes.
Sanity testing is an aspect of regression testing where testers verify if the whole functionality is working properly. The objective is to ensure that new features are working as expected and that defects previously reported are resolved.
Also known as Build Verification Testing, it is the process to check whether deployed builds are stable or not, so the QA team can proceed with further testing. Put simply, we are ensuring that the most important features are working and there no showstoppers in the build.
Optimizing Regression Testing for Agile
- Regression testing can cause delays and challenges for Agile teams. This is because it means increased time and efforts. Often, teams spend the full sprint on regression. The increased time and efforts add up to the costs. How can you create an effective regression strategy for agile teams?
- Prioritize test cases with critical functionality and core features and those that pose bigger risks of failure
- Choose test cases that have the highest code coverage or those that had the maximum number of defects in the past.
- Break the regression testing into two different cycles – iteration regression and complete regression. Where the team performs iteration regression at the end of each sprint and engineers run the full regression before product releases and major milestones.
- Create a test-first culture. So that developers create automated unit tests for new code.
- Use CI server to run unit and integration tests for every build.
- Always ensure that your final check off includes regression testing and that all software changes have passed regression tests.
Regression tests are ideal candidates for test automation. They need to be repeated often and are comparatively stable. Automating your regression testing suite allows you test in parallel or on short notice and ensuring their continuous integration.
It is one of the most important aspects of testing because it helps to ensure the overall quality at various stages in the software testing lifecycle.