Regression is ‘‘the phenomenon that the quality of the system as a whole falls back due to individual changes’ (TMap Next, 2006). To prevent damage caused by these software errors, a regression test is performed. The objective of performing a regression test, is to find out if all system parts are still functioning after making one or more changes to the same system. Also, right after solving test defects it’s recommendable to perform a regression test.
In short, performing a regression test is a valuable asset to maintain continuity in your IT landscape.
A list of 10 important aspects to consider, when building your regression test set:
- State in the acceptance criteria of each project, to maintain and overall test ware to the organisation, as soon as the project ends. This test ware contains a good basis for the regression test set of the developed system(parts). Re-using test ware saves time and prevents having to invent the wheel again.
- The Regression test focusses mainly on the connection of changed and non-changes parts, as the chance of errors is bigger there. Tests regarding functionality within a system part aren’t as relevant.
- Limited time and resources are always a challenge in every form of testing. Therefore, it is recommendable to use product risks to base your regression set on. The system parts that cause the most damage when containing errors, are tested first and most thorough.
- Although in projects the happy flow as well as negative paths are tested, it would be expedient to confine the regression test to only the happy flows.
- A big challenge in Regression testing is to keep the test cases and test scenarios up-to-date. So, make sure you draft the regression set in such a way that it will be easy to maintain. It functionality changes, the tests should be adjusted as well. In Testersuite, test scenarios are composed of test cases. In case of a modification, only one test case needs to be changed to adjust all test scenarios containing this test case.
- Although composing a regression test set is a time-consuming business at first, the invested time will return the longer the test sets are re-used. Completely besides the (cost)return by prevented interruptions of productive systems.
- At large releases, often a complete regression test set is performed. However, it is not exceptional that a small ad-hoc change causes large regression problems. The conduct to solve these production issues fast is quite often bigger than the sense to test every solution thoroughly on regression risks. By making a regression test set scalable, it’s possible to just plan a subset of tests for each change. This way, one can use the test set both for ad-hoc changes and large planned maintenance.
- Maintaining large numbers of test cases and test scenarios is a huge challenge, especially since the results of executed test cases and test scenarios should be preserved as well. Content management of test cases and test scenarios on a network station is a full-time job. Make sure to keep track and revision control by central storage, for example by using a test management solution such as Testersuite.
- WWho is the owner (a.k.a. guardian!) of the regression test set? This should preferably be someone with testing skills, who isn’t involved in the project organisation. This person can oversee the use and re-use of the regression test set and take care it is adjusted and maintained if needed.
- Whenever the regression test is performed more often, the test set will get more detailed and better. This enables to eventually automate partly (!) the tests that are performed frequently and don’t change much. This will save considerable time!
Regarding small changes, the test effort to exclude regression problems can be many times bigger than the effort to test the change itself. For this reason, it is advisable to consider implementing multiple changes by release. This way, a bigger regression test can be planned and the effort will be more proportionate to the number of changes.
Work towards a mature regression test set
Next to the 10 principles mentioned above, the main principle of regression testing is off course to ensure continuity towards the business (users, customers, suppliers, etc.). Improve your test set step by step toward a mature regression test set. We would be happy to support or help you simplify your task with our solutions.« back