Automate the Testing of Software Updates and Patches to Comprehensively Ensure Software Quality

Updated: Aug 5, 2019

Here we delve into automated process regression testing, considering the opportunity for automating a common and important process in software development and roll-out of software updates. The largest benefit of automated process regression testing is agility. With faster testing, Quality Assurance teams can focus more on analysis than on the actual testing and collection of test data. They can do their jobs more effectively, so that you can be more agile and make changes quickly to keep your project or initiative on track.

Regression testing is a branch of software testing that confirms a recent program or code change has not adversely affected existing features. It can encompass a full or partial selection of already executed test cases which are re-executed to ensure existing functionalities work as intended. It is important to ensure that new code changes do not have side effects on existing functionalities by reconciling outputs from each test process within the application with the expected outputs.


Regression testing is required when there is a:

  • Change in requirements and code is modified according to the requirement

  • New feature added to the software

  • Bug or defect fix

  • Performance issue fix


The largest benefit of automated process regression testing is agility. With faster testing, Quality Assurance teams can focus more on analysis than on the actual testing and collection of test data. They can do their jobs more effectively, so that you can be more agile and make changes quickly to keep your project or initiative on track.


With that increased speed and agility also comes an opportunity for performance tuning throughout the process. Using an automated testing process and the tools that help facilitate that process, you can get a better look at opportunities for performance enhancement within your software, looking ahead at what could impact your performance in the future.


The Bot in a Nutshell:


RPA applied to process regression testing would involve a robot undertaking pre-configured or dynamic tests of individual and combinations of software functionalities through its ability to interact directly with the UI environment and record corresponding application outputs.


The result will be a more structured, efficient, and informed approach to regression testing, without the need for a human to perform a large quantity of highly repetitive operations manually, and where the chances of missing small deviations in outputs due to human error are eliminated.


Regression testing is a notoriously expensive exercise, requiring a large amount of time and resources. It is common practice to use test cases based on business impact, criticality, and frequency of use prioritization to make the task manageable. However, this results in only a subset of the software being tested with each update. With RPA, a robot can quickly and robustly retest all functions, including many combinations of functions by scaling up the testing process. The application of RPA to regression testing will thus result in a faster and more comprehensive assessment of software quality.


Importantly, as well as testing updates and patches to applications in isolation, the robot would check that integrated software was not negatively impacted, validating that data continues to be passed between applications as expected. Integration testing will take into account the communication flows and detect any collateral effects that updates may have against other connected components in the IT ecosystem. It opens the door to holistic process regression testing automation.


The robot can also be integrated with other solutions such as BPM to help automate a complex process regression testing end-to-end, inclusive of integration between different components (e.g software, applications) of the system environment.


As well as regression testing, this robot can be extended to:

  • Integration testing by ensuring that installed patches and hotfixes do not negatively affect other software

  • User Acceptance Testing (UAT) to ensure that new applications meet user requirements


Main Functions:

  • Automate testing of software or software changes through direct interaction of the robot with the application UI

  • Collection of application outputs data e.g. text, documents, screenshots etc.


Results:

  • More comprehensive set of regression tests

  • Structured and organised outputs making it easier to compare results between tests over different software versions

  • Reduced labor and cost

  • Improved quality and accuracy

  • Improved employee morale

  • Cycle time improvements and deployments


High-Level Plan


Main Benefits:

  • Customer satisfaction

  • Process reliability

  • Cost & scalability


Hands-On Considerations


Pain points targeted:

  • Manual and resource intensity of software testing

  • Slow turn-around time when trying to update software due to time-lag in testing


Challenges to expect:

  • UI changes between software updates which alter the underlying structure of the UI environment

  • Difficulties where more ‘human’ inputs are needed to run the application


What will the bot do?