Last week I gave a challenge to my colleagues – Find one industry that does not use smart machines or artificial intelligence. And to our surprise, we all found it difficult to discover one industry that DOES NOT use any form of smartness or intelligence embedded in machine/software. A lot of industries are already leveraging AI to transform business operations and boost workforce productivity.
To start with a few industries, the automotive industry has revolutionized the way AI can be used, giving us self-driving cars. In marketing, we already use tools that give us insight into lead qualification and let us know which leads are worth pursuing and which need more nurturing. In the industrial sector, AI tools play an important role in predictive and prescriptive analysis to understand which machines need servicing and which functioning optimally. Another such industry is healthcare where the prediction model has already proved itself pivotal in finding patterns and backtracking diseases. Software testing is not far behind. AI in software testing is expected to make testing more exact, viable, and convey better caliber. The AI-driven software testing effectively improves automation, reduces duplicity of tasks, improves test maintenance, and offers better quality analytics/insights.
Current Trends and Statistics
According to the World Quality Report 2020, 51% of enterprises want to maximize test automation as part of their agile QA process. Moreover, digital transformation has taken the front seat in most enterprises, especially in the wake of the COVID-19 pandemic. This pitches AI and ML as the favorite contenders to implement the use of technology in software testing.
This brings us to the question of where CIOs and company leadership will direct funding. A Gartner study states that IT/Technology counts for 6.9% of spending in most organizations. These companies are leveraging emerging technologies and slowly replacing or modernizing legacy tools after the rise of digital transformation. In testing, AI and ML will increase continuous integration and delivery and create a feedback loop to improve the efficiency of automated QA dashboards.
The World Quality Report 2021 mentions that 88% of organizations believe AI to be the strongest growth factor for testing activities while 80% plan to use AI to generate test environments and test data.
QMetry’s Approach to Foster AI-Driven Software Testing
QMetry is the #1 Digital Quality Platform with a focus on revolutionizing the software testing process through AI-driven test authoring, test execution, and reporting for Agile/DevOps teams.
There are several layers to the QMetry Digital Quality Platform. They are test authoring, test execution, test planning, and most importantly, shareable, traceable, and visible reporting capabilities across the board.
Test Authoring and Test Generation
Speaking of test authoring, QMetry Digital Quality Platform offers an easy way to create test cases for existing applications such as Salesforce or SAP-based apps which usually have standard workflows. Automated test authoring is slowly becoming the accepted means to generate test cases. This is where the Automated Test Creator module comes in. It supports exploratory testing and auto-generates test cases based on the application being tested using model-based testing technologies and AI algorithms. The algorithms leverage user behaviour to enable testers to create relevant test cases. Once these test cases are created, their execution is automated as well. The module also offers self-healing capabilities so that bugs and errors in test cases can be rectified in a timely manner.
Test Coverage and Analysis
After test authoring and test execution, it’s time for test analysis. This is tackled in two phases in the Digital Quality Platform. First is QMetry Wisdom which leverages real-time optimization and actionable intelligence. But what do they do? Let’s take an example. A user is running a series of P1 regression tests. Now based on historical data, the user may find that if P1 regression tests pass, there is no need to conduct P2 or P3 regression tests. QMetry Wisdom will automatically ensure the continuous delivery pipeline by indicating that the user can skip P2 and P3 test cases as the P1 test cases provide enough test coverage depending on the risk the user is willing to take. This risk-based testing approach enables organizations to reduce cost of testing along with the cost of infrastructure needed for it.
Auditing and Compliance
Finally, QMetry Digital Quality Platform also provides compliance management functions to help organizations, especially in the healthcare and pharmaceutical sector to stay updated with and follow compliance regulations properly. Features like eSignatures and audit and compliance workflows will enable enterprises to accelerate their digital transformation journey.
All in all, QMetry Digital Quality Platform offer various benefits-
- 50% faster go-live speed achieved by test automation and self-healing test cases
- 30% increase in ROI since testers will not be spending too much time writing test cases thanks to automated test cases generation and assurance of continuous delivery
- 25% improvement in Quality from Day 1 brought about by reducing manual test authoring and execution and planning test activities from the get-go
Integration with multiple tools is an essential aspect of successful testing. In an organization there are various tools like project management applications, test automation tools, open source and proprietary tools.
QMetry Digital Quality Platform integrates with several tools like Jira, Jenkins, Selenium, SauceLabs, WorkSoft and many more.
AI in Testing – Use Cases
Use cases for AI in testing will range across all three layers – test authoring, test execution, and test analysis. Within these layers, AI can be leveraged through detection, prediction, and risk analysis.
The detection phase with AI in testing has many aspects to it. Let’s look at them-
AI can identify flaky test cases ahead of time. Flaky test cases are those that fail to produce the same result each time an analysis is run, without any change in the test case. They lead to false positives and incur unnecessary infrastructural costs while testing. To identify this flakiness, automation powered by AI plays a big role.
Test Execution Trend
Test execution trends depict the overall picture of the entire testing process. Testers can identify failed test cases against passed test cases, unused/obsolete test cases that are clogging the pipeline and even test scripts that have failed the greatest number of times. Once identified, these factors can lead to a better, seamless, and optimized testing cycle.
Test Execution Time Analysis
Test cases are executed on multiple browsers, for different devices making it a time-consuming activity. AI makes it easy to uncover data such as test steps taking the most time, browsers or mobile devices that take up more time and multiple execution cycles, and finally identify the slow performing test cases.
There are many application suites such as Salesforce, SAP, or any Oracle suite of applications that offer quarterly releases. In such cases, AI-enabled comparative analysis helps identify the changes between the previous releases and the new versions. Based on those changes, testers can find new test cases to run or change. This comparison will give testers a head start for the new test cycle
Predictive and prescriptive analytics are an important piece of the testing cycle. They are carried out in various ways-
Defect prediction will give testers an indication based on historical data about the likelihood of defects in the test release cycle. While it may not provide an accurate measure of the actual defects, it will serve as a red signal ahead of time so that testers can prepare ahead of testing. The AI-driven prediction will help in better test planning to ensure that test cases are updated or changed, thereby minimizing defects in the test cycle.
Capture User Behaviour
Feedback mechanism or understanding user behaviour is still an unexplored territory in the field of testing. This is where AI can be leveraged to capture user behaviour and prescribe the optimum test cases or scripts for execution. For example, if analytics suggest that Google Chrome is used by the maximum number of users, then testers can optimize testing sprint on a particular browser as opposed to other browsers that are less used by the users.
With the help of AI, testers can see the most updated and trending test data. Based on test execution trend analysis, prescriptive analytics can prescribe the optimum test plan for the current test cycle. Thanks to prescriptive analytics, trending data is available earlier in the project lifecycle rather than later.
AI plays a huge role in risk analysis as a potential catalyst to improve and optimize the testing cycle and software quality. Risk Analysis is done in several steps-
Root Cause Analysis
Root cause analysis identifies the failures in testing and the reasons behind them. In situations where there are hundreds of test cases being run, and majority of them fail, testers have to spend precious hours in debugging issues. Thus, it is important to have a mechanism in place that pinpoints the real issues behind the failures. Once the issues are uncovered, testing teams can isolate them to understand each of them better and start working on them earlier in the lifecycle. This makes the entire process of risk analysis less time-consuming and complex.
The risk-based Testing approach offers great value in terms of its use of AI. Risk-based testing provides a convenient way to understand the risk appetite of the testing team. For example, if the risk appetite is 70%, then testers may need to run only 100 test cases in the regression cycle instead of 5000 test cases. Testing teams can save a significant amount of time and effort by clearly earmarking the risk margin for a particular test cycle.
Risk Identification and Impact Analysis
Since testers can identify the risk earlier in the test cycle, they can also gauge the impact it might have on their outcome. For example, if testers make changes to the login module, impact analysis can show how and if the changes will affect other modules. It will also depict which testing layer- be it functional, non-functional, performance or security testing will be impacted.
The use of AI in software testing is essential at the very base level. AI can transform every aspect of the testing process and affect change in improving software quality. These are only a few use cases where AI can work its magic. Test engineers around the world are finding new use cases to leverage the power of data engineering.
QMetry’s AI-enabled Digital Quality Platform is one step forward towards making the use of AI in software testing a possibility. Our Digital Quality Platform unlocks the power of data (such as project documentation, test artefacts, defect logs, test results, production incidents, and so on), identifies defects ahead of time, optimises testing, and predicts failure points — all while lowering overall costs and increasing customer satisfaction.