Most of the advancements in artificial intelligence (AI) and machine learning (ML) I’ve heard on my podcast are around functional UI test automation.
One exception was my discussion with Joel Montvelisky, Co-Founder and Chief Solution Architect of PractiTest, a comprehensive test management solution that streamlines QA processes, fosters collaboration, and delivers actionable insights for informed decision-making. This interview enlightened me on how AI can be applied to test management.
This got me thinking.
How can someone use AI or ML to assist them with managing their test and test runs?
So here are three ways I came up with based on Joel's interview.
But first, what is AI test management?
What is AI Test Management
AI in test management isn't just about improving automation scripts or making test execution faster. It's about intelligent decision-making, risk analysis, and optimizing the testing process. AI can analyze vast amounts of data to provide insights on test coverage, effectiveness, and areas that need attention.
Now that we are on the same page, here are the three ways it can help you.
1. AI-Powered Decision Making
AI-powered decision-making in software testing refers to using artificial intelligence to analyze, prioritize, and make decisions on test execution and evaluation. It aims to optimize the testing process, making it more efficient and effective.
In my experience, determining which tests to prioritize can be a challenge.
Enter the “value score” concept introduced by Joel Montvelisky.
This unique metric, generated using AI and machine learning, evaluates the significance of each test case in a database. Analyzing patterns in test results, like how often a test discovers bugs or its pass-fail consistency, the value score identifies which tests are the most “interesting” or valuable. The Value Score AI-powered feature evaluates the significance of your test cases.
This helps teams focus on tests that offer the most insight and prompts discussions on making every test more effective.
How does it work?
The AI algorithm analyzes the history of all your previous test runs to determine which test cases are the most “interesting” or valuable to run. It considers various parameters, including the test's history of detecting bugs, modification frequency, and pass/fail results patterns.
In real-world scenarios, this system proves invaluable. It streamlines your testing process, catches critical issues early, and boosts the testing team's efficiency.
By highlighting the most impactful tests, the value score allocates resources to the most crucial areas, strengthening the entire software development process and making your SDLC process more robust and reliable.
The AI-driven value score not only identifies the most valuable tests but also prompts testers to improve the value score of other tests. It suggests actions like increasing the test's run frequency or modifying it to detect more bugs, leading to a continuous improvement cycle in the testing process.
2. Risk Analysis and Prediction Test Orchestration
Another area where AI Test Management plays a crucial role is in test orchestration.
This can help you and your teams manage and optimize their testing efforts effectively. AI can help you and your teams correctly manage and analyze results from various testing tools. It provides insights into areas of the application that are over-tested or under-tested, enabling teams to focus their efforts where they are most needed.
This is huge!
Some of the best test results I have seen are when teams focus on risk areas, and these insights help you find where to focus your testing efforts.
Joel mentioned that the goal is to trace tests back to user stories to requirements, and by doing so, you start seeing those overcrowded places that are sparsely covered.
To me, this is one of the most powerful features of ML/AI.
The ability to predict potential problem areas in software applications by analyzing historical data and identifying codebase sections prone to bugs is a game changer. Leveraging these new machine learning algorithms, AI can analyze patterns and trends in past test results, bug reports, and code changes to identify areas more likely to contain bugs.
This analysis considers factors such as the frequency of failures, the complexity of the code, and the historical performance of specific modules or components. By identifying these prone areas, teams can prioritize their testing efforts, allocate resources effectively, and proactively address potential issues before they impact the end users.
This predictive capability of AI empowers teams to improve the overall quality and reliability of their software applications.
3. Optimizing the Testing Process
I’m certain this last point will be unpopular with many testers.
Confession: I hate running all my automated tests for every check-in developers make.
To me, it’s not only wasteful and not very scientific, it causes more noise than anything.
What do I mean?
If a developer changed one area of the application, wouldn’t it be better if we knew what tests mapped exactly to that change to verify that the change did not break anything? If a test fails, it is most likely a cause of the change.
The reverse is if you run your complete 2000 test suite and many tests fail. Is it due to the check-in for tests that have nothing to do with the change? Are the tests flaky?
I know there is a better way.
How can AI streamline testing, reducing the time and resources required? Letting AI create the balance between speed and quality ensures no compromises. By automating repetitive and time-consuming tasks, AI can accelerate the testing process, allowing teams to test more efficiently and effectively.
This automation can include test case generation, test execution, and result analysis. AI algorithms can analyze vast amounts of data and identify patterns to optimize the testing process, ensuring that critical areas are thoroughly tested while minimizing redundant or unnecessary tests.
This balance between speed and quality is crucial in software testing, as teams strive to deliver high-quality software within tight deadlines.
AI helps achieve this balance by leveraging its capabilities to expedite the testing process without compromising the thoroughness and accuracy of the tests. By reducing manual effort and providing intelligent insights, AI empowers teams to achieve faster testing cycles, improve time-to-market, and deliver high-quality software to end-users.
But does this work in the real world?
How Facebook uses AI in Test Management (Case Study)
If you don’t know, I have a weekly testing news show covering the most relevant news items in the software testing world.
One LinkedIn article that caught my attention was about how Facebook used AI to significantly reduce testing times while improving test coverage and overall quality.
How?
Facebook has incorporated AI algorithms into its testing framework, enabling the generation of test cases automatically.
By leveraging AI, Facebook has uncovered critical issues early in the development cycle and improved the overall quality of its products. The AI algorithms analyze code complexity, function usage patterns, and historical data to prioritize tests based on the areas most likely to have issues.
This approach has allowed Facebook to optimize its testing efforts, reducing the time required for testing while ensuring that critical areas are thoroughly tested. The result is a more efficient testing process that saves time and improves test coverage and quality, leading to more reliable and robust software releases.
Believer in AI in Test Management?
After speaking with Joel, I believe that AI is not just a tool but a strategic ally in enhancing test management. These are just three examples of what I think will soon be a growing list of other ways AI can assist testers with test management.
By intelligently analyzing data, prioritizing test cases, and predicting risks, AI empowers testing teams to work more efficiently and effectively. The future of test management is not just automated but also intelligent, with AI playing a pivotal role in transforming data into actionable insights for better software quality.
In addition, by leveraging AI, testers can make informed decisions, prioritize valuable test cases, and focus on areas significantly impacting software quality.
Will AI replace the need for a tester in test management?
No!
Remember, AI is a powerful assistant and ally, not a replacement or silver bullet.
So why not try it yourself and see how it can assist you and your teams?
Check out a free trial of PractiTest End-to-End Test Management Platform now and see what Joel was talking about for yourself 🙂
