Testing the ‘Most Valuable’ AI Investment Across the Software Development Lifecycle – Intelligent CIO North America

A report by Tricentis reveals that AI-powered DevOps tools will save teams around 40 hours per month, which is equivalent to an entire workweek.

Tricentis has released the results of its latest AI report: AI-augmented DevOps: Trends Shaping the Future.

This year’s research aims to understand to what extent the expected benefits of AI in DevOps have already been achieved and how lack of trust, skills or other challenges may impact its implementation.

When asked to rate the areas that have the greatest impact on AI investments throughout the delivery cycle – such as planning, coding, deploying, and releasing – DevOps practitioners ranked testing as the most valuable (60%).

This finding was foreshadowed in a 2022 Tricentis study, which found that testing is where organizations expect the greatest value from AI-powered DevOps, with nearly 70% of respondents rating the potential of AI-powered testing as extremely or very valuable.

Research has shown that DevOps teams are seeing the benefits of AI, and mature DevOps teams that have implemented AI are significantly more likely (30%) to rate their teams as extremely or very effective.

The biggest challenges faced by DevOps teams using AI are developer team productivity (60%), closing the skills gap (54%), cost reduction (47%) and software quality (42%).

Nearly one-third (32%) of respondents estimate that AI-powered DevOps tools will save teams more than 40 hours per month – the equivalent of an entire workweek.

The 2024 results show that teams are using AI to augment a wide range of testing tasks, including test planning/deciding what to test (47.5%), test case generation (44%), and test results analysis (32%).

Additionally, nearly half (42%) of respondents expect AI to perform risk analysis of code changes, helping QA teams focus on areas of the code where the risk of quality errors is highest.

The report surveyed more than 500 DevOps practitioners, managers and directors from small, medium and large enterprises around the world and across industries including financial services, healthcare and manufacturing.

Other findings from 2024 reveal:

Regulation is expected to help build trust, but others fear it will limit innovation and performance. Nearly two-thirds (63%) of respondents believe that tightening regulations is a way to build trust in AI across the organization, while a smaller – but not insignificant – group of DevOps practitioners (16%) believe that tightening regulations will hinder or stifle the potential impact of AI on organizations.
Humans still play a critical role in software quality, but AI skills are in short supply. The data shows that people are still very much on the fence, with more than two-thirds (71%) of respondents checking results at least half the time, and almost one in five (19%) saying they check AI results all the time. However, a lack of AI skills (28%) is seen as the biggest barrier to AI adoption in DevOps.
Generative AI (Gen AI) and AI co-pilots are key drivers for AI adoption: GenAI (45%) is currently the most widely adopted type of AI used by DevOps practitioners. In particular, there is also a growing number of AI co-pilots whose use cases include planning, code development, and software testing.

“Artificial intelligence is an exciting technology that is evolving at a pace never seen before in our industry,” said Mav Turner, chief product and strategy officer at Tricentis.

“As AI technologies continue to evolve, training software development and quality engineering teams in the skills needed to work effectively with AI will be critical.”

“DevOps teams looking to start using AI should look no further than their testing processes. AI in testing helps detect, automatically fix, and predict defects during development, as well as identify which tests to run based on high risk.

“This, combined with low-code/no-code technology, means that regardless of the technical competence of the team, AI can significantly contribute to the overall quality of the software.

“As DevOps teams mature, testing will be critical to realizing their investments in AI-powered DevOps tools and practices.”

Click below to share this article