Leapwork survey shows that only 16% of companies find their software testing efficient

Although 85% of total respondents have integrated AI apps into tech stacks in the past year, most (68%) have experienced issues with their performance, accuracy, and reliability.

Copenhagen, September 17, 2024 – Against the backdrop of high-profile outages disrupting global infrastructure, only 16% of businesses think their current testing practices are efficient. The surge in AI adoption is exacerbating this issue. This is according to research from Leapwork, surveying 401 senior/technical professionals in the US and UK.

Although 85% of total respondents have integrated AI apps into tech stacks in the past year, most (68%) have experienced issues with their performance, accuracy, and reliability. The lack of sufficient testing processes is having a detrimental impact on the usability of AI apps, which are becoming more and more widespread. For C-Suites, which accounted for half of the overall survey, this perception is even higher at 73%, compared to 62% for software engineering or technical leads, which made up the other half of the sample. This has made the need for thorough software testing greater than ever.

"For all its advancements, AI has limitations, and I think people are coming around to that fact pretty quickly," says Robert Salesas, CTO at Leapwork. "The rapid automation enabled by AI can dramatically increase output, but without thorough testing, this could also lead to more software vulnerabilities, especially in untested applications. It makes sense that C-Suite executives would be especially sensitive to this because of the implications for customer experience and negative publicity. There’s an opportunity here for cross-industry collaboration to ensure more testing tools are up to scratch for the challenges of the modern world where AI apps are more and more widespread."

The rapid advancement of AI apps has presented challenges when it comes to onboarding them into enterprises' workflows. Specifically, the most common bugs in AI apps identified by respondents were integration failures (21%) and security vulnerabilities (23%). Respondents also identified the following three top challenges while trying to integrate AI into their software:

• Resistance to change within the organisation (20%)
• Inconsistent performance and reliability of AI applications (19%)
• Managing the rapid pace of AI advancements and updates (19%)

Consequently, most organizations (77%) now see testing of AI as essential, but notable gaps exist in testing resources and practices. Nearly a quarter of organizations (24%) do not have a dedicated team or individual responsible for testing AI apps, and over a quarter (26%) do not have a commercial testing platform. Nearly a third (30%) believe their current testing processes cannot ensure reliable AI apps.

"There have been too many outages this year alone, many of which affected millions of customers for big brands. We've been given a wake-up call no one can ignore," says Christian Brink Frederiksen, CEO of Leapwork. "What makes digital infrastructure today so tricky to test is the copious amount of complex, interconnected applications. A tiny error in one application could have a monumental cascading effect and shut down businesses. Whether big or small, all updates need appropriate testing, but many businesses have outdated, siloed approaches. It shouldn’t be about testing one individual app – it should be about testing the entire user journey."

With AI now embedded in nearly every facet of business operations, the stakes have never been higher for ensuring these systems perform flawlessly. CIOs and CTOs must recognize that the future of their digital infrastructure depends on moving beyond traditional, isolated testing methods. A holistic approach that ensures every application and user journey is thoroughly vetted will be key to maintaining both operational resilience and customer trust in this new era of AI-powered solutions.

EDITOR'S NOTE

Leapwork commissioned research via Censuswide to gather responses from 401 respondents across US and UK organizations. These included 201 C-Suite executives (CTO/CIOs) and 200 technical leads (including VP of IT, Director of IT, IT Manager, software engineering leads, QA test manager/director. The respondents were surveyed across sectors that include technology/software, finance and banking, healthcare, manufacturing, retail, and government. The respondents were aged 18-55+. Find the report here.