close
close

Businesses looking to implement artificial intelligence face a difficult reality

Posted on by darion

Enterprises trying to leverage generative AI to increase productivity and grow revenues struggle with expensive, immature technology with unattainable return on investment.

This is according to recent reports from KPMG, McKinsey & Co., and Goldman Sachs, which reflect the GenAI experience of many large organizations. A critical challenge with this technology is generating a return that justifies the high costs of deploying GenAI infrastructure at scale.

A recent AI study by management consultant McKinsey found that only 11% of surveyed companies had implemented GenAI at scale, Aamer Baig, a senior partner at the firm, told attendees at the MIT Sloan CIO Symposium in May. In addition, only 15% of respondents reported improved profits with GenAI.

Cost justification is critical because AI infrastructure is expensive. For example, Nvidia’s previous-generation H100 AI GPU costs about $30,000, and the number needed can range from a few hundred to thousands, depending on the model and its size.

“The technology is brand new, barely works and is very expensive,” said Anshul Chaturvedi, managing director of World Wide Technology (WWT), an IT services provider.

The large language model (LLM) today is a “black box” that is difficult to make accurate and provide consistent answers, said Rob Mason, CTO of Applause, a company that tests AI results in software from a user perspective.

“In the companies that are building GenAI, people are examining what they’ve built to figure out what it will do, as opposed to what they want it to do,” Mason said.

The Fortune 100 company that hired AI applications specialist Trustbit, which was acquired this year by IT consulting firm Timetoact Group, had to deploy three different models internally to ensure response accuracy of at least 95%, said Rinat Abdullin, a technical consultant at Trustbit.

If three models gave the same answer, it was considered correct. If it was worse, a human would decide which answer was correct. Accuracy was critical because customers could hold the company accountable for an incorrect answer.

“Companies don’t always need 100% accuracy, but they do want to make sure that when the model gives an answer, if it’s not sure, they know about it,” Abdullin said.

Transitioning to Revenue-Generating AI

Some companies are strategically shifting their GenAI use from employee productivity to revenue generation, according to a second-quarter KPMG survey of 100 U.S.-based CEOs and business executives. The respondents come from organizations with annual revenues of $1 billion or more.

In Q1, using GenAI to boost employee productivity was the top ROI metric for 51% of respondents, the KPMG study found. In Q2, revenue generation became No. 1 at 52%, while improving productivity fell to No. 3 at 40%.

“Leaders are starting to see GenAI investment and adoption as a stake,” said Steve Chase, vice president of AI and digital innovation at KPMG. “They are now focused on how to translate that investment into competitive advantage.”

Chaturvedi, who has seen a similar shift among WWT customers, said he believes it’s more about confusion about how to get the most value from GenAI. He recommends starting with applications that improve employee productivity and customer service before moving into more expensive, revenue-generating use cases.

“Using it to increase employee productivity is about creating a small, safe playground where if you make a mistake, it doesn’t matter as much,” Chaturvedi said.

The model of immaturity

The hype about GenAI’s potential to transform businesses belies the immaturity of the technology and the need for much more research. Daron Acemoglu, an economics professor at the Massachusetts Institute of Technology, said he predicts GenAI won’t be ready to drastically change business operations for at least 10 years.

“Many of the tasks that humans perform today, such as in transportation, manufacturing, mining, etc., are multi-faceted and require real-world interaction that AI will not be able to improve anytime soon,” Acemoglu said in a Goldman Sachs GenAI report.

Acemoglu said the current architecture used in LLM will need to change to mimic many types of human cognition, its ability to process sensory stimuli and its ability to reason.

“Today’s large language models have proven to be more impressive than many people could have predicted, but it still takes a huge leap of faith to believe that a next-word-in-a-sentence prediction architecture will achieve capabilities as intelligent as HAL 9000 in 2001: A Space Odyssey,” he said.

Antone Gonsalves is TechTarget Editorial Editor, covering industry trends that matter to enterprise technology buyers. He has worked in technology journalism for 25 years and is based in San Francisco. Got a story? Email him.