Call center workers (Getty Images)

How can AI direct your call

Finally, hard data on a real-world AI business use case: It’s huge for customer service

Researchers measured gains and trade-offs from human customer service reps using an AI assistant. One big win? Fewer angry customers.

Jon Keegan

2/13/25 11:52AM

The tech giants are spending hundreds of billions to build out massive infrastructure needed for an imminent economy where AI supercharges productivity and boosts economic growth. That’s the theory, anyway. But how much will AI actually boost productivity in business? It’s very much an open question, with a paucity of data available.

But in the current issue of the Quarterly Journal of Economics, a research paper presents some hard data on the topic. Titled “Generative AI at Work,” the paper shares research by Erik Brynjolfsson, Danielle Li, and Lindsey Raymond, which measured productivity of real workers in an industry prime for AI-powered improvements: customer service. Their findings offer an important, early look at the value of incorporating AI in business.

The researchers found that when human customer service agents used an AI assistant, the company saw a 15% boost in productivity on average. But the gains were not evenly distributed. Less experienced agents saw the biggest boosts in productivity and speed, while more experienced agents saw smaller gains in speed and a slight decrease in quality. AI really helped when handling “moderately rare problems,” where the human agent might lack the knowledge to resolve the issue.

One of the other benefits of the use of AI assistants was that it helped international workers improve their fluency in English. After AI was introduced, the data showed big jumps in scores for “comprehensibility” and “native fluency” (i.e. the person seemed like a native American English speaker).

Most of us have had a frustrating customer service experience and the accompanying rage that it may induce. The study notes, “We see regular instances of swearing, verbal abuse, and ‘yelling’ (typing in all caps).” This can lead to attrition among customer service workers, which adds to costs.

“AI assistance significantly improves how customers treat agents”

In what might be the most promising (and surprising) finding in the study, the authors found that AI-assisted customer service resulted in customers being more polite and less likely to ask to speak to a manager.

“We find access to AI assistance significantly improves how customers treat agents of all skill and experience levels, with the largest effects for agents in the lower to lower-middle range of both the skill and tenure distributions,” the authors wrote.

The study followed 5,172 customer support agents at a Fortune 500 firm that sells business software. They staggered the introduction of an AI chatbot assistant that suggests responses to customer queries over eight months starting in late 2020 through 2021.

It’s important to note that in the fast-moving world of AI, this experiment took place a relatively long time ago. The chatbot used in this study was built using GPT-3, the model that preceded ChatGPT’s launch in November 2022. It was trained using successful customer service calls from top-performing agents.

As tech giants like Microsoft, Google, and Salesforce race to squeeze AI into the software we use, perhaps the greatest benefit will be that it not only helps us solve our problems faster, but it makes humans treat each other better.

Rani Molla2h

Elon Musk’s SpaceX reportedly in talks to merge with xAI

Tesla CEO Elon Musk is reportedly exploring a merger between SpaceX and his artificial intelligence startup xAI, a move that would bundle rockets, satellites, the social media site X, and AI under one company ahead of SpaceX’s long-anticipated IPO.

According to Reuters reporting, the deal would swap xAI shares for SpaceX stock, potentially valuing the combined operation north of $1 trillion.

Reuters reports:

Two entities have been set up in Nevada to facilitate the transaction, the person said.

Reuters could not determine the value of the deal, its ‌primary rationale, or its potential timing.

Corporate filings in Nevada show that those entities were set up on January 21. One of them, a limited liability company, lists SpaceX and Bret Johnsen, the company's chief financial officer, as managing members, while the other lists Johnsen as the company's only officer, the filings show.

The combined companies could also set the narrative groundwork for putting data centers in space — an idea that Musk and a number of other tech billionaires have been floating lately but that may not get off the ground.

In its earnings filings yesterday, Tesla disclosed that it recently made a $2 billion investment in xAI. Last year Musk’s xAI bought Musk’s X in an all-stock deal.

Exclusive: Musk's SpaceX in merger talks with xAI ahead of planned IPO, source says

Reuters reports:

Two entities have been set up in Nevada to facilitate the transaction, the person said.

Reuters could not determine the value of the deal, its ‌primary rationale, or its potential timing.

Corporate filings in Nevada show that those entities were set up on January 21. One of them, a limited liability company, lists SpaceX and Bret Johnsen, the company's chief financial officer, as managing members, while the other lists Johnsen as the company's only officer, the filings show.

In its earnings filings yesterday, Tesla disclosed that it recently made a $2 billion investment in xAI. Last year Musk’s xAI bought Musk’s X in an all-stock deal.

Translating Microsoft’s CEO Satya Nadella

Translating Nadella’s jargon to understand his strategy for meeting intense demand for AI computing.

Jon Keegan2h

Rani Molla3h

Driverless Waymo struck a child near school in California

A Google Waymo struck a child near a Santa Monica elementary school during morning drop-off last week, as self-driving cars by Waymo, Tesla, and others continue their expansion across the country. In a blog post, Waymo said the fully driverless car detected the child as they emerged from behind a parked SUV, braked sharply, and reduced speed from approximately 17 mph to under 6 mph before striking the child. The child suffered minor injuries and walked away.

The company reported the incident to the National Highway Traffic Safety Administration, which is currently investigating, adding fresh scrutiny to how robotaxis perform in the wild.

Jon Keegan5h

Digging into Microsoft’s cloud backlog

Microsoft’s Azure cloud computing unit is seeing huge demand. In yesterday’s second-quarter earnings call, Microsoft CFO Amy Hood said the company’s commercial bookings increased 230% thanks to large commitments from OpenAI and Anthropic and healthy demand for its Azure cloud computing platform.

Hood said that the company’s “remaining performance obligations” (RPO) ballooned to a staggering $625 billion, up 110% from the same period last year. How long will it take for Microsoft to fulfill these booked services? Hood said the weighted average duration was “approximately two and a half years,” but a quarter of that will be recognized in revenue in the next 12 months.

Shares of Microsoft tanked today, down over 11%, despite the strong beat on revenue and earnings. The drop puts the stock on track to have its worst single-day drop since March of 2020.

Investors may be concerned that while huge, that extra demand was coming only from OpenAI, an issue that Oracle recently experienced.

But Hood said the non-OpenAI RPO still grew 28% year on year, which reflects “ongoing broad customer demand across the portfolio.”

Meta and Tesla are funding the future with their core businesses — but only one of them is still growing

The two tech giants, on back-to-back earnings calls, made it sound like they’re selling the same AI-powered future. But the picture of the underlying businesses, and how they’re using AI to furnish current sales, couldn’t be more different.

Rani Molla5h