Amazon CEO Andy Jassy at AWS re:Invent 2024 (Noah Berger/Getty Images)

super mega ultra

Amazon’s AI plans: custom chips, an Anthropic “ultracluster,” and its own foundation model

While Amazon’s new models appear to be competitive in terms of features and performance, that isn’t the main thing that the company is touting — it’s the cost.

Jon Keegan

12/4/24 2:11PM

This week at Amazon’s AWS re:Invent conference in Las Vegas, the company fleshed out its plans to both serve and compete with the larger AI industry.

AWS is largely AI agnostic. Customers can use pretty much any of the major AI models on the cloud-computing platform, running on servers that use chips from Nvidia, AMD, Qualcomm, and others.

But Amazon has also been building and selling computing powered by its purpose-built AI chips, including its latest Trainium2 chip, which Amazon is now making widely available on AWS’s EC2 service. Amazon says these new Trainium2 instances are built for training and deploying jumbo-sized large language models with better price performance than its current offerings.

Amazon also deepened its partnership with AI startup Anthropic, announcing that it’s building an “ultracluster” of “hundreds of thousands” of Trainium2 servers to train Anthropic’s next-generation LLM. Amazon recently doubled its investment in Anthropic, bringing the total to $8 billion.

Probably the most significant announcement was Amazon’s late entry to the foundational AI-model club. Named “Amazon Nova,” the new LLM comes in four flavors: a text-only Micro and three multimodal models, Lite, Pro, and Premier. Amazon touted benchmark scores for the Nova models, which place it in the same class as OpenAI’s GPT-4o and Meta’s Llama 3. Amazon’s multimodal Nova models can ingest and generate images and videos, like many of the other top models out there today.

While Amazon’s new models appear to be competitive in terms of features and performance, that isn’t the main thing that the company is touting — it’s the models’ low, low cost.

Running Amazon models on Amazon servers, powered by Amazon chips, yields significant cost savings and low latency. Amazon says its Nova models are “at least 75% less expensive” than the best-performing models available on AWS today.

Rani Molla4h

Lyft and Uber jump after announcing expanded robotaxi partnerships with Nvidia

Uber and Lyft both announced expanded AI and autonomous vehicle partnerships with Nvidia at the company’s GTC event, sending both ride-hailing stocks up after-hours.

Uber was recently up 3.3%, while Lyft rose 3%.

Uber said Nvidia-powered Level 4 robotaxis will launch on its platform in Los Angeles and San Francisco in 2027, with plans to scale to 28 cities globally by 2028. Meanwhile, Lyft said it will use Nvidia’s AI infrastructure to improve ride-matching, mapping, and efficiency, while also using Nvidia’s DRIVE Hyperion platform as a foundation for future autonomous fleets.

Separately, Nvidia announced expanded autonomous driving partnerships with Kia and Hyundai.

The announcements highlight Nvidia’s growing push to provide the AI hardware and software powering next-generation robotaxi networks — packaging the technology needed for self-driving cars into a platform that other companies can use to compete with Tesla.

Rani Molla7h

Tesla’s Robotaxi program has disclosed its 15th accident, Electrek reports, citing the latest filing from the National Highway Traffic Safety Administration. According to Electrek’s estimation, extrapolated from the last time Tesla disclosed mileage figures, that amounts to a crash every 57,000 miles — about 9x the rate for humans.

The latest crash involved a Model Y hitting a fixed object at 9 mph in January while the autonomous system was engaged.

Humans are very much still involved with Tesla’s so-called autonomous driving service. Despite the service announcing in January that it had started removing safety monitors from the front seats, only two unsupervised vehicles have been spotted in the last month, per Robotaxi Tracker. The entire fleet has also dwindled from around 50 vehicles to just 35. Their mileage is unavailable.

Rani Molla11h

Meta’s reported 20% layoff could bring headcount to its lowest level since 2021

Meta is rising Monday morning after Reuters reported the tech giant is planning to lay off 20% of its employees in an effort to use AI to make its workforce more efficient and offset its surging AI capex costs.

On the company’s last earnings call, CEO Mark Zuckerberg touted 30% efficiency gains for its software engineers and said some “power users” of the company’s AI coding tools saw productivity jump as high as 80% — what some saw as a veiled threat to employees who failed to use AI to boost their output.

Meta’s headcount was nearly 79,000 last quarter, having steadily risen since its layoffs during the self-described “year of efficiency” in 2023. A 20% cut would bring headcount to around 63,000 — the company’s lowest level since 2021.

Shares were recently up 2.7%.

Exclusive: Meta planning sweeping layoffs as AI costs mount

Shares were recently up 2.7%.

Rani Molla

12h

What we know about Tesla’s Terafab project, which Musk says is launching in a few days

Elon Musk says the goal is to produce 100 billion to 200 billion AI chips a year.

Jon Keegan13h

Report: Amid safety failures, ChatGPT’s planned “adult mode” caused concern within OpenAI, with minors misclassified as adults 12% of the time

Despite a series of alarming mental health safety failures that resulted in ChatGPT users allegedly using the product to plan suicides and murder, OpenAI decided to double down on its plan to roll out an “adult mode,” allowing the AI chatbot to produce erotic content.

That decision raised alarms within the company, warning that users could develop unhealthy emotional dependence on the chatbot and that the new age estimation feature was imperfect — and therefore likely to allow minors to access the feature — according to a new report from The Wall Street Journal. Per the report, some 12% of the time, the age estimation feature mistakenly classified minors as adults.

OpenAI’s council of mental health experts were “furious” and unanimous in their opposition to the plans to move forward with the adult mode feature after they were told about the decision in January, with concerns about creating a “sexy suicide coach.”

Earlier this month, the company said it would delay the new feature to focus on other products.

OpenAI’s Bid to Allow X-Rated Talk Is Freaking Out Its Own Advisers