Amazon rolls out updated Trainium chip, new AI models at re:Invent conference

Today in Las Vegas, at its annual re:Invent developer conference, Amazon AWS made a bunch of product announcements that reveal its plan to continue capitalizing on the AI boom. 

AWS currently enjoys a substantial lead in the cloud computing market, serving up AI infrastructure in the cloud to customers large and small using the models of their choice.

Trainium3

Probably the most consequential announcement was about the company’s new Trainium3 custom AI chip. Three years out from ChatGPT’s disruptive debut, AI companies are diversifying their computing resources away from GPU juggernaut Nvidia and starting to sample the price and performance benefits of alternative custom chips, such as Trainium.

Today, AWS announced the Trainium3 UltraServer, powered by the new Trainium3 chips. The company said the Trainium3 chips are 4x as fast and can train models for half the cost of the previous generation. AWS CEO Matt Garman acknowledged the misleading name of the chip, which excels not only at training AI models but also running AI models, known as inference.

“People often give us a little bit of a hard time about product naming in AWS. No, no, it’s true. Well, it turns out Trainium is no exception. We named it Trainium because it’s designed to be an awesome chip for Al training, and it is, but as it turns out, Trainium2 is actually the best system in the world currently for inference,” Garman said.

The company said work on Trainium4 is well underway, and the chips would be compatible with Nvidia systems.

Nova 2 models

Amazon announced updates to its own frontier AI model family called Nova:

• Nova Lite (fast and cheap for everyday tasks),

• Nova Pro (for complex reasoning workloads),

• Nova Sonic (a new speech-to-speech model), and

• Nova Omni (an “all-in-one model for multimodal reasoning and image generation”).

The new models may help the company offer cheaper, more efficient AI computing for its customers versus running competing frontier models on popular Nvidia GPUs.

Nova Forge

Complementing the new Nova 2 models is a product called “Nova Forge,” which makes it easier to train customized models using the Nova models as a base. Customers can bring custom data into the training process to easily create specialized expert models using the tool.

AI Factories

Amazon is also rolling out a way for customers to run AWS services inside their own data centers, keeping their data under their control. Amazon AI Factories works with both Amazon Trainium chips as well as Nvidia GPUs. Amazon said an AI factory is like deploying your own private AWS region.

Agents, agents, agents

AI agents were mentioned quite a bit during the keynote, and the company had lots of new offerings to help customers deploy autonomous AI helpers.

Nova Act: A new AWS service that lets customers spin up “fleets of reliable AI agents for automating production UI workflows.”

DevOps Agent: A new “frontier agent” that autonomously helps solve software failures on production systems when human engineers aren’t online. Amazon describes the new agent as “our always-on, autonomous on-call engineer.”