Amazon’s AI plans: custom chips, an Anthropic “ultracluster,” and its own foundation model
While Amazon’s new models appear to be competitive in terms of features and performance, that isn’t the main thing that the company is touting — it’s the cost.
This week at Amazon’s AWS re:Invent conference in Las Vegas, the company fleshed out its plans to both serve and compete with the larger AI industry.
AWS is largely AI agnostic. Customers can use pretty much any of the major AI models on the cloud-computing platform, running on servers that use chips from Nvidia, AMD, Qualcomm, and others.
But Amazon has also been building and selling computing powered by its purpose-built AI chips, including its latest Trainium2 chip, which Amazon is now making widely available on AWS’s EC2 service. Amazon says these new Trainium2 instances are built for training and deploying jumbo-sized large language models with better price performance than its current offerings.
Amazon also deepened its partnership with AI startup Anthropic, announcing that it’s building an “ultracluster” of “hundreds of thousands” of Trainium2 servers to train Anthropic’s next-generation LLM. Amazon recently doubled its investment in Anthropic, bringing the total to $8 billion.
Probably the most significant announcement was Amazon’s late entry to the foundational AI-model club. Named “Amazon Nova,” the new LLM comes in four flavors: a text-only Micro and three multimodal models, Lite, Pro, and Premier. Amazon touted benchmark scores for the Nova models, which place it in the same class as OpenAI’s GPT-4o and Meta’s Llama 3. Amazon’s multimodal Nova models can ingest and generate images and videos, like many of the other top models out there today.
While Amazon’s new models appear to be competitive in terms of features and performance, that isn’t the main thing that the company is touting — it’s the models’ low, low cost.
Running Amazon models on Amazon servers, powered by Amazon chips, yields significant cost savings and low latency. Amazon says its Nova models are “at least 75% less expensive” than the best-performing models available on AWS today.