Microsoft’s new AI chip makes it less reliant on Nvidia’s GPUs

Microsoft is announcing its third-generation Maia in-house AI chip today, designed to be the “primary AI inference system” for Microsoft’s products and services. Maia 300 not only improves the cost and efficiency of running AI workloads in the cloud, it’s also more capable of handling the latest reasoning models and agentic tasks.

Maia 300 will start being rolled out to production in 2027, and it has been designed with efficiency in mind. “We expect Maia 300 to deliver 40 percent more tokens per megawatt compared to leading GPU-based systems,” says Scott Guthrie, executive vice president of Microsoft’s cloud group. “And in a full 96-accelerator configuration, it delivers up to 7x the rack-scale token throughput of Maia 200.”

These efficiency improvements will allow Microsoft to rely mainly on its in-house AI chips soon, instead of ones from Nvidia and AMD. “We will scale our Azure Maia family to multi-gigawatt deployments starting in 2027, making Maia the primary AI inference system for Microsoft products and services,” says Guthrie.

Maia 300 is a successor to the 200, which was only announced in January. Maia 200 is already running on production AI workloads across Microsoft, including Microsoft Copilot, GitHub Copilot, Microsoft Foundry, and Microsoft’s own AI models.

“Maia 200 is the best performing AI infrastructure in our fleet, measured in real world metrics for AI services,” says Guthrie. “Maia 300 is a complete AI system designed for the new types of AI workloads emerging today: Long-context reasoning, latency-sensitive conversational experiences, and sustained, multi-step agentic tasks.”

Microsoft is using a new chiplet architecture to get its all-important efficiency gains. Built on TSMC’s N2P process node, the latest Maia 300 architecture separates compute, memory controller, and I/O functions. “This increases the silicon area dedicated to compute performance and allows each function to use the technology best suited to its role,” says the Azure hardware and infrastructure team.

Maia 300 is also a key part of Microsoft’s own AI model work, and could help the company achieve its goal of catching up with Google DeepMind, OpenAI, and Anthropic. “We have to work across the full stack, all the way down to the silicon,” says Microsoft AI CEO Mustafa Suleyman. “At that scale we need a platform that we own, and that we co-design with our models. That’s why running our Microsoft AI models on top of the Maia chip is a critical focus for our team.”