Microsoft’s Magma AI Can Manipulate and Control Robots

Microsoft just introduced Magma, a new AI model designed to help robots see, understand and act more intelligently. Unlike traditional artificial intelligence models, Magma processes different types of data all at once – an effort Microsoft is calling a big leap toward “agentic AI,” or systems that can plan and execute tasks on a user’s behalf.

The model, which uses a combination of vision and language processing, is trained on videos, images, robotics data and interface interactions so as to make it more versatile than previous models. 

On its Github page, the Microsoft Research team outlined how Magma can perform tasks, such as how it can manipulate robots and navigate user interfaces like clicking buttons. 

To develop the technology, the company partnered with researchers from the University of Maryland, the University of Wisconsin-Madison and the University of Washington.

The launch comes as tech giants race to develop AI agents that can automate more aspects of daily life. Google has been advancing robotics-focused language models, while OpenAI’s Operator tool is designed to handle mundane tasks like making reservations, ordering groceries and filling out forms via typing, clicking and scrolling within a specialized browser.