NVIDIA Alpamayo

NVIDIA Cosmos is a developer-first platform of state-of-the-art generative World Foundation Models (WFMs), advanced video tokenizers, guardrails, and an accelerated data processing and curation pipeline designed to supercharge physical AI development. It enables developers working on autonomous vehicles, robotics, and video analytics AI agents to generate photorealistic, physics-aware synthetic video data, trained on an immense dataset including 20 million hours of real-world and simulated video, to rapidly simulate future scenarios, train world models, and fine‑tune custom behaviors. It includes three core WFM types; Cosmos Predict, capable of generating up to 30 seconds of continuous video from multimodal inputs; Cosmos Transfer, which adapts simulations across environments and lighting for versatile domain augmentation; and Cosmos Reason, a vision-language model that applies structured reasoning to interpret spatial-temporal data for planning and decision-making.

Various sensors, such as LiDAR, cameras and radar collect environmental data surrounding the vehicle. Using sensor fusion technology perception algorithms can determine in real time the type, location, velocity and orientation of objects on the road. This autonomous perception system is backed by both Baidu’s big data and deep learning technologies, as well as a vast collection of real world labeled driving data. The large-scale deep-learning platform and GPU clusters. Simulation provides the ability to virtually drive millions of kilometers daily using an array of real world traffic and autonomous driving data. Through the simulation service, partners gain access to a large number of autonomous driving scenes to quickly test, validate, and optimize models with comprehensive coverage in a way that is safe and efficient.