Runway wants to turn AI video generation into a live stream you control in real time
decoder ·
Runway is researching real-time video generation in which users describe a scene and steer a streamed video instead of waiting for a finished clip. Its approach builds each new segment from all previously generated frames and aims to reduce the time to the first frame, revision delays, and GPU costs. Runway says a key challenge is error accumulation, so it trains models on their own imperfect outputs to help them correct deviations. The company sees applications in education, gaming, robotics, and autonomous-vehicle testing, where generated environments need to respond immediately. This work connects to Runway’s GWM-1, Runway Characters, Solaris, GWM Robotics, and a collaboration with Nvidia targeting a first frame in under 100 milliseconds on Vera Rubin hardware. Runway has not announced when its real-time system will become available, while related efforts include Decart’s MirageLSD, Google DeepMind’s Genie 3, and Waymo’s road-focused world model.
decoder