At Reka, our mission is to develop AI that understands, simulates, and acts in the physical world. We build our models to solve the most complex real-world challenges, powering everything from intelligent robots to next-generation wearables and media.

Together with reasoning and visual understanding, video generation is a core capability towards that goal, providing the foundation for systems that simulate possible futures and interact with dynamic environments. For many of the most compelling applications, however, generating high-quality video is not enough. The model must generate it in real time. A real-time model can respond continuously as a user, agent, or environment provides new input, supporting interactive applications where the model not only describes the physical world, but also participates in it.

Today, we are sharing our first step toward that goal: a real-time version of our existing video-generation model. Our team collaborated closely with NVIDIA to achieve this milestone.

Available in closed beta, our system generates an infinite, continuous video stream in an autoregressive fashion. Users can steer the stream as it unfolds by providing instructions in natural language.

We achieved this without sacrificing visual quality: generation runs smoothly at 720p and 24 fps.

Unlocking real-time generation for a model of this scale was a formidable technical challenge. At 30B parameters, our transformer is significantly larger than most video generation models. Running a model of this size fast enough required optimizing both the model itself and the inference engine that serves it. The result? We unlocked real-time performance even on previous-generation hardware like NVIDIA H100 GPUs.

Timestep Distillation with NVIDIA FastGen

The main bottleneck in generating with diffusion models is the iterative denoising process. A diffusion model does not produce a sample in one shot: it starts from noise and refines it over many forward passes, removing a little noise at each step. This is further exacerbated by Classifier-Free Guidance (CFG), which improves prompt adherence and visual quality by evaluating the model on a negative prompt and pushing the output away from it. Put together, our default configuration for the original model required over 100 forward passes to generate each segment. At 30B parameters, that puts real-time generation far out of reach.

A common mitigation strategy is training-free caching. Methods like TeaCache and Cache-DiT exploit the fact that intermediate activations change very little between adjacent denoising steps, and reuse them instead of recomputing. However, the speedups brought by these techniques are modest and not guaranteed, as how much can be safely reused depends on the input and the sampling trajectory.

The principled solution is timestep distillation: adapting the model to do the same job in far fewer generation steps. Rather than approximating the sampling trajectory more cheaply, distillation changes the model itself: the student learns to cover in one step what the original model may have accomplished over several. In addition, guidance is typically folded into the model’s own weights, eliminating the need for additional forward passes from CFG.

To perform the distillation, we adopted NVIDIA FastGen, a flexible framework supporting a wide family of distillation methods, including consistency models, distribution matching, and self-forcing, among others. We considered implementing these methods directly in our training codebase, but opted to integrate it into FastGen instead: this gave us ready-to-use access to a large family of methods inside a robust, scalable framework, making it possible to compare approaches without independently rebuilding each one.

The integration and early experimentation were conducted in close collaboration with NVIDIA engineers. Despite our model’s size, we were able to fit the required training workloads on our NVIDIA H100 cluster and run the distillation process efficiently at scale.

Our final distilled model was trained through a multi-stage process, achieving an 11.8× end-to-end speedup over the original model, with no major degradation in generation quality in our blind human evaluation.

An efficient inference engine

A fast model does not automatically result in a real-time system. In conventional offline generation settings, framework overhead tends to be insignificant compared with the total generation time. In a real-time system, every millisecond matters. Communication between processes, scheduling delays, memory transfers, preprocessing, and post-processing can all interrupt the generation pipeline or prevent the system from sustaining its target frame rate.

To address this, we have developed a proprietary inference engine based on vLLM-Omni, focusing on the following extensions:

• Modular Real-Time Orchestration Layer. We have developed a new layer that coordinates generation as a persistent stream, managing the dependencies between successive segments while continuing to accept new instructions from the user. Clients can interact with the model through a new real-time endpoint. The design is modular, making it easy to integrate any model supported by vLLM-Omni or our extension.

• Low-Latency System Optimizations. We have drastically reduced the framework overhead originating from inter-process communication (IPC), tensor transfers, and post-processing/decoding logic.

• Low-Precision Quantization. Continuing the legacy of Reka Quant, we have applied low-precision quantization across both the core transformer network and the text encoder, managing to fit the entire system into memory-constrained single-node configurations (such as NVIDIA H100 setups) without relying on CPU offloading, while also benefiting from faster low-precision computation on modern accelerators.

The Road Ahead: World Language Action Models (WLAM)

Real-time generation is an important milestone, but it is not the endpoint. Our team is currently training the first version of our upcoming World Language Action Model (WLAM): an omni model designed to unify reasoning, visual understanding, generation and action within a single system. Focused heavily on egocentric and physical-world applications, WLAM will perceive environments, simulate physical outcomes in real time, and assist with real-world planning and action execution.

We designed our inference engine with this direction in mind. Its modular architecture will support real-time generation with WLAM from day one, allowing us to carry the infrastructure and optimization lessons into our next generation of models.

Real-time generation is currently available in closed beta. To learn more about its commercial applications or book a live demonstration, contact us at contact@reka.ai.