NVIDIA UniRelight: AI Video Relighting for Filmmakers

NVIDIA's Toronto AI Lab published UniRelight, a system that changes lighting conditions in existing videos while preserving scene geometry, material properties, and temporal consistency. The work was accepted at NeurIPS 2025 and builds on NVIDIA's Cosmos World Foundation Models. The research is led by Kai He and Ruofan Liang at NVIDIA, the University of Toronto, and the Vector Institute.

Why Relighting Existing Footage Is Difficult

Standard relighting pipelines treat albedo estimation and lighting synthesis as separate steps. The disconnect produces artifacts where shadows, specular highlights, and material responses in one frame disagree with the next, breaking temporal coherence across the clip.

Complex surfaces compound the problem. Glass, transparent objects, and anisotropic materials produce lighting responses that conventional decomposition methods cannot model accurately, because the physical assumptions built into most approaches break down at those surface types.

Joint Decomposition and Synthesis

UniRelight addresses the separation problem by predicting the relit video output and an albedo map simultaneously in a single forward pass. Each prediction constrains the other, giving the model a structural prior that single task approaches lack.

The architecture uses a finetuned DiT video diffusion model with a VAE encoder and decoder for the latent space. Target lighting is supplied as environment maps (HDRI), whose features are concatenated with video latents at inference time. Training combines synthetic multi illumination datasets with automatically labeled real world video footage, covering the range of materials and conditions the model encounters in practice.

What the Model Produces

UniRelight handles specular highlights on metallic and polished surfaces, glass and transparent objects, and anisotropic materials that respond differently based on viewing and lighting angles. All outputs remain consistent from frame to frame throughout the video sequence.

Dramatic lighting changes are within scope: switching from daylight to nighttime conditions, hard directional sources, and colored environments. The authors report that UniRelight outperforms DiffusionRenderer on standard benchmarks for intrinsic decomposition and relighting quality. For commercial video generation workflows without research restrictions, AI FILMS Studio provides a licensed suite of tools.

Gallery

Availability and License

UniRelight's code is on GitHub at nv-tlabs/UniRelight and model weights are published on Hugging Face. The project is released under the NVIDIA OneWay Noncommercial License. Commercial deployment is not permitted. Use is restricted to research and academic purposes.

This release follows the same noncommercial model as other NVIDIA NVLabs projects. NVIDIA LongLive, the lab's real time interactive video generation system, and NVIDIA SANA Video, an efficient long form text-to-video model, both ship under the same terms. NVIDIA Lyra 2.0, from the Spatial Intelligence Lab, takes the opposite approach with an Apache 2.0 license that permits commercial deployment.

Sources

arXiv: UniRelight: Learning Joint Decomposition and Synthesis for Video Relighting

GitHub: nv-tlabs/UniRelight

Project Page: research.nvidia.com/labs/toronto-ai/UniRelight