EditorNodesPricingBlog

NVIDIA GEN3C: Open Source 3D Camera Control Video AI

November 17, 2026
Updated: November 18, 2026
NVIDIA GEN3C: Open Source 3D Camera Control Video AI

NVIDIA Toronto AI Lab, GEN3C (arXiv:2503.03751, Apache 2.0)

Share this post:

NVIDIA GEN3C: Open Source 3D Camera Control Video AI

GEN3C (3D-Informed World-Consistent Video Generation with Precise Camera Control, arXiv:2503.03751) from NVIDIA Toronto AI Lab is an open source, 3D cache guided video diffusion model recognized as a CVPR 2025 Highlight and released under the Apache 2.0 license. Available in both a 7B Cosmos Diffusion Transformer checkpoint (nvidia/GEN3C-Cosmos-7B) and a lighter Stable Video Diffusion (SVD) variant, GEN3C maintains an explicit spatiotemporal 3D point cloud cache so virtual camera moves follow exact 6-DoF trajectories without geometry drift or object popping.

Studio availability. GEN3C is not available in AI FILMS Studio. This article covers a public research release. To control camera movement in the Studio, see the models listed on the Motion Control page or open the video workspace.

GEN3C-Cosmos-7B rendering 3D cache point cloud projections into world consistent camera arcs

Single view image unprojected into a 3D cache and rendered along custom 6-DoF dolly and orbit paths

Solving Object Popping in Camera Conditioned Video Diffusion

Standard camera conditioned video generators pass 6-DoF camera extrinsics or Plücker ray embeddings directly into attention layers and rely on the neural network to imagine how 3D geometry rotates across frames. When a camera pans away from an actor or building and returns seconds later, models without explicit 3D memory hallucinate new architecture or alter lighting.

Explicit 3D reconstruction methods try to fit a neural radiance field or Gaussian splat before rendering, but monocular depth estimation errors leave torn edges and missing regions wherever the camera reveals previously occluded areas. GEN3C bridges both paradigms by pairing an explicit spatiotemporal 3D point cloud cache with a generative video diffusion backbone that inpaints disoccluded regions and repairs depth inaccuracies automatically.

GEN3C 3D point cloud unprojection and novel viewpoint rendering motivation

GEN3C unprojects monocular depth into a 3D cache and renders novel viewpoints for the video generator. Credit: NVIDIA Toronto AI Lab (Apache 2.0).

Given a single input image or a dynamic monocular video, GEN3C estimates metric pixel depth and unprojects the colored pixels into 3D world space. When the user specifies a new camera path, the 3D cache is rasterized along those exact camera intrinsics and extrinsics.

Because the 3D cache handles perspective projection mathematically, the video diffusion model only needs to synthesize missing disoccluded pixels and refine surface shading.

Spatiotemporal 3D Cache & Multi View Feature Aggregation

To handle both static environments (from one or more photographs) and dynamic moving scenes (from monocular or multi camera video rigs), GEN3C organizes its cache across both space and time.

For a dynamic video of length `T` captured from `N` viewpoints, the spatiotemporal cache maintains an array of `N x T` point clouds. Rendering a target camera trajectory at timestamp `t` projects the `N` point clouds from frame `t` into 2D RGB images and binary disocclusion masks.

Those rendered RGB plates and visibility masks are encoded into latent space and concatenated with the noisy diffusion latents inside `GEN3C-Cosmos-7B`, allowing the transformer to resolve multi view conflicts and keep lighting consistent across 121 frame sequences.

GEN3C spatiotemporal 3D cache creation, 2D projection, and video diffusion pipeline

GEN3C spatiotemporal 3D cache pipeline and multi view latent conditioning. Credit: Xuanchi Ren et al., NVIDIA (Apache 2.0).

Hardware, VRAM & Model Variant Matrix

Checkpoint / Component Base Architecture Resolution & Frame Length Hardware & VRAM License
GEN3C-Cosmos-7B NVIDIA Cosmos Diffusion-7B DiT 704x1280 (720p), 121 frames 28 GB to 40 GB VRAM (or FP8 / offloading on 24 GB) Apache 2.0 (Code) / NVIDIA Open Model License
GEN3C-SVD Stable Video Diffusion UNet (img2vid-xt) 576x1024, 25 frames 16 GB to 24 GB VRAM (consumer RTX 4090) Apache 2.0
Depth Estimator (MoGe / Video-Depth-Anything) Monocular metric depth unprojection Matches input image or video plate Runs alongside main GPU pipeline Apache 2.0 / MIT

Running GEN3C and the Interactive Trajectory GUI Locally

Clone nv-tlabs/GEN3C, create the Conda environment, and download nvidia/GEN3C-Cosmos-7B from Hugging Face. GEN3C includes an interactive 3D browser GUI (proteus) where directors can load an image, orbit or dolly the virtual camera in real time against the point cloud preview, and export the exact trajectory matrix for diffusion rendering.

git clone https://github.com/nv-tlabs/GEN3C.git
cd GEN3C

conda env create --file environment.yml
conda activate gen3c
pip install -r requirements.txt

huggingface-cli download nvidia/GEN3C-Cosmos-7B --local-dir ./checkpoints/GEN3C-Cosmos-7B

Generate a 121 frame camera controlled shot from a single reference image using preset trajectories (zoom_in, orbit_left, spiral), or launch the interactive trajectory editor.

# Single image to 3D camera controlled video with GEN3C-Cosmos-7B
python cosmos_predict1/diffusion/inference/gen3c_single_image.py \
  --checkpoint_dir ./checkpoints \
  --input_image_path assets/demo/street_plate.png \
  --trajectory orbit_left \
  --video_save_folder ./outputs

# Dynamic monocular video rerendering along a novel camera path
python cosmos_predict1/diffusion/inference/gen3c_dynamic.py \
  --checkpoint_dir ./checkpoints \
  --input_video_path assets/demo/actor_walk.mp4 \
  --trajectory zoom_in \
  --video_save_folder ./outputs

Virtual Production Use Cases & AI FILMS Studio Workflows

In previsualization and virtual production, GEN3C turns a single concept illustration or location scout photo into a multi angle shot list. Because every camera move reads from the same 3D point cloud cache, a wide establishing shot, a medium dolly push, and a reverse angle over the shoulder maintain identical spatial geometry.

For related NVIDIA and open source world and camera guides, read our coverage of NVIDIA Cosmos 3 open world video generation, NVIDIA Lyra 2.0 3D world generation, SANA-WM minute scale world modeling, FaceCam portrait camera control, and SplatAnything and Pi4 3D and 4D scene reconstruction.

To direct camera movement and character motion in the cloud without local CUDA setup, open the Motion Control page or connect image plates and video generators inside the Nodes Graph Editor.


Sources