
NVIDIA / GradientSpaces, World from Motion (arXiv:2607.01202) and Tongji University & Tencent, GeoState (arXiv:2610.22343)
Share this post:
GeoState & World from Motion: Open Source 4D Camera AI
GeoState (Tongji University, HKPolyU, and Tencent, arXiv:2610.22343, October 2026) and World from Motion (GradientSpaces/World-from-Motion, NVIDIA researchers, NeurIPS 2026, arXiv:2607.01202) attack the core frustration of AI cinematography: locking physical scene motion independently of the camera lens. GeoState introduces a view decoupled 3D geometry state so autoregressive world models forecast scene evolution without entangling it with camera shake, while World from Motion converts any ordinary monocular video into a freely reshootable dynamic 3D Gaussian Splatting (4DGS) scene that directors can rerender from brand new camera trajectories.
Studio availability. GeoState and World from Motion are not available in AI FILMS Studio. This article covers public open source research releases from late 2026. To direct camera motion and shot angles inside the Studio today, use Kling 3 Motion Control, WAN 3.0, or Seedance 2.5 on the Image to Video page or in the video workspace.

Why 2D Video Models Struggle with True Reshooting
In a standard 2D video generator, camera movement and subject motion are baked into the same flat pixel grid. If you love an actor's performance in a handheld take but wish the camera had orbited 40 degrees left on a crane, you have to regenerate the entire shot from scratch, losing the exact timing, expression, and lighting of the original take.
GeoState and World from Motion separate what happens in 3D space over time from where the camera happens to sit:
- View decoupled geometry states (
GeoState,arXiv:2610.22343). Instead of predicting future 2D RGB frames tied to one camera frustum, GeoState encodes a scene tokenizer built on 3D geometry foundation features. An autoregressive world predictor rolls the 3D scene state forward independently of camera motion, and a ray queried geometry decoder renders any user specified or predicted camera path cleanly without compounding camera drift. - Generative 4D Gaussian reconstruction from one camera (
World from Motion,liyzzz/World-from-Motion). Starting from a single monocular video clip (shot on an iPhone, cinema camera, or generated by an AI video model), World from Motion builds an initial 4D geometric scaffold, conditions a video diffusion model on dense 4D renderings to hallucinate occluded back sides and fix disocclusion holes, and distills those consistent multi view renders back into one unified, real time renderable dynamic 3D Gaussian Splatting (4DGS) asset.
GeoState architecture decoupling scene geometry evolution from camera ray querying. Credit: Tongji University & Tencent (arXiv:2610.22343).
Entangling camera motion with scene dynamics forces 2D world models to waste capacity re-predicting static background parallax every time the lens pans.
GeoState's ray queried decoder lets directors forecast physical world motion once and inspect it from arbitrary virtual camera tracks.
Hardware, Capabilities, and Workflow Comparison
| Framework | Input Modality | Core 3D/4D Representation | Key Filmmaking Output | Hardware Tier |
|---|---|---|---|---|
GeoState (arXiv:2610.22343) |
Multi view / RGB frames + camera poses | View decoupled 3D geometry state tokens | Long horizon 3D world forecasting under arbitrary camera rays | 24 GB – 48 GB VRAM research GPU |
World from Motion (liyzzz/World-from-Motion) |
Single monocular RGB video | Unified Dynamic 3D Gaussian Splatting (4DGS) |
Reshootable 4D scene playable at interactive frame rates | 24 GB – 48 GB VRAM (RTX 4090 / A6000) |
World from Motion bridges real world cinematography and generative AI VFX.
By conditioning a video diffusion model on dense 4D projections before distilling back into dynamic Gaussians, it fills unseen angles behind moving actors cleanly instead of leaving black stretched tears.
World from Motion pipeline lifting monocular video into generative multi view video and unified 4D Gaussians. Credit: GradientSpaces / NVIDIA (arXiv:2607.01202).
Parameter Reference Table (World from Motion)
| Parameter | Recommended Value | Effect on 4D Reconstruction |
|---|---|---|
--video_path |
24 fps or 30 fps monocular clip (3–8s) | Source performance to lift into dynamic 3D Gaussians. |
--novel_orbit_deg |
-45 to +45 degrees |
Angular range for generative disocclusion synthesis around the hero subject. |
--num_gaussians |
250k – 600k |
Higher Gaussian counts sharpen hair, cloth folds, and specular highlights at higher VRAM cost. |
--export_format |
.ply sequence / .splat |
Standard 4D Gaussian format loadable in Blender, Unreal Engine, or web viewers. |
Local Setup with Python (World from Motion)
Clone GradientSpaces/World-from-Motion and pull the official Hugging Face weights from liyzzz/World-from-Motion:
git clone https://github.com/GradientSpaces/World-from-Motion.git
cd World-from-Motion
pip install -r requirements.txt
huggingface-cli download liyzzz/World-from-Motion --local-dir ./checkpoints/world-from-motion
python reconstruct_4d.py \
--video_path ./footage/dancer_handheld_take03.mp4 \
--checkpoint_dir ./checkpoints/world-from-motion \
--camera_trajectory ./trajectories/crane_orbit_35deg.json \
--output_dir ./outputs/dancer_4dgs_reshoot
Filmmaker Production Playbook
- Fix framing mistakes in post without booking a reshoot. Lift a locked handheld or AI generated take through World from Motion into a 4D Gaussian asset, then animate a smooth dolly push or Dutch angle in Blender or Unreal Engine.
- Pair with
SplatAnythingandPi4for metric scale. Extract metric camera poses and dense 4D point tracks first so your novel camera trajectory stays locked to real world floor planes without floating feet. - Avoid extreme 180 degree reveals on occluded faces. Stay within
±45to±60degrees of the original camera frustum when reshooting close up dialogue so generative disocclusion stays faithful to the actor's true likeness.
Related Reading
- SplatAnything & Pi4: Open Source 3D and 4D Scene AI
- WorldCrafter & Hy-Nomas: Open Source 3D World Video AI
- TRELLIS.2 LoRA: Open Source 3D PBR Asset Fine Tuning
- ATAS (Any Time, Any Space): Open Source Multi Shot Video
Sources
- GradientSpaces/World-from-Motion Official GitHub Repository
- liyzzz/World-from-Motion Weights on Hugging Face
- World from Motion: Generative Dynamic Gaussian Reconstruction (arXiv:2607.01202)
- GeoState Official Project Page
- GeoState: Learning View-Decoupled Geometry States for 3D World Modeling (arXiv:2610.22343)
Continue Reading
Video & LipSync
- Video Generator
- Text to Video
- Image to Video
- Start-End Frame to Video
- Draw to Video
- Motion Control
- Video Enhancer
- Video Upscaler
- Video Extend
- Video to Video LipSync
- Audio to Video LipSync
- Image to Video LipSync
- Video FaceSwap
- Seedance 2.5
- WAN 3.0
- Seedance 2
- FLUX 3
- Minimax H3
- Vidu Q3 Pro
- Grok Imagine Video 1.5
- Gemini Omni
- Google Veo 3.1
- Kling 3.0 Pro
- Luma Ray 3.2
- LTX 2.3
- Happy Horse 1.1
- Kling 3.0 Motion
- ByteDance Upscaler
- InfiniteTalk
- InsightFace
Image & Edit
- AI Character
- AI Actor
- Art Generator
- Text to Image
- Image to Image
- LoRA Image to Image
- Draw to Edit
- Image LoRA Training
- Remove Background
- Image Enhancer
- MidJourney 8.0
- Krea 2
- OpenAI GPT Image 2.0
- Nano Banana Pro Ultra
- Nano Banana 2 Lite
- Nano Banana Pro
- Minimax Image
- Clarity Pro
- Z-Image Turbo LoRA
- Krea 2 Turbo LoRA
- Z-Image LoRA Trainer
- Qwen Image LoRA Trainer
- Background Erase Network 2 (BEN2)


