
ZJU 3DV SplatAnything (arXiv:2611.08264) and Pi4 Authors (arXiv:2611.23144)
Share this post:
SplatAnything & Pi4: Open Source 3D & 4D Scene AI
SplatAnything (zju3dv/SplatAnything and zju3dv/InfiniSplat, arXiv:2611.08264) and Pi4 (π^4: A Foundation Model for 4D Scene Geometry and Motion, yyfz/Pi4, arXiv:2611.23144) are two November 2026 open source 3D and 4D reconstruction foundation models designed for virtual production and matchmove pipelines. By replacing slow per scene optimization with Gaussian Spacing Normalization (SplatAnything) and a single pass 4D geometry plus motion transformer (Pi4), both models turn uncalibrated multi view photos or dynamic monocular videos into navigable 3D Gaussian Splatting environments and synchronized 4D motion tracks in seconds.
Studio availability. SplatAnything and Pi4 are not available in AI FILMS Studio. This article covers public research releases from November 2026. To generate video scenes or direct camera motion in the Studio, see the models listed on the Motion Control and Image to Video pages.
Pi4 single pass 4D geometry and motion inference on fast figure skating footage
Dynamic outdoor cycling sequence reconstructed into synchronized 4D world coordinates
Why Standard 3D Gaussian Splatting Breaks on Arbitrary Film Sets
Traditional 3D Gaussian Splatting (3DGS) requires a 20 to 40 minute per scene optimization loop after running Structure from Motion (COLMAP). Feed forward 3DGS networks skip per scene training, yet existing models assume uniform camera distances and fail when a filmmaker mixes close up tabletop shots with wide drone aerials or long corridor views.
Moving actors introduce a second bottleneck. Static 3DGS treats moving people or vehicles as transient outliers, while 4D dynamic reconstruction pipelines typically chain separate depth, optical flow, and camera tracking networks that drift out of alignment across long takes.
SplatAnything (arXiv:2611.08264) and Pi4 (arXiv:2611.23144, released November 2026 with yyfz233/Pi4 weights) solve both arbitrary scale static sets and dynamic 4D action in a single feed forward pass.
- Gaussian Spacing Normalization (
SplatAnything/InfiniSplat v3). Developed by Zhejiang University's 3DV Lab, SplatAnything normalizes the spatial density of 3D Gaussians relative to local depth and ray spacing before decoding splat attributes. Whether the input views capture a tiny prop macro shot or an expansive aerial city block, the feed forward network outputs uniform, hole free 3D Gaussian scenes without manual scale tuning. - Single pass 4D scene geometry and motion (
Pi4/π^4). Trained on the large scaleGameRealmsynthetic 4D dataset alongside real world video corpora,Pi4ingests a sequence of video frames and predicts dense 3D point maps, camera intrinsics and extrinsics, and 3D scene flow trajectories in one network forward pass. - Flexible geometry only or full 4D query modes. In
Pi4, omitting--query-frameruns fast 3D geometry and camera pose recovery across every sampled frame, while specifying a query frame adds dense 4D point motion tracking for VFX matchmoving and dynamic rotoscoping.
SplatAnything feed forward 3D Gaussian Splatting architecture with Gaussian Spacing Normalization. Credit: ZJU 3DV Lab (arXiv:2611.08264).
When camera baselines vary widely between an interior room and an exterior aerial shot, unnormalized pixel aligned Gaussians either clump into blurry blobs or separate into sparse dots.
Gaussian Spacing Normalization rescales splat covariance automatically so feed forward 3DGS renders remain sharp across arbitrary focal lengths and scene depths.
Try Pi4 and SplatAnything Live in the Browser
Both Pi4 (yyfz233-pi4.hf.space) and SplatAnything (ritianyu-splatanything.hf.space) provide active, no login Gradio Spaces embedded below. Upload a video clip or multi view image set to inspect 3D/4D point clouds and Gaussian splats directly in your browser.
Inside `Pi4`, a shared spatiotemporal transformer backbone processes all video frames jointly rather than stitching pairwise frame matches.
This single pass formulation eliminates accumulated drift on fast sports footage (`figure_skating.mp4`, `kart.mp4`, `cycling.mp4`) and outputs synchronized global 4D geometry alongside per pixel motion vectors.
Pi4 unified transformer predicting 3D scene geometry, camera poses, and 4D motion in one pass. Credit: Pi4 Authors (arXiv:2611.23144).
Checkpoints, Capabilities & Hardware Matrix
| Model / Repository | Primary Input | Output Representation | Hardware & VRAM | License / Status |
|---|---|---|---|---|
Pi4 (π^4) (yyfz233/Pi4, arXiv:2611.23144) |
Monocular dynamic video (examples/*.mp4) |
4D point maps, 6-DoF camera poses, 3D motion trajectories | 16 GB to 24 GB VRAM (25-frame window) up to 80 GB A800 (long sequences) | Open Weights (model.safetensors) |
SplatAnything / InfiniSplat v3 (zju3dv/InfiniSplat, arXiv:2611.08264) |
Arbitrary scale multi view RGB images or video frames | Feed forward 3D Gaussian Splatting (3DGS) .ply |
12 GB to 24 GB VRAM | Open Source Research |
IDOL & OOOSplat |
Single image or sparse multi view | Feed forward 3D human / object Gaussian splats | 12 GB to 24 GB VRAM | Open Weights |
Running Pi4 and SplatAnything (InfiniSplat v3) Locally
To run Pi4 (yyfz/Pi4) locally on a dynamic video clip, clone the repository and run example.py. The script downloads yyfz233/Pi4 (model.safetensors) from Hugging Face automatically on first run.
git clone https://github.com/yyfz/Pi4.git
cd Pi4
pip install -r requirements.txt
# Optional: pre-download weights directly from Hugging Face
huggingface-cli download yyfz233/Pi4 model.safetensors --local-dir ./checkpoints
# Full 4D geometry + motion inference with stride sampling and query frame
python example.py examples/figure_skating.mp4 \
--stride 2 \
--query-frame 0 \
--output-dir outputs/skating_4d
# Fast geometry-only camera and 3D scene pass (omit --query-frame)
python example.py examples/cycling.mp4 \
--stride 2 \
--output-dir outputs/cycling_3d_geom
For SplatAnything, note that ZJU 3DV consolidated active maintenance into zju3dv/InfiniSplat (InfiniSplat_v3 subfolder). Clone InfiniSplat to run feed forward Gaussian Spacing Normalization across indoor, aerial, or street sequences.
git clone https://github.com/zju3dv/InfiniSplat.git
cd InfiniSplat/InfiniSplat_v3
pip install -r requirements.txt
# Reconstruct a 3D Gaussian Splat scene from uncalibrated multi-view frames
python infer.py \
--input_dir ../InfiniSplat_v1/examples/data/rgb_demo \
--output_ply outputs/courtyard_splat.ply
Production Playbook for Filmmakers
- Extract 6-DoF camera tracks and 4D motion from AI video. When an AI generated clip from
WAN 3.0,MiniMax H3, orKling 3.0needs 3D VFX titles, particle effects, or composited props locked into the room, runPi4in geometry mode to recover per frame camera extrinsics and dense 3D point maps in one pass. - Scale memory on 16 GB to 24 GB GPUs with
--stride. On consumer GPUs, increase--strideor cap the frame window to 25 frames inPi4to keep peak VRAM within 16 GB to 24 GB while retaining full trajectory coverage. - Compare across 3D and 4D filmmaking tools. Read our companion guides on IDOL and OOOSplat 3D Gaussian Splatting for film, Light-X 4D video camera and relighting control, Diffusion as Shader 3D aware video control, and WorldCrafter and Hy-Nomas 3D world video models.
To animate concept frames or control camera trajectories in the cloud without local 3D setup, visit the Motion Control and Image to Video pages, or link 3D plates and video generation nodes visually in the Nodes Graph Editor.
Sources
- yyfz/Pi4 Official GitHub Repository
- Pi4 Official Project Page
- Pi4: A Foundation Model for 4D Scene Geometry and Motion (arXiv:2611.23144)
- yyfz233/Pi4 Model Weights & Interactive Space on Hugging Face
- SplatAnything: Gaussian Spacing Normalization for Feed-Forward 3DGS (arXiv:2611.08264)
- zju3dv/InfiniSplat (InfiniSplat v3 / SplatAnything) Official GitHub Repository
- ritianyu/SplatAnything Interactive Space on Hugging Face
Continue Reading
Video & LipSync
- Video Generator
- Text to Video
- Image to Video
- Start-End Frame to Video
- Draw to Video
- Motion Control
- Video Enhancer
- Video Upscaler
- Video Extend
- Video to Video LipSync
- Audio to Video LipSync
- Image to Video LipSync
- Video FaceSwap
- Seedance 2.5
- WAN 3.0
- Seedance 2
- FLUX 3
- Minimax H3
- Vidu Q3 Pro
- Grok Imagine Video 1.5
- Gemini Omni
- Google Veo 3.1
- Kling 3.0 Pro
- Luma Ray 3.2
- LTX 2.3
- Happy Horse 1.1
- Kling 3.0 Motion
- ByteDance Upscaler
- InfiniteTalk
- InsightFace
Image & Edit
- AI Character
- AI Actor
- Art Generator
- Text to Image
- Image to Image
- LoRA Image to Image
- Draw to Edit
- Image LoRA Training
- Remove Background
- Image Enhancer
- MidJourney 8.0
- Krea 2
- OpenAI GPT Image 2.0
- Nano Banana Pro Ultra
- Nano Banana 2 Lite
- Nano Banana Pro
- Minimax Image
- Clarity Pro
- Z-Image Turbo LoRA
- Krea 2 Turbo LoRA
- Z-Image LoRA Trainer
- Qwen Image LoRA Trainer
- Background Erase Network 2 (BEN2)


