EditorNodesPricingBlog

SplatAnything & Pi4: Open Source 3D & 4D Scene AI

November 18, 2026
SplatAnything & Pi4: Open Source 3D & 4D Scene AI

ZJU 3DV SplatAnything (arXiv:2611.08264) and Pi4 Authors (arXiv:2611.23144)

Share this post:

SplatAnything & Pi4: Open Source 3D & 4D Scene AI

SplatAnything (zju3dv/SplatAnything and zju3dv/InfiniSplat, arXiv:2611.08264) and Pi4 (π^4: A Foundation Model for 4D Scene Geometry and Motion, yyfz/Pi4, arXiv:2611.23144) are two November 2026 open source 3D and 4D reconstruction foundation models designed for virtual production and matchmove pipelines. By replacing slow per scene optimization with Gaussian Spacing Normalization (SplatAnything) and a single pass 4D geometry plus motion transformer (Pi4), both models turn uncalibrated multi view photos or dynamic monocular videos into navigable 3D Gaussian Splatting environments and synchronized 4D motion tracks in seconds.

Studio availability. SplatAnything and Pi4 are not available in AI FILMS Studio. This article covers public research releases from November 2026. To generate video scenes or direct camera motion in the Studio, see the models listed on the Motion Control and Image to Video pages.

Pi4 single pass 4D geometry and motion inference on fast figure skating footage

Dynamic outdoor cycling sequence reconstructed into synchronized 4D world coordinates

Why Standard 3D Gaussian Splatting Breaks on Arbitrary Film Sets

Traditional 3D Gaussian Splatting (3DGS) requires a 20 to 40 minute per scene optimization loop after running Structure from Motion (COLMAP). Feed forward 3DGS networks skip per scene training, yet existing models assume uniform camera distances and fail when a filmmaker mixes close up tabletop shots with wide drone aerials or long corridor views.

Moving actors introduce a second bottleneck. Static 3DGS treats moving people or vehicles as transient outliers, while 4D dynamic reconstruction pipelines typically chain separate depth, optical flow, and camera tracking networks that drift out of alignment across long takes.

SplatAnything (arXiv:2611.08264) and Pi4 (arXiv:2611.23144, released November 2026 with yyfz233/Pi4 weights) solve both arbitrary scale static sets and dynamic 4D action in a single feed forward pass.

  1. Gaussian Spacing Normalization (SplatAnything / InfiniSplat v3). Developed by Zhejiang University's 3DV Lab, SplatAnything normalizes the spatial density of 3D Gaussians relative to local depth and ray spacing before decoding splat attributes. Whether the input views capture a tiny prop macro shot or an expansive aerial city block, the feed forward network outputs uniform, hole free 3D Gaussian scenes without manual scale tuning.
  2. Single pass 4D scene geometry and motion (Pi4 / π^4). Trained on the large scale GameRealm synthetic 4D dataset alongside real world video corpora, Pi4 ingests a sequence of video frames and predicts dense 3D point maps, camera intrinsics and extrinsics, and 3D scene flow trajectories in one network forward pass.
  3. Flexible geometry only or full 4D query modes. In Pi4, omitting --query-frame runs fast 3D geometry and camera pose recovery across every sampled frame, while specifying a query frame adds dense 4D point motion tracking for VFX matchmoving and dynamic rotoscoping.
SplatAnything Gaussian Spacing Normalization feed forward 3DGS pipeline across arbitrary scales

SplatAnything feed forward 3D Gaussian Splatting architecture with Gaussian Spacing Normalization. Credit: ZJU 3DV Lab (arXiv:2611.08264).

When camera baselines vary widely between an interior room and an exterior aerial shot, unnormalized pixel aligned Gaussians either clump into blurry blobs or separate into sparse dots.

Gaussian Spacing Normalization rescales splat covariance automatically so feed forward 3DGS renders remain sharp across arbitrary focal lengths and scene depths.

Try Pi4 and SplatAnything Live in the Browser

Both Pi4 (yyfz233-pi4.hf.space) and SplatAnything (ritianyu-splatanything.hf.space) provide active, no login Gradio Spaces embedded below. Upload a video clip or multi view image set to inspect 3D/4D point clouds and Gaussian splats directly in your browser.

Inside `Pi4`, a shared spatiotemporal transformer backbone processes all video frames jointly rather than stitching pairwise frame matches.

This single pass formulation eliminates accumulated drift on fast sports footage (`figure_skating.mp4`, `kart.mp4`, `cycling.mp4`) and outputs synchronized global 4D geometry alongside per pixel motion vectors.

Pi4 foundation model architecture predicting 4D geometry and 3D motion in a single forward pass

Pi4 unified transformer predicting 3D scene geometry, camera poses, and 4D motion in one pass. Credit: Pi4 Authors (arXiv:2611.23144).

Checkpoints, Capabilities & Hardware Matrix

Model / Repository Primary Input Output Representation Hardware & VRAM License / Status
Pi4 (π^4) (yyfz233/Pi4, arXiv:2611.23144) Monocular dynamic video (examples/*.mp4) 4D point maps, 6-DoF camera poses, 3D motion trajectories 16 GB to 24 GB VRAM (25-frame window) up to 80 GB A800 (long sequences) Open Weights (model.safetensors)
SplatAnything / InfiniSplat v3 (zju3dv/InfiniSplat, arXiv:2611.08264) Arbitrary scale multi view RGB images or video frames Feed forward 3D Gaussian Splatting (3DGS) .ply 12 GB to 24 GB VRAM Open Source Research
IDOL & OOOSplat Single image or sparse multi view Feed forward 3D human / object Gaussian splats 12 GB to 24 GB VRAM Open Weights

Running Pi4 and SplatAnything (InfiniSplat v3) Locally

To run Pi4 (yyfz/Pi4) locally on a dynamic video clip, clone the repository and run example.py. The script downloads yyfz233/Pi4 (model.safetensors) from Hugging Face automatically on first run.

git clone https://github.com/yyfz/Pi4.git
cd Pi4
pip install -r requirements.txt

# Optional: pre-download weights directly from Hugging Face
huggingface-cli download yyfz233/Pi4 model.safetensors --local-dir ./checkpoints

# Full 4D geometry + motion inference with stride sampling and query frame
python example.py examples/figure_skating.mp4 \
  --stride 2 \
  --query-frame 0 \
  --output-dir outputs/skating_4d

# Fast geometry-only camera and 3D scene pass (omit --query-frame)
python example.py examples/cycling.mp4 \
  --stride 2 \
  --output-dir outputs/cycling_3d_geom

For SplatAnything, note that ZJU 3DV consolidated active maintenance into zju3dv/InfiniSplat (InfiniSplat_v3 subfolder). Clone InfiniSplat to run feed forward Gaussian Spacing Normalization across indoor, aerial, or street sequences.

git clone https://github.com/zju3dv/InfiniSplat.git
cd InfiniSplat/InfiniSplat_v3
pip install -r requirements.txt

# Reconstruct a 3D Gaussian Splat scene from uncalibrated multi-view frames
python infer.py \
  --input_dir ../InfiniSplat_v1/examples/data/rgb_demo \
  --output_ply outputs/courtyard_splat.ply

Production Playbook for Filmmakers

  1. Extract 6-DoF camera tracks and 4D motion from AI video. When an AI generated clip from WAN 3.0, MiniMax H3, or Kling 3.0 needs 3D VFX titles, particle effects, or composited props locked into the room, run Pi4 in geometry mode to recover per frame camera extrinsics and dense 3D point maps in one pass.
  2. Scale memory on 16 GB to 24 GB GPUs with --stride. On consumer GPUs, increase --stride or cap the frame window to 25 frames in Pi4 to keep peak VRAM within 16 GB to 24 GB while retaining full trajectory coverage.
  3. Compare across 3D and 4D filmmaking tools. Read our companion guides on IDOL and OOOSplat 3D Gaussian Splatting for film, Light-X 4D video camera and relighting control, Diffusion as Shader 3D aware video control, and WorldCrafter and Hy-Nomas 3D world video models.

To animate concept frames or control camera trajectories in the cloud without local 3D setup, visit the Motion Control and Image to Video pages, or link 3D plates and video generation nodes visually in the Nodes Graph Editor.


Sources