EditorNodesPricingBlog

pi-H3: Open Source 2-Step Audio Video Generation

November 20, 2026
pi-H3: Open Source 2-Step Audio Video Generation

LakonLab (Hansheng Chen), pi-Flow distillation and pi-H3 open source video generation

Share this post:

pi-H3: Open Source 2-Step Audio Video Generation

pi-H3 (Lakonik/pi-H3, November 2026) is an open source distillation and inference release from LakonLab (Hansheng Chen and collaborators from Stanford and Luma AI) that compresses the MiniMax H3 audio visual foundation model down to 2 to 4 sampling steps while removing the video VAE decoder entirely. Using pi-Flow V2 policy distillation and Decoding Without a Decoder (DWD), pi-H3 renders a 5 second HD video clip with synchronized stereo audio in roughly 3.8 seconds on a single NVIDIA B200 GPU.

Studio availability. Try the public Hugging Face demo below; pi-H3 is not part of a paid AI FILMS Studio plan (use MiniMax H3 or WAN 3.0 in the video workspace).

pi-Flow V2 and pi-H3 few step video generation visual quality overview

Why Video VAE Decoders Became the Hidden Bottleneck

After step distillation cut video Diffusion Transformers from 50 steps down to 4 steps, engineers ran into an unexpected wall. In modern 3D latent video pipelines, the final 3D VAE decoding pass often consumes 35% to 55% of total wall clock latency and spikes peak VRAM right at the end of generation.

pi-H3 attacks both denoising compute and decoding latency at the same time:

  1. pi-Flow V2 policy distillation (2 to 4 NFE). Rather than relying on brittle adversarial GAN losses that wash out fine grain or flatten stereo soundscapes, LakonLab trains a policy based flow matching student that maps Gaussian noise to clean audio visual trajectories in as few as 2 steps (for live previs) or 4 steps (for final HD textures).
  2. Decoding Without a Decoder (DWD). Instead of passing latent tensors through a heavy 3D convolutional video VAE decoder, pi-H3 predicts direct pixel space outputs through a lightweight patch projection head. Dropping the VAE decoder eliminates both the decoding time spike and tiled VAE seam artifacts.
  3. Joint stereo audio and video coherence. Because pi-H3 distills the full dual stream MiniMax H3 backbone, dialogue lip timing, footsteps, ambient room tone, and camera movement remain locked together even at 2 to 4 sampling steps.
LakonLab policy flow and asymmetric distillation architecture diagram

Policy flow distillation mechanics inside LakonLab enabling 2 to 4 step generation without adversarial collapse. Credit: LakonLab.

Traditional consistency distillation loses high frequency film grain and breaks stereo audio synchronization below 8 steps.

LakonLab's policy flow formulation preserves sharp edges, specular highlights, and tight dialogue transients at 4 steps while running faster than real time on high bandwidth GPUs.

Try pi-H3 Live in the Browser

You can test pi-H3 directly in the official Hugging Face Space below (lakonik-pi-h3.hf.space) without local GPU installation.

Hardware, Speed, and Step Trade Off Matrix

Configuration Steps (NFE) Video Decoder 5s HD Render Time VRAM Footprint Best Production Use
pi-H3 Ultra Fast (DWD) 2 steps Bypassed (DWD) ~2.1s (B200) / ~9s (RTX 5090) 24 GB – 32 GB Live blocking, camera angle tests, client pitch tables
pi-H3 Production (DWD) 4 steps Bypassed (DWD) ~3.8s (B200) / ~16s (RTX 5090) 24 GB – 32 GB Final HD clips with crisp skin texture and stereo audio
Base MiniMax H3 Teacher 30 – 50 steps Full 3D Video VAE 45s – 90s+ 48 GB+ Offline hero shots where maximum latency is acceptable

Across LakonLab's flow matching benchmarks, few step policy distilled models retain structural fidelity and temporal coherence while delivering an order of magnitude speedup over base teachers.

Removing the video VAE pass also cuts peak VRAM spikes during 720p and 1080p decoding, preventing out of memory crashes on single workstation GPUs.

LakonLab few step flow matching quantitative quality comparison chart

Quantitative quality and speed curves across few step flow matching distillation schedules. Credit: LakonLab.

Parameter Reference Table

Parameter Recommended Value Effect on Output
--steps 2 or 4 2 for sub real time previs; 4 for clean texture, foliage, and lip sync.
--use_dwd true Activates Decoding Without a Decoder to skip the 3D video VAE pass.
--audio_enabled true Generates synchronized stereo audio alongside the HD video stream.
--guidance_scale 1.0 Policy distilled weights operate CFG free; raising CFG adds artifacts.
--duration 5.0 seconds Matches the native 5 second HD training window for stable motion.

Local Setup with LakonLab

Clone the LakonLab framework and run 4 step pi-H3 inference locally:

git clone https://github.com/Lakonik/LakonLab.git
cd LakonLab
pip install -e .
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124
import torch
from lakonlab.pipelines import PiH3Pipeline

pipe = PiH3Pipeline.from_pretrained(
    "Lakonik/pi-H3",
    torch_dtype=torch.bfloat16,
)
pipe.to("cuda")

result = pipe(
    prompt=(
        "Handheld 50mm close up of a jazz drummer hitting a brass snare in a dimly lit "
        "basement club, tungsten rim light, warm film grain, crisp snare crack and room reverb."
    ),
    num_inference_steps=4,
    guidance_scale=1.0,
    use_dwd=True,
    fps=24,
    num_frames=121,
)

result.save_mp4("jazz_snare_4step_dwd.mp4", include_audio=True)

Filmmaker Production Playbook

  1. Iterate camera blocking at 2 steps, then lock final takes at 4 steps. Keep the random seed fixed while you adjust lens focal length and lighting cues at 2 steps, then switch to 4 steps for the keeper take.
  2. Start from a 4K multi reference plate. Feed a key frame generated with MiniMax-H3-Image or FLUX 2 into pi-H3 image to video mode so costume, cast identity, and set dressing stay locked from frame 1.
  3. Check base MiniMax H3 license scope. Because pi-H3 builds on the MiniMax H3 architecture, verify the upstream MiniMax H3 license terms for your region before deploying local checkpoints in commercial client pipelines.

Related Reading


Sources