
LakonLab (Hansheng Chen), pi-Flow distillation and pi-H3 open source video generation
Share this post:
pi-H3: Open Source 2-Step Audio Video Generation
pi-H3 (Lakonik/pi-H3, November 2026) is an open source distillation and inference release from LakonLab (Hansheng Chen and collaborators from Stanford and Luma AI) that compresses the MiniMax H3 audio visual foundation model down to 2 to 4 sampling steps while removing the video VAE decoder entirely. Using pi-Flow V2 policy distillation and Decoding Without a Decoder (DWD), pi-H3 renders a 5 second HD video clip with synchronized stereo audio in roughly 3.8 seconds on a single NVIDIA B200 GPU.
Studio availability. Try the public Hugging Face demo below; pi-H3 is not part of a paid AI FILMS Studio plan (use MiniMax H3 or WAN 3.0 in the video workspace).

Why Video VAE Decoders Became the Hidden Bottleneck
After step distillation cut video Diffusion Transformers from 50 steps down to 4 steps, engineers ran into an unexpected wall. In modern 3D latent video pipelines, the final 3D VAE decoding pass often consumes 35% to 55% of total wall clock latency and spikes peak VRAM right at the end of generation.
pi-H3 attacks both denoising compute and decoding latency at the same time:
- pi-Flow V2 policy distillation (
2to4NFE). Rather than relying on brittle adversarial GAN losses that wash out fine grain or flatten stereo soundscapes, LakonLab trains a policy based flow matching student that maps Gaussian noise to clean audio visual trajectories in as few as 2 steps (for live previs) or 4 steps (for final HD textures). - Decoding Without a Decoder (
DWD). Instead of passing latent tensors through a heavy 3D convolutional video VAE decoder, pi-H3 predicts direct pixel space outputs through a lightweight patch projection head. Dropping the VAE decoder eliminates both the decoding time spike and tiled VAE seam artifacts. - Joint stereo audio and video coherence. Because pi-H3 distills the full dual stream MiniMax H3 backbone, dialogue lip timing, footsteps, ambient room tone, and camera movement remain locked together even at 2 to 4 sampling steps.
Policy flow distillation mechanics inside LakonLab enabling 2 to 4 step generation without adversarial collapse. Credit: LakonLab.
Traditional consistency distillation loses high frequency film grain and breaks stereo audio synchronization below 8 steps.
LakonLab's policy flow formulation preserves sharp edges, specular highlights, and tight dialogue transients at 4 steps while running faster than real time on high bandwidth GPUs.
Try pi-H3 Live in the Browser
You can test pi-H3 directly in the official Hugging Face Space below (lakonik-pi-h3.hf.space) without local GPU installation.
Hardware, Speed, and Step Trade Off Matrix
| Configuration | Steps (NFE) | Video Decoder | 5s HD Render Time | VRAM Footprint | Best Production Use |
|---|---|---|---|---|---|
| pi-H3 Ultra Fast (DWD) | 2 steps |
Bypassed (DWD) |
~2.1s (B200) / ~9s (RTX 5090) |
24 GB – 32 GB |
Live blocking, camera angle tests, client pitch tables |
| pi-H3 Production (DWD) | 4 steps |
Bypassed (DWD) |
~3.8s (B200) / ~16s (RTX 5090) |
24 GB – 32 GB |
Final HD clips with crisp skin texture and stereo audio |
| Base MiniMax H3 Teacher | 30 – 50 steps |
Full 3D Video VAE | 45s – 90s+ |
48 GB+ |
Offline hero shots where maximum latency is acceptable |
Across LakonLab's flow matching benchmarks, few step policy distilled models retain structural fidelity and temporal coherence while delivering an order of magnitude speedup over base teachers.
Removing the video VAE pass also cuts peak VRAM spikes during 720p and 1080p decoding, preventing out of memory crashes on single workstation GPUs.
Quantitative quality and speed curves across few step flow matching distillation schedules. Credit: LakonLab.
Parameter Reference Table
| Parameter | Recommended Value | Effect on Output |
|---|---|---|
--steps |
2 or 4 |
2 for sub real time previs; 4 for clean texture, foliage, and lip sync. |
--use_dwd |
true |
Activates Decoding Without a Decoder to skip the 3D video VAE pass. |
--audio_enabled |
true |
Generates synchronized stereo audio alongside the HD video stream. |
--guidance_scale |
1.0 |
Policy distilled weights operate CFG free; raising CFG adds artifacts. |
--duration |
5.0 seconds |
Matches the native 5 second HD training window for stable motion. |
Local Setup with LakonLab
Clone the LakonLab framework and run 4 step pi-H3 inference locally:
git clone https://github.com/Lakonik/LakonLab.git
cd LakonLab
pip install -e .
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124
import torch
from lakonlab.pipelines import PiH3Pipeline
pipe = PiH3Pipeline.from_pretrained(
"Lakonik/pi-H3",
torch_dtype=torch.bfloat16,
)
pipe.to("cuda")
result = pipe(
prompt=(
"Handheld 50mm close up of a jazz drummer hitting a brass snare in a dimly lit "
"basement club, tungsten rim light, warm film grain, crisp snare crack and room reverb."
),
num_inference_steps=4,
guidance_scale=1.0,
use_dwd=True,
fps=24,
num_frames=121,
)
result.save_mp4("jazz_snare_4step_dwd.mp4", include_audio=True)
Filmmaker Production Playbook
- Iterate camera blocking at 2 steps, then lock final takes at 4 steps. Keep the random seed fixed while you adjust lens focal length and lighting cues at 2 steps, then switch to 4 steps for the keeper take.
- Start from a 4K multi reference plate. Feed a key frame generated with MiniMax-H3-Image or FLUX 2 into
pi-H3image to video mode so costume, cast identity, and set dressing stay locked from frame 1. - Check base MiniMax H3 license scope. Because
pi-H3builds on the MiniMax H3 architecture, verify the upstream MiniMax H3 license terms for your region before deploying local checkpoints in commercial client pipelines.
Related Reading
- MiniMax H3 Image: Open Weights 4K Multi Reference AI
- FreeVideo, MiniMax H3, and Pika H3 Flex: Open Source Video AI
- DMAD & DuoMatching: Open Source 4-Step Audio Video AI
- Apple FastFrames: Open Source Real Time Video Diffusion
Sources
Continue Reading
Video & LipSync
- Video Generator
- Text to Video
- Image to Video
- Start-End Frame to Video
- Draw to Video
- Motion Control
- Video Enhancer
- Video Upscaler
- Video Extend
- Video to Video LipSync
- Audio to Video LipSync
- Image to Video LipSync
- Video FaceSwap
- Seedance 2.5
- WAN 3.0
- Seedance 2
- FLUX 3
- Minimax H3
- Vidu Q3 Pro
- Grok Imagine Video 1.5
- Gemini Omni
- Google Veo 3.1
- Kling 3.0 Pro
- Luma Ray 3.2
- LTX 2.3
- Happy Horse 1.1
- Kling 3.0 Motion
- ByteDance Upscaler
- InfiniteTalk
- InsightFace
Image & Edit
- AI Character
- AI Actor
- Art Generator
- Text to Image
- Image to Image
- LoRA Image to Image
- Draw to Edit
- Image LoRA Training
- Remove Background
- Image Enhancer
- MidJourney 8.0
- Krea 2
- OpenAI GPT Image 2.0
- Nano Banana Pro Ultra
- Nano Banana 2 Lite
- Nano Banana Pro
- Minimax Image
- Clarity Pro
- Z-Image Turbo LoRA
- Krea 2 Turbo LoRA
- Z-Image LoRA Trainer
- Qwen Image LoRA Trainer
- Background Erase Network 2 (BEN2)


