
Yong Liu et al., UltraVSR (arXiv:2505.19958, Apache 2.0) & SVI360 (ECCV 2026)
Share this post:
SVI360 & UltraVSR: Open Source 360 Video & 4K Upscaling
SVI360 (Spherical Video Interpolation, arXiv:2607.11710, accepted at ECCV 2026) and UltraVSR (Achieving Ultra-Realistic Video Super Resolution with Efficient One Step Diffusion Space, arXiv:2505.19958, Apache 2.0) address the two post processing bottlenecks that separate raw AI video generations from cinema and LED volume delivery: frame rate interpolation on 360 degree spherical panoramas and one step 4x spatial super resolution.
Studio availability. SVI360 and UltraVSR are not available in AI FILMS Studio. This article covers public research releases. To upscale video or interpolate start and end frames in the Studio, see the models listed on the Video Upscaler and Start to End Frame pages.
SVI360 spherical 360 degree video frame interpolation eliminating zenith and nadir polar stretching
UltraVSR one step diffusion 4x video super resolution restoring sharp textures on degraded footage
How SVI360 Eliminates Polar Distortion in 360 Degree Panorama Interpolation
When virtual production teams generate 360 degree equirectangular environment plates for Unreal Engine skyboxes or LED volumes, planar frame interpolation models (RIFE, AMT, or FILM) fail near the top and bottom poles. Because an equirectangular projection stretches a single polar point across the entire top and bottom width of the 2D image, standard 2D optical flow estimators predict chaotic horizontal vectors at the zenith and nadir, creating visible seams when wrapped onto a sphere.
SVI360 pairs primitive and 90 degree rotated orthogonal spherical branches with Cross View Spherical Refinement. Credit: Islam et al., SVI360 (ECCV 2026).
SVI360 resolves polar distortion through a dual branch architecture. Alongside the Primitive Branch (standard equirectangular orientation), an Orthogonal Branch rotates the 3D sphere by 90 degrees so the distorted north and south poles move to the distortion free equator.
A Cross View Spherical Refinement (CSR) block then fuses both views using geodesic spherical distances (`Flow360` benchmark), boosting PSNR by over `1.12 dB` at the poles and eliminating wrap around seams.
How UltraVSR Achieves 4x Video Super Resolution in One Diffusion Step
Multi step diffusion video upscalers (20 to 50 steps) take minutes per shot and introduce random temporal noise between frames. UltraVSR compresses diffusion video restoration into a deterministic one step diffusion space built on Stable Diffusion 2.1 (stabilityai/stable-diffusion-2-1-base) under an Apache 2.0 license.
Instead of starting from pure Gaussian noise, UltraVSR introduces a Degradation aware Restoration Schedule (DRS) that estimates the exact degradation level of the low resolution input clip and jumps directly to the matching single denoising timestep.
To lock frame to frame stability without heavy 3D attention overhead, a lightweight Recurrent Temporal Shift (RTS) module shifts feature channels bidirectionally across adjacent frames during the single UNet pass.
UltraVSR one step diffusion restoration pipeline with Recurrent Temporal Shift (RTS). Credit: Yong Liu et al. (Apache 2.0).
Hardware, VRAM & Model Comparison
| Framework | Repository & Backbone | Core Filmmaking Task | Hardware & VRAM | License |
|---|---|---|---|---|
SVI360 (SVI360-AMT-S / RIFE) |
ICB-Vision-AI/video360_interpolation |
360 degree equirectangular video frame interpolation (2x / 4x / 8x FPS) | 8 GB to 16 GB VRAM | Open Source (ECCV 2026) |
| UltraVSR | yongliuy/UltraVSR (SD 2.1 + LoRA + RTS) |
One step 4x diffusion video super resolution (--upscale 4) |
12 GB to 24 GB VRAM (supports --vae_decoder_tiled_size) |
Apache 2.0 |
Running SVI360 and UltraVSR Locally
To interpolate a 360 degree equirectangular video plate into smooth 60 FPS motion for an LED volume or VR headset, clone ICB-Vision-AI/video360_interpolation and run demo.py.
git clone https://github.com/ICB-Vision-AI/video360_interpolation.git
cd video360_interpolation
conda env create -f environment.yml
conda activate svi360
python demo.py \
--img0 assets/demo1/img0.png \
--img1 assets/demo1/img1.png \
--ckpt pretrained/SVI360_AMT-S.pth \
--out_dir ./outputs
To upscale a low resolution AI video clip by 4x in a single diffusion step with UltraVSR, clone yongliuy/UltraVSR, enable tiled VAE decoding for consumer GPUs, and run test_ultravsr.py.
git clone https://github.com/yongliuy/UltraVSR.git
cd UltraVSR
pip install -r requirements.txt
python test/test_ultravsr.py \
--pretrained_model_path checkpoints/stable-diffusion-2-1-base \
--ultravsr_path checkpoints/ultravsr.pkl \
--input_dir ./inputs/low_res_frames \
--output_dir ./outputs/upscaled_4k \
--upscale 4 \
--process_size 512 \
--vae_decoder_tiled_size 224
Virtual Production Mastering & AI FILMS Studio Workflows
In virtual production pipelines, SVI360 and UltraVSR form the finishing pass for panoramic environment plates and 720p AI shots. A technical director generates a 360 degree panorama sequence, interpolates its frame rate without polar tearing in SVI360, and upscales the final plate 4x in UltraVSR for projection on an LED wall.
For related open source 360 panorama and video upscaling guides, read our coverage of DiT360 panoramic image generation, HY-World 2.0 3D world and 360 panorama generation, SeedVR2-7B and Real-ESRGAN video upscaling, and FlashVSR real time video super resolution.
To upscale video clips or morph start to end keyframes directly in the browser, visit the Video Upscaler and Start to End Frame pages, or chain generation and upscaling nodes inside the Nodes Graph Editor.
Sources
- ICB-Vision-AI/video360_interpolation Official GitHub Repository
- SVI360 Official Project Page (ECCV 2026)
- SVI360: Spherical Video Interpolation (arXiv:2607.11710)
- yongliuy/UltraVSR Official GitHub Repository (Apache 2.0)
- UltraVSR: Achieving Ultra-Realistic Video Super Resolution with Efficient One Step Diffusion Space (arXiv:2505.19958)
Continue Reading
Video & LipSync
- Video Generator
- Text to Video
- Image to Video
- Start-End Frame to Video
- Draw to Video
- Motion Control
- Video Enhancer
- Video Upscaler
- Video Extend
- Video to Video LipSync
- Audio to Video LipSync
- Image to Video LipSync
- Video FaceSwap
- Seedance 2.5
- WAN 3.0
- Seedance 2
- FLUX 3
- Minimax H3
- Vidu Q3 Pro
- Grok Imagine Video 1.5
- Gemini Omni
- Google Veo 3.1
- Kling 3.0 Pro
- Luma Ray 3.2
- LTX 2.3
- Happy Horse 1.1
- Kling 3.0 Motion
- ByteDance Upscaler
- InfiniteTalk
- InsightFace
Image & Edit
- AI Character
- AI Actor
- Art Generator
- Text to Image
- Image to Image
- LoRA Image to Image
- Draw to Edit
- Image LoRA Training
- Remove Background
- Image Enhancer
- MidJourney 8.0
- Krea 2
- OpenAI GPT Image 2.0
- Nano Banana Pro Ultra
- Nano Banana 2 Lite
- Nano Banana Pro
- Minimax Image
- Clarity Pro
- Z-Image Turbo LoRA
- Krea 2 Turbo LoRA
- Z-Image LoRA Trainer
- Qwen Image LoRA Trainer
- Background Erase Network 2 (BEN2)


