EditorNodesPricingBlog

SVI360 & UltraVSR: Open Source 360 Video & 4K Upscaling

November 17, 2026
SVI360 & UltraVSR: Open Source 360 Video & 4K Upscaling

Yong Liu et al., UltraVSR (arXiv:2505.19958, Apache 2.0) & SVI360 (ECCV 2026)

Share this post:

SVI360 & UltraVSR: Open Source 360 Video & 4K Upscaling

SVI360 (Spherical Video Interpolation, arXiv:2607.11710, accepted at ECCV 2026) and UltraVSR (Achieving Ultra-Realistic Video Super Resolution with Efficient One Step Diffusion Space, arXiv:2505.19958, Apache 2.0) address the two post processing bottlenecks that separate raw AI video generations from cinema and LED volume delivery: frame rate interpolation on 360 degree spherical panoramas and one step 4x spatial super resolution.

Studio availability. SVI360 and UltraVSR are not available in AI FILMS Studio. This article covers public research releases. To upscale video or interpolate start and end frames in the Studio, see the models listed on the Video Upscaler and Start to End Frame pages.

SVI360 spherical 360 degree video frame interpolation eliminating zenith and nadir polar stretching

UltraVSR one step diffusion 4x video super resolution restoring sharp textures on degraded footage

How SVI360 Eliminates Polar Distortion in 360 Degree Panorama Interpolation

When virtual production teams generate 360 degree equirectangular environment plates for Unreal Engine skyboxes or LED volumes, planar frame interpolation models (RIFE, AMT, or FILM) fail near the top and bottom poles. Because an equirectangular projection stretches a single polar point across the entire top and bottom width of the 2D image, standard 2D optical flow estimators predict chaotic horizontal vectors at the zenith and nadir, creating visible seams when wrapped onto a sphere.

SVI360 dual primitive and orthogonal spherical branch architecture for 360 video interpolation

SVI360 pairs primitive and 90 degree rotated orthogonal spherical branches with Cross View Spherical Refinement. Credit: Islam et al., SVI360 (ECCV 2026).

SVI360 resolves polar distortion through a dual branch architecture. Alongside the Primitive Branch (standard equirectangular orientation), an Orthogonal Branch rotates the 3D sphere by 90 degrees so the distorted north and south poles move to the distortion free equator.

A Cross View Spherical Refinement (CSR) block then fuses both views using geodesic spherical distances (`Flow360` benchmark), boosting PSNR by over `1.12 dB` at the poles and eliminating wrap around seams.

How UltraVSR Achieves 4x Video Super Resolution in One Diffusion Step

Multi step diffusion video upscalers (20 to 50 steps) take minutes per shot and introduce random temporal noise between frames. UltraVSR compresses diffusion video restoration into a deterministic one step diffusion space built on Stable Diffusion 2.1 (stabilityai/stable-diffusion-2-1-base) under an Apache 2.0 license.

Instead of starting from pure Gaussian noise, UltraVSR introduces a Degradation aware Restoration Schedule (DRS) that estimates the exact degradation level of the low resolution input clip and jumps directly to the matching single denoising timestep.

To lock frame to frame stability without heavy 3D attention overhead, a lightweight Recurrent Temporal Shift (RTS) module shifts feature channels bidirectionally across adjacent frames during the single UNet pass.

UltraVSR Degradation aware Restoration Schedule and Recurrent Temporal Shift one step network

UltraVSR one step diffusion restoration pipeline with Recurrent Temporal Shift (RTS). Credit: Yong Liu et al. (Apache 2.0).

Hardware, VRAM & Model Comparison

Framework Repository & Backbone Core Filmmaking Task Hardware & VRAM License
SVI360 (SVI360-AMT-S / RIFE) ICB-Vision-AI/video360_interpolation 360 degree equirectangular video frame interpolation (2x / 4x / 8x FPS) 8 GB to 16 GB VRAM Open Source (ECCV 2026)
UltraVSR yongliuy/UltraVSR (SD 2.1 + LoRA + RTS) One step 4x diffusion video super resolution (--upscale 4) 12 GB to 24 GB VRAM (supports --vae_decoder_tiled_size) Apache 2.0

Running SVI360 and UltraVSR Locally

To interpolate a 360 degree equirectangular video plate into smooth 60 FPS motion for an LED volume or VR headset, clone ICB-Vision-AI/video360_interpolation and run demo.py.

git clone https://github.com/ICB-Vision-AI/video360_interpolation.git
cd video360_interpolation
conda env create -f environment.yml
conda activate svi360

python demo.py \
  --img0 assets/demo1/img0.png \
  --img1 assets/demo1/img1.png \
  --ckpt pretrained/SVI360_AMT-S.pth \
  --out_dir ./outputs

To upscale a low resolution AI video clip by 4x in a single diffusion step with UltraVSR, clone yongliuy/UltraVSR, enable tiled VAE decoding for consumer GPUs, and run test_ultravsr.py.

git clone https://github.com/yongliuy/UltraVSR.git
cd UltraVSR
pip install -r requirements.txt

python test/test_ultravsr.py \
  --pretrained_model_path checkpoints/stable-diffusion-2-1-base \
  --ultravsr_path checkpoints/ultravsr.pkl \
  --input_dir ./inputs/low_res_frames \
  --output_dir ./outputs/upscaled_4k \
  --upscale 4 \
  --process_size 512 \
  --vae_decoder_tiled_size 224

Virtual Production Mastering & AI FILMS Studio Workflows

In virtual production pipelines, SVI360 and UltraVSR form the finishing pass for panoramic environment plates and 720p AI shots. A technical director generates a 360 degree panorama sequence, interpolates its frame rate without polar tearing in SVI360, and upscales the final plate 4x in UltraVSR for projection on an LED wall.

For related open source 360 panorama and video upscaling guides, read our coverage of DiT360 panoramic image generation, HY-World 2.0 3D world and 360 panorama generation, SeedVR2-7B and Real-ESRGAN video upscaling, and FlashVSR real time video super resolution.

To upscale video clips or morph start to end keyframes directly in the browser, visit the Video Upscaler and Start to End Frame pages, or chain generation and upscaling nodes inside the Nodes Graph Editor.


Sources