SeedVR2 & Real-ESRGAN: Open Source Video Upscaling AI

Most generative video models synthesize their base latents at 480p, 540p, or 720p to keep attention compute tractable. Bringing those generated clips into a 1080p or 4K theatrical timeline requires a dedicated video super resolution (VSR) pass that sharpens fine textures without introducing frame to frame shimmer. Two open source architectures anchor modern AI filmmaking finishing pipelines across very different hardware budgets. SeedVR2 (developed by Jianyi Wang, Shanchuan Lin, Zhijie Lin, YuchuanPB, Meng Wei, Zongsheng Yue, Shangchen Zhou, Hao Chen, Yi Zhao, Ceyuan Yang, Xuefeng Xiao, Chen Change Loy, and Lu Jiang at Nanyang Technological University S-Lab and ByteDance Seed, published at ICLR 2026 under Apache 2.0) performs one step Diffusion Transformer (DiT) video restoration at arbitrary resolutions inside 14 GB of VRAM. Real-ESRGAN Video x4 (released by Xintao Wang et al. under BSD-3-Clause) delivers lightweight 4x spatial GAN upscaling requiring only 4 GB of VRAM.

Both SeedVR2 Video Upscaler and Real-ESRGAN Video Upscaler are live in AI FILMS Studio inside the Video Generation Workspace and on the AI Video Upscaler page, and both models also release open weights for local workstation pipelines.

Why AI Generated Footage Needs Specialized Video Upscaling

Upscaling AI generated video (AIGC) is fundamentally harder than upscaling clean camera footage. Diffusion video backbones frequently leave behind three specific artifacts that defeat traditional Lanczos or bicubic resizers:

1. VAE latent grid softness. 3D causal VAEs compress spatial dimensions by $8\times8$, smoothing out high frequency details such as skin pores, distant foliage, eyelash definition, and woven fabric.
2. Temporal boiling in background textures. When a traditional frame by frame image upscaler sharpens each frame independently, slight sub pixel variations in the underlying latent cause stone walls, hair strands, or film grain to flicker rapidly at 24 frames per second.
3. Motion blur and compression halos. Fast pans or multi step guidance often smear high contrast edges, requiring a generative prior that can reconstruct plausible structure rather than merely amplifying edge contrast.

Inside SeedVR2: One Step Diffusion Restoration with Adaptive Window Attention

Earlier diffusion based video super resolution models (such as Upscale-A-Video, MGLD-VSR, and VEnhancer) required 20 to 50 iterative denoising steps over a pre trained video diffusion backbone, making a 5 second 1080p upscale take several minutes on datacenter GPUs. SeedVR2 (arXiv:2506.05301, available in 3B and 7B parameter checkpoints) compresses the entire generative restoration trajectory into a single forward diffusion step (1 NFE).

Diffusion Adversarial Post Training and Feature Matching Loss

Distilling a multi step video diffusion transformer into one step typically destabilizes training or produces plastic, over smoothed surfaces. To prevent mode collapse without keeping a heavy pre trained teacher network in GPU memory during distillation, the SeedVR2 authors train the generator directly against real high resolution video data using diffusion adversarial post training.

Instead of relying on computationally expensive LPIPS networks that spike VRAM usage at 1080p, SeedVR2 extracts multi scale intermediate features directly from its own discriminator and minimizes a custom feature matching loss alongside a relativistic adversarial objective (RpGAN) and annealed Approximate Langevin Dynamics (ALD) regularization. This training schedule allows ByteDance-Seed/SeedVR2-7B and ByteDance-Seed/SeedVR2-3B to run 4x faster than multi step diffusion VSR baselines while preserving natural film texture and temporal consistency across frames.

Adaptive Window Attention for Arbitrary Resolutions

Fixed window Swin style 3D attention breaks down when an input video resolution is not evenly divisible by the predefined window size (e.g., 768x768). At 1080p (1920x1080) or widescreen cinema aspect ratios, unequal partial windows along the right and bottom borders create visible grid seams during one step generation.

SeedVR2 replaces fixed size windowing with Adaptive Window Attention. Before computing self attention, the transformer dynamically calculates window dimensions as a proportional fraction of the target spatial resolution (requiring only that height and width be multiples of 32). Every attention window across the frame contains balanced token counts regardless of whether the output is vertical 1080x1920, standard 1920x1080, or anamorphic widescreen.

Real-ESRGAN Video x4: Fast 4 GB VRAM Spatial Super Resolution

When a workstation has limited GPU memory (4 GB to 8 GB VRAM) or when an editor needs rapid 4x spatial magnification for stylized 2D animation, storyboards, or clean CG renders, Real-ESRGAN Video x4 (xinntao/Real-ESRGAN, arXiv:2107.10833) remains one of the most efficient open source tools available.

Built on a Residual in Residual Dense Block (RRDB) generator trained with a U-Net spectral norm discriminator and high order synthetic degradation pipelines (repeated Gaussian and isotropic blurs, Poisson and Gaussian noise, JPEG compression, and sinc ringing filters), RealESRGAN_x4plus (67 MB checkpoint) and realesr-animevideov3 process video streams frame by frame via OpenCV and ffmpeg worker threads.

Because Real-ESRGAN Video operates spatially without a multi frame 3D temporal attention window, it excels on clean line art, anime (realesr-animevideov3), and static camera plates where temporal hallucination must be zero, whereas SeedVR2 is purpose built for photorealistic AIGC shots that need temporal de flickering and generative detail synthesis. For additional research on low latency video super resolution, see our coverage of FlashVSR real-time AI video upscaling, SparkVSR interactive video super-resolution, and SwiftVR real-time video restoration.

Hardware & Technical Comparison: SeedVR2 vs Real-ESRGAN Video

Feature	SeedVR2-3B / SeedVR2-7B	Real-ESRGAN Video x4
Hugging Face / Repo	ByteDance-Seed/SeedVR2-7B (SeedVR2-3B)	xinntao/Real-ESRGAN (ai-forever/Real-ESRGAN)
Architecture	One Step Diffusion Transformer (DiT) + 3D VAE	RRDB GAN (RealESRGAN_x4plus / animevideov3)
Diffusion Steps	1 Step (1 NFE one step restoration)	Single pass GAN feedforward
Minimum VRAM	12 GB (3B) to 14 GB (7B with FP8/BF16 tiling)	4 GB VRAM (or 2 GB with tile splitting)
Weight Size	~6.8 GB (3B) / ~14 GB (7B)	~67 MB (RealESRGAN_x4plus.pth)
Temporal Modeling	Full 3D spatiotemporal adaptive window attention	Frame by frame spatial (animevideov3 tuned for video)
Best Filmmaking Use Case	Photorealistic AI video (AIGC), de flickering, 1080p/2K mastering	Fast 4x spatial upscaling, 2D animation, low VRAM laptops
Open Source License	Apache 2.0	BSD-3-Clause
AI FILMS Studio Status	Live (SeedVR2 Video Upscaler)	Live (Real-ESRGAN Video Upscaler)

Running SeedVR2 and Real-ESRGAN Locally vs in AI FILMS Studio

To run Real-ESRGAN Video x4 locally on a 4 GB to 8 GB GPU with ffmpeg audio passthrough and tile splitting enabled:

git clone https://github.com/xinntao/Real-ESRGAN.git
cd Real-ESRGAN
pip install basicsr facexlib gfpgan ffmpeg-python
python setup.py develop

# Upscale an AI film shot 4x with tile size 512 to stay under 4 GB VRAM
python inference_realesrgan_video.py \
  -i input_shot_540p.mp4 \
  -n RealESRGAN_x4plus \
  -s 4 \
  --tile 512 \
  --fp32

For SeedVR2-7B (ByteDance-Seed/SeedVR2-7B), downloading the 14 GB checkpoint from Hugging Face and running single step torchrun inference restores arbitrary length clips to 1080p (--res_h 1080 --res_w 1920) using --sp_size 1 on a single 16 GB or 24 GB GPU.

Cloud & Node Graph Workflows in AI FILMS Studio

If you want to upscale shots immediately without configuring PyTorch, basicsr, or flash_attn builds locally, both models are available right now inside AI FILMS Studio:

1. Direct Workspace Upscaling. Open the Video Generation Workspace or visit the AI Video Upscaler page and select SeedVR2 Video Upscaler (50 credits per second) for one step diffusion restoration or Real-ESRGAN Video Upscaler (50 credits per second) for fast spatial scaling. Studio also hosts Topaz Video Upscaler and ByteDance Video Upscaler alongside the open source models, plus dedicated still frame enhancers on the AI Image Upscaler page.
2. Nodes Graph Editor Chaining. In the Nodes Graph Editor, you can wire the output of any text-to-video or start and end frame generation node directly into a SeedVR2 Video Upscaler or Real-ESRGAN Video Upscaler node, building an automated multi stage pipeline that generates a 720p base cut and finishes it at 1080p or 4K in a single graph execution.

---

Sources

SeedVR2 Paper (arXiv:2506.05301 / ICLR 2026) | SeedVR GitHub (ByteDance-Seed/SeedVR) | Real-ESRGAN Paper (arXiv:2107.10833) | Real-ESRGAN GitHub (xinntao/Real-ESRGAN)