FramePack & Wan 2.1 FLF2V: Open Source Start-End Video AI

Controlling both the opening frame and the final frame of a generated shot is one of the most requested capabilities in narrative AI filmmaking. Standard image-to-video pipelines accept an initial reference image and extrapolate forward in time, which frequently causes the camera or subject to drift away from the next shot in a storyboard. Two open source video architectures solve this boundary constraint through distinct technical approaches. FramePack (developed by Lvmin Zhang, Shengqu Cai, Muyang Li, Gordon Wetzstein, and Maneesh Agrawala at Stanford University and MIT, and accepted at NeurIPS 2025) compresses historical and boundary frame contexts into a constant token budget that runs on consumer GPUs starting at 6 GB to 12 GB of VRAM. Wan 2.1 FLF2V 14B (released by the Wan Team under an Apache 2.0 license) trains a 14 billion parameter Diffusion Transformer (DiT) natively conditioned on both first and last keyframes at 720p (1280x720) resolution.

Studio availability. FramePack FLF2V and Wan 2.1 FLF2V are not available in AI FILMS Studio. This article covers a public research release. To generate video in the Studio, see the models listed on the start-to-end frame video page.

Why Start and End Frame Control Matters for AI Filmmakers

In a production storyboard, every cut connects two intentional compositions. When an artist generates Shot A and Shot B using an image foundation model, the transition between the end of Shot A and the start of Shot B requires exact spatial alignment, matching lighting, and controlled camera trajectory. Forward only text-to-video and image-to-video models treat generation as an initial value problem. They know where the camera starts, but they have no mathematical anchor for where the subject must land at frame 81 or frame 240.

First and last frame to video (FLF2V) reformulates video synthesis as a boundary value problem. By locking both the opening keyframe ($I_{\text{start}}$) and the closing keyframe ($I_{\text{end}}$), filmmakers can execute three high value production tasks:

1. Storyboard keyframe bridging. Generate exact transitions between two art directed concept frames without manual morphing or optical flow artifacts.
2. Match cuts and blocking transitions. Move an actor from a wide establishing posture to a specific close up framing while preserving garment texture and background geometry.
3. Continuous looping and cyclical backgrounds. Set the first frame and last frame to the exact same image (or an inverted pair) to synthesize continuous environmental loops for virtual production plates.

How FramePack Compresses Context to Run on 6 GB to 12 GB VRAM

Traditional autoregressive and next frame prediction video models suffer from quadratic attention scaling and linear KV cache growth as frame count increases. If a Diffusion Transformer processes 480p latents where each frame equals 1,536 latent tokens after a (1, 2, 2) patchification kernel, storing 60 seconds of video at 30 frames per second (1,800 frames) would require over 2.7 million context tokens, exhausting even 80 GB datacenter GPUs.

FramePack (arXiv:2504.12626) eliminates this memory bottleneck by applying a progressive geometric compression schedule to context frames based on their temporal distance from the frames currently being generated. Because adjacent frames in a video contain high visual redundancy, FramePack assigns progressive 3D patchification kernels with compression rate $\lambda = 2$:

• Anchor and immediate frames ($F_0$). Patchified with kernel (1, 2, 2), consuming the full 1,536 latent tokens per frame.
• Secondary history window. Patchified with kernel (2, 4, 4) (compression factor $2^3 = 8$), reducing representation to 192 tokens per frame.
• Distant history window. Patchified with kernel (4, 8, 8) (compression factor $64$), reducing representation to 24 tokens per frame, followed by an optional global average pooling tail.

As the geometric series converges, the total context length $L$ across an arbitrarily long video is bounded by a constant:

$$L \le L_f \cdot \left(S + \frac{\lambda}{\lambda - 1}\right)$$

Where $L_f = 1536$ is the single frame token count and $S$ is the small window of uncompressed frames generated per step. Regardless of whether the timeline spans 120 frames or 1,800 frames, the transformer attention matrix sees a fixed token budget under 4,608 tokens. This $O(1)$ complexity enables a 13 billion parameter HunyuanVideo backbone (lllyasviel/FramePackI2V_HY) to generate 60 second videos on a 6 GB RTX 3060 laptop GPU, or run full start and end frame interpolation (framepack-flf2v) inside 12 GB of VRAM using bfloat16 precision.

Inverted Anti Drifting Sampling for Start and End Keyframes

A well known failure mode of causal next frame prediction is temporal drifting. As errors accumulate over sequential sections, color saturation shifts, background details blur, and character anatomy degrades. FramePack solves this by breaking strict causal order through bi directional context conditioning and inverted anti drifting sampling.

Instead of generating frames strictly from index $0 \to N$, FramePack first anchors both the initial user keyframe ($F_0$) and the final target keyframe ($F_N$) inside the uncompressed (1, 2, 2) high priority slot (1,536 tokens each). During subsequent denoising sections, the sampler fills in the intermediate trajectory in reverse or bi directional order (inverted anti-drifting). Because every intermediate generation step directly attends to the uncompressed first frame and last frame simultaneously, error propagation cannot push the trajectory off target. On an NVIDIA RTX 4090, FramePack achieves 2.5 seconds per frame natively and 1.5 seconds per frame when combined with TeaCache acceleration.

How Wan 2.1 FLF2V 14B Synthesizes Native 720p Keyframe Transitions

While FramePack adapts a 13B backbone via progressive context packing and section by section decoding, Wan 2.1 FLF2V 14B (Wan-AI/Wan2.1-FLF2V-14B-720P-diffusers, documented in arXiv:2503.20314) approaches start and end frame generation through full sequence joint spatiotemporal attention at 720p (1280x720).

The Wan 2.1 architecture consists of three tightly integrated components:

1. Wan-VAE (3D Causal Variational Autoencoder). Compresses raw RGB video by $4 \times 8 \times 8$ across time, height, and width while preserving temporal causality via feature caching. Unlike standard 3D VAEs that run out of memory when decoding long clips, Wan-VAE encodes and decodes arbitrary length 1080p and 720p streams chunk by chunk without temporal flickering at chunk boundaries.
2. First and Last Frame Conditioning Mask. To condition the 14B Diffusion Transformer on both endpoints, the pipeline places the encoded first frame latent at index $t=0$ and the encoded end frame latent at index $t=T-1$, filling intermediate slots with zeroed conditioning latents alongside a binary temporal mask that flags which frames are fixed ground truth and which frames require denoising.
3. Full Spatiotemporal DiT with CLIP and UMT5 Conditioning. In addition to the 512 token multilingual UMT5 text embedding via cross attention, Wan2.1-FLF2V-14B-720P extracts CLIP visual embeddings from both the start frame and the end frame. These visual tokens are injected into the transformer layers so that even when a camera pans 90 degrees between the opening and closing compositions, the model maintains global semantic awareness of the destination scene.

Because Wan 2.1 FLF2V denoises the entire 81 frame 720p window jointly across 40 to 50 flow matching steps, it excels at complex physical transformations such as liquid pours, smoke dissipation, full body athletic motion, and camera crane movements where the start and end perspectives differ substantially.

Hardware Requirements & Open Source Comparison

Filmmakers building local pipelines or evaluating open source weights can choose between the two architectures based on their GPU memory ceiling and target clip duration:

Specification	FramePack FLF2V (HunyuanVideo 13B)	Wan 2.1 FLF2V 14B 720P
Hugging Face Checkpoint	lllyasviel/FramePackI2V_HY	Wan-AI/Wan2.1-FLF2V-14B-720P-diffusers
Parameter Count	13 Billion (bfloat16 / fp8)	14 Billion (bfloat16 / fp8)
Minimum VRAM	6 GB (with offload) / 12 GB recommended	18 GB (with CPU offload) / 24 GB native
Disk Footprint	~47 GB (backbone + Flux Redux + encoders)	~28 GB (diffusers bfloat16 weights)
Output Resolution	Up to 832x480 / 704x544 (30 fps)	Native 1280x720 (720p, 16 fps / 24 fps)
Clip Duration Range	1 second to 60 seconds (1,800 frames)	5 seconds (81 frames per generation pass)
Sampling Mechanism	Section based inverted anti drifting ($O(1)$ context)	Full sequence Flow Matching DiT
License	Apache 2.0 (FramePack code) + Hunyuan Community	Apache 2.0 (code and weights)

For creators working on similar camera trajectory and first frame conditioning problems, see our technical breakdowns of Time-to-Move (TTM) motion control and First Frame Go video customization.

Running Wan 2.1 FLF2V and FramePack Locally in Python

Both models integrate with the standard PyTorch and Hugging Face diffusers stack. Below is the minimal local inference script for running Wan-AI/Wan2.1-FLF2V-14B-720P-diffusers on a 24 GB GPU (such as an RTX 3090 or RTX 4090) using model CPU offloading to keep peak VRAM within budget:

import torch
from diffusers import AutoencoderKLWan, WanImageToVideoPipeline
from diffusers.utils import export_to_video, load_image
from transformers import CLIPVisionModel

model_id = "Wan-AI/Wan2.1-FLF2V-14B-720P-diffusers"

image_encoder = CLIPVisionModel.from_pretrained(
    model_id, subfolder="image_encoder", torch_dtype=torch.float32
)
vae = AutoencoderKLWan.from_pretrained(
    model_id, subfolder="vae", torch_dtype=torch.float32
)
pipe = WanImageToVideoPipeline.from_pretrained(
    model_id,
    vae=vae,
    image_encoder=image_encoder,
    torch_dtype=torch.bfloat16,
)
pipe.enable_model_cpu_offload()

first_frame = load_image("storyboard_shot_start.png").resize((1280, 720))
last_frame = load_image("storyboard_shot_end.png").resize((1280, 720))

output = pipe(
    image=first_frame,
    last_image=last_frame,
    prompt="Cinematic dolly in across a rain-slicked neon alley, anamorphic lens, 35mm film grain",
    negative_prompt="overexposed, static, blurred details, deformed hands, temporal flicker",
    height=720,
    width=1280,
    num_frames=81,
    guidance_scale=5.0,
).frames[0]

export_to_video(output, "flf2v_transition_720p.mp4", fps=16)

For FramePack FLF2V on a 12 GB or 16 GB GPU, enabling TeaCache (use_teacache=True) cuts diffusion step time by nearly 40 percent, though filmmakers grading final shots should run a pass with TeaCache disabled (25 steps) to preserve maximum skin and fabric micro texture.

Generating Start and End Frame Video in AI FILMS Studio Today

While FramePack FLF2V and Wan 2.1 FLF2V run as standalone open source packages, AI FILMS Studio provides hosted start-to-end keyframe models directly inside the Video Generation Workspace and on the Start-End Frame Video Generator page. Filmmakers who want to bridge two storyboard frames without managing local 28 GB to 47 GB weight downloads can select from three production ready cloud models:

• FLUX 3 Start-End to Video. Built for cinematic transitions with multi shot coherence and native audio synchronization. Read our FLUX 3 video generation tutorial for prompt structure and camera movement tips.
• Vidu Q3 Pro Start-End to Video. Designed for expressive character blocking and high motion anime or live action keyframe interpolation up to 1080p.
• ByteDance Seedance 2 First-Last Frame. Optimized for fluid physical motion and complex camera arcs between wide establishing shots and close up keyframes.

Inside the Nodes Graph Editor, you can also chain two image generation nodes directly into a start and end video node, allowing you to iterate on both boundary frames and preview the resulting camera transition inside a single visual graph.

---

Sources

FramePack Paper (arXiv:2504.12626) | FramePack GitHub (lllyasviel/FramePack) | Wan 2.1 Technical Report (arXiv:2503.20314) | Wan-AI Hugging Face (Wan2.1-FLF2V-14B-720P)