OmniDirector: Camera Motion Cloning from Multi Shot Videos

OmniDirector is an open source method from Kuaishou's Kling team that clones camera motions from reference videos and applies them to animate static images. Released under the MIT license on June 11, 2026, it handles dolly zooms, bullet time effects, aerial fly throughs, and dramatic descents without requiring paired training data matching camera movements to specific scenes.

The model comes from the Kling research team at Kuaishou Technology, the same group behind Kling AI motion control and the broader Kuaishou AI video platform. Jiwen Liu and Shujuan Li are equal first authors, with additional collaborators from Tsinghua and Peking universities. The paper is arXiv:2606.13432.

Camera Grid Representation

OmniDirector introduces a Camera Grid Representation. Rather than learning from matched video pairs showing the same scene from different camera positions, it renders camera poses from a reference video into empty 3D space as a visual encoding. The model reads that grid and transfers the encoded trajectory to animate a source image.

This decoupling of motion from content eliminates the need for cross paired training data. Other camera control methods require matched examples tied to specific scene types. OmniDirector extracts camera behavior from any reference video regardless of what it depicts, making it applicable across portraits, architecture, wildlife, and AI generated imagery. A Hierarchical Prompt Expansion Agent coordinates character action and camera motion for multimodal control.

Dynamic Motion

The Dynamic Motion capability transfers camera trajectories from a single reference shot to a new scene. The model preserves scene geometry through aggressive moves, including rapid pans and fast dolly sequences, that would lose spatial coherence without the grid representation anchoring the trajectory.

Multi Shot Camera Cloning

Multi shot cloning applies a camera sequence across several consecutive shots, preserving compositional logic and transitions between angles. The model keeps scene coherence as the virtual camera cuts between positions, producing visual continuity that normally requires coordinated planning across multiple physical takes.

This capability connects directly to the challenge of multi shot video generation, where maintaining visual grammar across cuts is one of the hardest problems in AI filmmaking. OmniDirector approaches it from the camera side rather than the character or scene side.

Scene Generalization

OmniDirector generalizes across different scene types without retraining for each domain. A camera trajectory extracted from a cityscape applies equally to a portrait, an animal shot, or AI generated content, because the camera grid encodes motion independently of scene content.

This cross domain capability is significant for practical filmmaking workflows. A director can reference any existing footage for its camera language, regardless of whether the content or setting matches the target scene. Shot vocabulary from one production can inform the visual grammar of an entirely different one.

Special Camera Effects

The special camera effects capability covers named cinematic techniques that are among the most technically demanding shots in physical production. The Hitchcock zoom, also called a dolly zoom, simultaneously moves the camera forward while zooming out, keeping the subject at the same apparent size while distorting the background. Bullet time effects freeze a moment and orbit the camera around it.

Other camera control research has approached these effects through separate models or constrained motion primitives. OmniDirector treats them as instances of the same camera cloning problem. A reference video that contains a dolly zoom provides the trajectory; the model executes it on any target image. Related approaches to camera control in AI video include CamCloneMaster, which focuses on extracting and reapplying movements from AI generated video sequences.

MIT License and Paper

OmniDirector is released under the MIT license, permitting commercial use. The paper is published on arXiv (2606.13432) alongside a code repository on GitHub. As of the June 11 release date, the repository contains the project documentation and reference implementation structure. The research comes from the same Kling team that has published consistently in AI video generation over the past two years.

Filmmakers experimenting with camera motion in AI video generation can use tools available today through AI FILMS Studio.

---

Sources

arXiv: OmniDirector: General Multi-Shot Camera Cloning without Cross-Paired Data GitHub: lisj575/OmniDirector Project Page: OmniDirector