EditorNodesPricingBlog

Meta SAM 3: Open Source Promptable Video Segmentation

November 20, 2026
Meta SAM 3: Open Source Promptable Video Segmentation

Meta AI Research — SAM 3 Promptable Concept Segmentation architecture

Share this post:

Meta SAM 3: Open Source Promptable Video Segmentation

SAM 3 (Segment Anything Model 3) (facebookresearch/sam3) extends Meta AI's foundation segmentation family into Promptable Concept Segmentation (PCS) across both images and video. Instead of clicking individual pixels on every person or prop in a crowded shot, VFX artists can type a short concept phrase such as "yellow taxi", "actor in red jacket", or "handheld microphone", and SAM 3 detects, masks, and tracks every matching instance cleanly across moving frames even through occlusions.

Studio availability. Try the public Hugging Face demo below; Meta SAM 3 is not part of a paid AI FILMS Studio plan (use BEN2 Background Remover or WAN 3.0 in the video workspace for example instead).

Why Promptable Concept Segmentation Changes VFX Rotoscoping

In SAM 2, isolating six background extras or three moving vehicles required clicking manual point prompts on each individual object on frame 1, and any new object entering the shot on frame 40 went untracked.

SAM 3 unifies open vocabulary detection and temporal mask tracking inside one architecture:

  1. Text and visual concept prompts. Supply a short noun phrase or an example visual crop to segment every matching instance simultaneously across the entire clip, automatically picking up newly entering objects mid shot.
  2. Unified detector and memory tracker. SAM 3 pairs a shared vision backbone with a concept presence head and temporal memory bank, preventing mask ID swaps when actors cross paths behind foreground columns.
  3. Production ready alpha masks for compositing. Exported binary or soft matte sequences plug directly into Nuke, After Effects, DaVinci Resolve Fusion, MatAnyone 2, or LBM Relighting for clean background replacement and color grading isolation.
Meta SAM 3 SA-Co concept segmentation dataset spanning dense multi object scenes

Dense open vocabulary concept masks trained on the SA-Co dataset for SAM 3. Credit: Meta AI Research.

Trained on Meta's large scale SA-Co concept dataset, SAM 3 handles nuanced film concepts including wardrobe items, vehicle parts, hand props, and lighting fixtures without custom fine tuning.

Colorists can pull instant secondary isolation mattes for faces, skies, or hero cars across a 200 frame take in seconds.

Try Meta SAM 3 Video Segmentation Live in the Browser

Test SAM 3 text prompted video segmentation directly in the embedded Hugging Face Space below (merve-sam3-video-segmentation.hf.space).

In fast sports and action sequences, SAM 3 holds tight edge boundaries around motion blurred limbs and tracks occluded subjects reliably as they re emerge behind other performers.

Pairing SAM 3 coarse masks with **MatAnyone 2** or **VideoMaMa** refines fine flying hair strands and semi transparent smoke for theatrical compositing.

Meta SAM 3 tracking fast moving sports players cleanly across video frames

Multi instance temporal tracking on fast moving subjects in SAM 3. Credit: Meta AI Research.

Hardware and Performance Matrix

Mode Prompt Inputs VRAM Needed Target Hardware Best VFX Use Case
SAM 3 Video Concept Tracking Text phrases + optional boxes/points 8 GB – 16 GB VRAM (BF16) RTX 3080 / RTX 4070 / RTX 4090 Multi actor rotoscoping, secondary color mattes, prop isolation
SAM 3 Image Concept Segmentation Text phrases + visual exemplars 6 GB – 10 GB VRAM Consumer laptop GPUs Keyframe matte extraction and inpainting mask prep

Local Setup with Python

git clone https://github.com/facebookresearch/sam3.git
cd sam3
pip install -e .
import torch
from sam3.model_builder import build_sam3_video_predictor

predictor = build_sam3_video_predictor().to("cuda")
session = predictor.start_session(video_path="street_chase_1080p.mp4")

predictor.add_prompt(session, text="motorcycle rider in black helmet")
for frame_idx, masks in predictor.propagate_in_video(session):
    pass  # Export per frame alpha matte PNGs for DaVinci Resolve or Nuke

Filmmaker Production Playbook

  1. Use SAM 3 for garbage mattes, then refine hair with alpha matting. SAM 3 excels at rapid semantic isolation across crowds; feed its binary mask into MatAnyone 2 or VideoMaMa for soft feathered hair edges.
  2. Isolate lighting zones for color grading. Pull clean "sky", "actor face", and "wet street" matte passes to grade foreground talent and background environments independently without hand drawing power windows.
  3. Harmonize composites with LBM. Drop your SAM 3 cutout onto a new plate and run LBM Relighting in 1 step so rim light and shadows match.

Related Reading


Sources