
Jamie Pine / Voicebox (MIT License)
Share this post:
Bark TTS, Voicebox & INSwapper: Open Source Voice & VFX
Bark TTS (suno/bark), Voicebox (jamiepine/voicebox), and INSwapper-128 (inswapper_128.onnx / inswapper-128) cover two specialized post production tasks on consumer hardware, expressive dialogue synthesis and frame by frame facial replacement. Released by Suno under the permissive MIT License, suno/bark is a transformer text to audio model (24000 Hz, 8 GB VRAM or <4 GB small mode) that generates multilingual dialogue alongside nonverbal vocal cues such as [laughs], [sighs], and [gasps]. Voicebox (also MIT License) packages 8 local TTS engines, Whisper speech to text, and a multi track Stories timeline editor into a native Rust and Tauri desktop studio. By contrast, inswapper-128 runs frame level ONNX video faceswapping in 4 GB of VRAM using insightface and onnxruntime-gpu, but carries a strict non commercial research only restriction under InsightFace's license policy.
Studio availability. Bark TTS (
bark), Voicebox, andINSwapper-128(inswapper-128) are not available in AI FILMS Studio. This article covers public open source and research releases for local deployment. To generate character voices, perform face swaps, or transfer motion in the Studio, open the voice workspace, the faceswap workspace, or the Motion Control page.
Suno Bark (bark): GPT Style Generative Text to Audio Under MIT
Unlike phoneme aligned text to speech pipelines that convert text into mel spectrograms at a fixed speaking rate, Bark (suno/bark) treats audio synthesis as an autoregressive language modeling problem built on EnCodec neural audio tokens.
- Three stage transformer hierarchy. Bark maps input text prompts to semantic tokens, predicts coarse acoustic tokens conditioned on speaker history prompts (
v2/en_speaker_0throughv2/en_speaker_9across 13+ languages), and decodes fine acoustic tokens into a24000 Hzwaveform (~5.5 GB checkpoint footprint, 8 GB VRAM inFP16). - Paralinguistic and ambient performance cues. Because Bark generates full audio tokens rather than isolated phonemes, scriptwriters can embed inline direction tags including
[laughs],[clears throat],[sighs],[gasps],...hesitations, and♪musical phrases directly in the dialogue string. - Commercial friendliness. Released under the MIT License, Bark outputs and checkpoints can be integrated into commercial indie film soundtracks, animatics, and game dialogue without royalty restrictions. Note that because Bark generates prosody autonomously, it does not accept an explicit playback speed multiplier.
Voicebox: Native Tauri Local Voice Studio With 8 TTS Engines
While suno/bark runs as a Python library or Hugging Face AutoProcessor + BarkModel pipeline, Voicebox (jamiepine/voicebox) provides a unified desktop workstation and REST/MCP server (MIT License) for assembling multi character dialogue scenes locally on Windows (CUDA), macOS (Apple Silicon MLX/Metal), and Linux.
- Eight integrated local TTS engines. Voicebox unifies
Qwen3-TTS,Qwen CustomVoice,Qwen VoiceDesign,LuxTTS,Chatterbox Multilingual,Chatterbox Turbo(supporting[laugh],[sigh], and[gasp]tags),HumeAI TADA, andKokoroacross 23 languages. - Stories multi track timeline & DSP rack. Editors arrange multi speaker dialogue scenes on a timeline with automatic long script chunking, crossfades, pitch shifting, reverb, delay, chorus, and compression.
- REST API and MCP agent integration. Every cloned or designed voice profile is exposed over a local HTTP API and the
voicebox_speakMCP tool for automated storyboard table reads.
INSwapper-128 (inswapper-128): 4 GB VRAM ONNX Faceswap & License Boundary
In visual effects previsualization and stunt double tests, inswapper_128.onnx (inswapper-128, ~554 MB ONNX weight file) performs fast frame by frame facial identity replacement on video files using only 4 GB of VRAM.
- Detection and 5 point affine alignment. For each video frame decoded via OpenCV (
cv2), InsightFace'sbuffalo_ldetector (det_10g.onnxand2d106det.onnx) locates the target actor's face and computes a 5 point similarity transform into a normalized128x128crop. - ArcFace identity injection. An ArcFace recognition network extracts a 512 dimensional identity embedding from the single
source_image_urlportrait (face.jpg), which conditions theinswapper_128.onnxgenerator viaCUDAExecutionProviderbefore pasting the swapped face back into the full resolution frame and muxing the original audio track withffmpeg. - Mandatory Non-Commercial Research License Restriction. Unlike
suno/bark(MIT) andjamiepine/voicebox(MIT),inswapper-128is strictly restricted to non commercial scientific research use only under InsightFace's policy (updated 2025-11-24). Community mirrors (such asCountFloyd/deepfake) are not authorized for commercial distribution, and any commercial film, advertising, or client production use requires a separate enterprise license directly from InsightFace (contact@insightface.ai). Filmmakers must also never use face replacement or voice cloning tools to impersonate real individuals without explicit written consent.
Hardware, VRAM & License Comparison Matrix
| Model / Tool | Model ID / File | Primary Filmmaking Task | Minimum VRAM | Download Size | License & Commercial Status |
|---|---|---|---|---|---|
| Suno Bark TTS | bark (suno/bark) |
Expressive TTS + nonverbal cues (24000 Hz) |
8 GB VRAM (<4 GB small mode) |
~5.5 GB | MIT License (Commercial OK) |
| Voicebox Studio | jamiepine/voicebox |
Multi track voice cloning desktop app & REST API | CPU / 4 to 8 GB VRAM / Apple MLX | ~150 MB app + engine weights | MIT License (Engine terms vary) |
| INSwapper-128 | inswapper-128 (inswapper_128.onnx) |
Frame by frame ONNX video faceswap | 4 GB VRAM (onnxruntime-gpu) |
~554 MB (+ buffalo_l) |
NON-COMMERCIAL RESEARCH ONLY |
Running Bark TTS and INSwapper-128 Locally in Python
To generate expressive scratch dialogue with suno/bark in FP16 on an 8 GB GPU using Hugging Face transformers, run the following Python script:
import torch
import scipy.io.wavfile as wavfile
from transformers import AutoProcessor, BarkModel
processor = AutoProcessor.from_pretrained("suno/bark")
model = BarkModel.from_pretrained(
"suno/bark",
torch_dtype=torch.float16,
).to("cuda")
# Optional: enable CPU offload on GPUs with 4 GB to 6 GB VRAM
# model.enable_cpu_offload()
inputs = processor(
text="[clears throat] We lost the signal at sector four... [sighs] we have to go back in.",
voice_preset="v2/en_speaker_6",
).to("cuda")
audio_array = model.generate(**inputs)
audio_np = audio_array.cpu().numpy().squeeze()
sample_rate = model.generation_config.sample_rate # 24000 Hz
wavfile.write("bark_dialogue_take.wav", rate=sample_rate, data=audio_np)
For non commercial research testing of inswapper_128.onnx on a 4 GB CUDA GPU, install insightface>=0.7, onnxruntime-gpu>=1.16, and opencv-python-headless>=4.8:
import cv2
import insightface
from insightface.app import FaceAnalysis
# Non-commercial research use only (InsightFace policy 2025-11-24)
app = FaceAnalysis(name="buffalo_l", providers=["CUDAExecutionProvider", "CPUExecutionProvider"])
app.prepare(ctx_id=0, det_size=(640, 640))
swapper = insightface.model_zoo.get_model(
"inswapper_128.onnx",
providers=["CUDAExecutionProvider", "CPUExecutionProvider"],
)
source_img = cv2.imread("source_actor_portrait.jpg")
source_faces = app.get(source_img)
source_face = source_faces[0]
cap = cv2.VideoCapture("input_plate.mp4")
fps = cap.get(cv2.CAP_PROP_FPS)
w = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
h = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
writer = cv2.VideoWriter("swapped_research_preview.mp4", cv2.VideoWriter_fourcc(*"mp4v"), fps, (w, h))
while True:
ret, frame = cap.read()
if not ret:
break
target_faces = app.get(frame)
for target_face in target_faces:
frame = swapper.get(frame, target_face, source_face, paste_back=True)
writer.write(frame)
cap.release()
writer.release()
Voice Design, Face Replacement & Motion in AI FILMS Studio
For related local audio and dubbing pipelines, read our guides on Open Source Voice Cloning with Qwen3-TTS, F5-TTS, and Kokoro, AudioLDM2, Stable Audio Open, and MusicGen local Foley, and LatentSync 1.6 and MuseTalk 1.5 lip sync dubbing.
To produce commercial ready voiceovers, character replacements, and motion controlled shots in the cloud without local ONNX setup or non commercial model restrictions, open the voice workspace, the faceswap workspace, the Motion Control page, or connect voice, lip sync, and video nodes inside the Nodes Graph Editor.
Sources
Continue Reading
Video & LipSync
- Video Generator
- Text to Video
- Image to Video
- Start-End Frame to Video
- Draw to Video
- Motion Control
- Video Enhancer
- Video Upscaler
- Video Extend
- Video to Video LipSync
- Audio to Video LipSync
- Image to Video LipSync
- Video FaceSwap
- Seedance 2.5
- WAN 3.0
- Seedance 2
- FLUX 3
- Minimax H3
- Vidu Q3 Pro
- Grok Imagine Video 1.5
- Gemini Omni
- Google Veo 3.1
- Kling 3.0 Pro
- Luma Ray 3.2
- LTX 2.3
- Happy Horse 1.1
- Kling 3.0 Motion
- ByteDance Upscaler
- InfiniteTalk
- InsightFace
Image & Edit
- AI Character
- AI Actor
- Art Generator
- Text to Image
- Image to Image
- LoRA Image to Image
- Draw to Edit
- Image LoRA Training
- Remove Background
- Image Enhancer
- MidJourney 8.0
- Krea 2
- OpenAI GPT Image 2.0
- Nano Banana Pro Ultra
- Nano Banana 2 Lite
- Nano Banana Pro
- Minimax Image
- Clarity Pro
- Z-Image Turbo LoRA
- Krea 2 Turbo LoRA
- Z-Image LoRA Trainer
- Qwen Image LoRA Trainer
- Background Erase Network 2 (BEN2)


