EditorNodesPricingBlog

Bark TTS, Voicebox & INSwapper: Open Source Voice & VFX

November 15, 2026
Bark TTS, Voicebox & INSwapper: Open Source Voice & VFX

Jamie Pine / Voicebox (MIT License)

Share this post:

Bark TTS, Voicebox & INSwapper: Open Source Voice & VFX

Bark TTS (suno/bark), Voicebox (jamiepine/voicebox), and INSwapper-128 (inswapper_128.onnx / inswapper-128) cover two specialized post production tasks on consumer hardware, expressive dialogue synthesis and frame by frame facial replacement. Released by Suno under the permissive MIT License, suno/bark is a transformer text to audio model (24000 Hz, 8 GB VRAM or <4 GB small mode) that generates multilingual dialogue alongside nonverbal vocal cues such as [laughs], [sighs], and [gasps]. Voicebox (also MIT License) packages 8 local TTS engines, Whisper speech to text, and a multi track Stories timeline editor into a native Rust and Tauri desktop studio. By contrast, inswapper-128 runs frame level ONNX video faceswapping in 4 GB of VRAM using insightface and onnxruntime-gpu, but carries a strict non commercial research only restriction under InsightFace's license policy.

Studio availability. Bark TTS (bark), Voicebox, and INSwapper-128 (inswapper-128) are not available in AI FILMS Studio. This article covers public open source and research releases for local deployment. To generate character voices, perform face swaps, or transfer motion in the Studio, open the voice workspace, the faceswap workspace, or the Motion Control page.

Suno Bark (bark): GPT Style Generative Text to Audio Under MIT

Unlike phoneme aligned text to speech pipelines that convert text into mel spectrograms at a fixed speaking rate, Bark (suno/bark) treats audio synthesis as an autoregressive language modeling problem built on EnCodec neural audio tokens.

  1. Three stage transformer hierarchy. Bark maps input text prompts to semantic tokens, predicts coarse acoustic tokens conditioned on speaker history prompts (v2/en_speaker_0 through v2/en_speaker_9 across 13+ languages), and decodes fine acoustic tokens into a 24000 Hz waveform (~5.5 GB checkpoint footprint, 8 GB VRAM in FP16).
  2. Paralinguistic and ambient performance cues. Because Bark generates full audio tokens rather than isolated phonemes, scriptwriters can embed inline direction tags including [laughs], [clears throat], [sighs], [gasps], ... hesitations, and ♪ musical phrases directly in the dialogue string.
  3. Commercial friendliness. Released under the MIT License, Bark outputs and checkpoints can be integrated into commercial indie film soundtracks, animatics, and game dialogue without royalty restrictions. Note that because Bark generates prosody autonomously, it does not accept an explicit playback speed multiplier.
Suno Bark transformer based text to audio architecture and prompt capabilities
Suno Bark synthesizes multilingual speech, paralinguistic cues, and ambient audio at 24 kHz under the MIT license. Credit: Suno AI (MIT).
Voicebox multi track Stories timeline editor and local voice cloning studio
Voicebox desktop studio showing voice profile management and multi track dialogue sequencing. Credit: Jamie Pine (MIT).

Voicebox: Native Tauri Local Voice Studio With 8 TTS Engines

While suno/bark runs as a Python library or Hugging Face AutoProcessor + BarkModel pipeline, Voicebox (jamiepine/voicebox) provides a unified desktop workstation and REST/MCP server (MIT License) for assembling multi character dialogue scenes locally on Windows (CUDA), macOS (Apple Silicon MLX/Metal), and Linux.

  • Eight integrated local TTS engines. Voicebox unifies Qwen3-TTS, Qwen CustomVoice, Qwen VoiceDesign, LuxTTS, Chatterbox Multilingual, Chatterbox Turbo (supporting [laugh], [sigh], and [gasp] tags), HumeAI TADA, and Kokoro across 23 languages.
  • Stories multi track timeline & DSP rack. Editors arrange multi speaker dialogue scenes on a timeline with automatic long script chunking, crossfades, pitch shifting, reverb, delay, chorus, and compression.
  • REST API and MCP agent integration. Every cloned or designed voice profile is exposed over a local HTTP API and the voicebox_speak MCP tool for automated storyboard table reads.

INSwapper-128 (inswapper-128): 4 GB VRAM ONNX Faceswap & License Boundary

In visual effects previsualization and stunt double tests, inswapper_128.onnx (inswapper-128, ~554 MB ONNX weight file) performs fast frame by frame facial identity replacement on video files using only 4 GB of VRAM.

  1. Detection and 5 point affine alignment. For each video frame decoded via OpenCV (cv2), InsightFace's buffalo_l detector (det_10g.onnx and 2d106det.onnx) locates the target actor's face and computes a 5 point similarity transform into a normalized 128x128 crop.
  2. ArcFace identity injection. An ArcFace recognition network extracts a 512 dimensional identity embedding from the single source_image_url portrait (face.jpg), which conditions the inswapper_128.onnx generator via CUDAExecutionProvider before pasting the swapped face back into the full resolution frame and muxing the original audio track with ffmpeg.
  3. Mandatory Non-Commercial Research License Restriction. Unlike suno/bark (MIT) and jamiepine/voicebox (MIT), inswapper-128 is strictly restricted to non commercial scientific research use only under InsightFace's policy (updated 2025-11-24). Community mirrors (such as CountFloyd/deepfake) are not authorized for commercial distribution, and any commercial film, advertising, or client production use requires a separate enterprise license directly from InsightFace (contact@insightface.ai). Filmmakers must also never use face replacement or voice cloning tools to impersonate real individuals without explicit written consent.

Hardware, VRAM & License Comparison Matrix

Model / Tool Model ID / File Primary Filmmaking Task Minimum VRAM Download Size License & Commercial Status
Suno Bark TTS bark (suno/bark) Expressive TTS + nonverbal cues (24000 Hz) 8 GB VRAM (<4 GB small mode) ~5.5 GB MIT License (Commercial OK)
Voicebox Studio jamiepine/voicebox Multi track voice cloning desktop app & REST API CPU / 4 to 8 GB VRAM / Apple MLX ~150 MB app + engine weights MIT License (Engine terms vary)
INSwapper-128 inswapper-128 (inswapper_128.onnx) Frame by frame ONNX video faceswap 4 GB VRAM (onnxruntime-gpu) ~554 MB (+ buffalo_l) NON-COMMERCIAL RESEARCH ONLY

Running Bark TTS and INSwapper-128 Locally in Python

To generate expressive scratch dialogue with suno/bark in FP16 on an 8 GB GPU using Hugging Face transformers, run the following Python script:

import torch
import scipy.io.wavfile as wavfile
from transformers import AutoProcessor, BarkModel

processor = AutoProcessor.from_pretrained("suno/bark")
model = BarkModel.from_pretrained(
    "suno/bark",
    torch_dtype=torch.float16,
).to("cuda")

# Optional: enable CPU offload on GPUs with 4 GB to 6 GB VRAM
# model.enable_cpu_offload()

inputs = processor(
    text="[clears throat] We lost the signal at sector four... [sighs] we have to go back in.",
    voice_preset="v2/en_speaker_6",
).to("cuda")

audio_array = model.generate(**inputs)
audio_np = audio_array.cpu().numpy().squeeze()
sample_rate = model.generation_config.sample_rate  # 24000 Hz

wavfile.write("bark_dialogue_take.wav", rate=sample_rate, data=audio_np)

For non commercial research testing of inswapper_128.onnx on a 4 GB CUDA GPU, install insightface>=0.7, onnxruntime-gpu>=1.16, and opencv-python-headless>=4.8:

import cv2
import insightface
from insightface.app import FaceAnalysis

# Non-commercial research use only (InsightFace policy 2025-11-24)
app = FaceAnalysis(name="buffalo_l", providers=["CUDAExecutionProvider", "CPUExecutionProvider"])
app.prepare(ctx_id=0, det_size=(640, 640))

swapper = insightface.model_zoo.get_model(
    "inswapper_128.onnx",
    providers=["CUDAExecutionProvider", "CPUExecutionProvider"],
)

source_img = cv2.imread("source_actor_portrait.jpg")
source_faces = app.get(source_img)
source_face = source_faces[0]

cap = cv2.VideoCapture("input_plate.mp4")
fps = cap.get(cv2.CAP_PROP_FPS)
w = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
h = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
writer = cv2.VideoWriter("swapped_research_preview.mp4", cv2.VideoWriter_fourcc(*"mp4v"), fps, (w, h))

while True:
    ret, frame = cap.read()
    if not ret:
        break
    target_faces = app.get(frame)
    for target_face in target_faces:
        frame = swapper.get(frame, target_face, source_face, paste_back=True)
    writer.write(frame)

cap.release()
writer.release()

Voice Design, Face Replacement & Motion in AI FILMS Studio

For related local audio and dubbing pipelines, read our guides on Open Source Voice Cloning with Qwen3-TTS, F5-TTS, and Kokoro, AudioLDM2, Stable Audio Open, and MusicGen local Foley, and LatentSync 1.6 and MuseTalk 1.5 lip sync dubbing.

To produce commercial ready voiceovers, character replacements, and motion controlled shots in the cloud without local ONNX setup or non commercial model restrictions, open the voice workspace, the faceswap workspace, the Motion Control page, or connect voice, lip sync, and video nodes inside the Nodes Graph Editor.


Sources