EditorNodesPricingBlog

Fun-CosyVoice 3.0: Open Source Multilingual Voice AI

November 20, 2026
Fun-CosyVoice 3.0: Open Source Multilingual Voice AI

Alibaba Tongyi Lab — CosyVoice 3.0 zero shot multilingual speech generation architecture

Share this post:

Fun-CosyVoice 3.0: Open Source Multilingual Voice AI

Fun-CosyVoice 3.0 (FunAudioLLM/Fun-CosyVoice3-0.5B, Apache 2.0 License) is Alibaba Tongyi Lab's compact 0.5 billion parameter zero shot multilingual speech foundation model engineered for cross lingual film dubbing, dialect faithful character voices, and ultra low latency streaming synthesis. Built on a refined supervised speech tokenizer and flow matching acoustic decoder, Fun-CosyVoice3-0.5B runs smoothly in under 4 GB of VRAM while preserving natural speaker identity across nine major languages and regional accents.

Studio availability. Try the public Hugging Face demo below; Fun-CosyVoice 3.0 is not part of a paid AI FILMS Studio plan (use the AI Voice Generator in the voice workspace for example instead).

Why Compact 0.5B Apache 2.0 Models Win in Post Production

Heavy multi billion parameter speech models often require dedicated 16 GB GPUs just to generate scratch dialogue for an animatic, competing for VRAM with local video and image generators on the same workstation.

Fun-CosyVoice 3.0 keeps resource overhead tiny while improving international localization accuracy:

  1. Cross lingual zero shot voice cloning. Supply a 5 second English or Spanish actor clip and synthesize fluent dialogue in Chinese, Japanese, Korean, German, French, Italian, or Russian while maintaining consistent vocal timbre.
  2. Instruction steered prosody and dialects. Guide speaking pace, whispering, laughter, or regional accents directly via short natural language instructions without training speaker LoRAs.
  3. Bi directional streaming synthesis. First packet latency drops into interactive ranges so directors and voice actors can audition alternate script lines in real time on a laptop GPU.
  4. Apache 2.0 commercial clearance. Both code and Fun-CosyVoice3-0.5B weights carry the permissive Apache 2.0 license.
CosyVoice 3.0 supervised speech tokenizer and flow matching acoustic decoder pipeline

CosyVoice 3.0 speech tokenization and flow matching waveform reconstruction pipeline. Credit: Alibaba Tongyi Lab (Apache 2.0).

Scaling multilingual pretraining data sharply reduces pronunciation errors on technical film terminology, foreign character names, and rapid code switched dialogue.

Because the model occupies under 4 GB of VRAM, editors can keep it loaded alongside local video editing tools without swapping weights.

Try Fun-CosyVoice 3.0 Live in the Browser

Test Fun-CosyVoice3-0.5B directly in Alibaba Tongyi Lab's official Hugging Face Space below (funaudiollm-fun-cosyvoice3-0-5b.hf.space).

Hardware and Specification Matrix

Metric Fun-CosyVoice3-0.5B Spec Production Benefit
Parameters 0.5B Runs simultaneously alongside local video/image models
VRAM Needed 2.5 GB – 4 GB VRAM (FP16) Fits on consumer laptops and modest GPUs
Streaming Mode Supported (low latency chunking) Instant line auditions during editorial sessions
License Apache 2.0 Commercial film, trailer, and localization use permitted

Local Setup with Python

git clone --recursive https://github.com/FunAudioLLM/CosyVoice.git
cd CosyVoice
pip install -r requirements.txt

huggingface-cli download FunAudioLLM/Fun-CosyVoice3-0.5B --local-dir ./pretrained_models/Fun-CosyVoice3-0.5B
import torchaudio
from cosyvoice.cli.cosyvoice import AutoModel

model = AutoModel(model_dir="./pretrained_models/Fun-CosyVoice3-0.5B")

for i, out in enumerate(
    model.inference_zero_shot(
        tts_text="The lighthouse beam cut through the fog three seconds before the siren sounded.",
        prompt_text="Transcript of the five second reference clip.",
        prompt_wav="./cast/narrator_ref_24k.wav",
        stream=False,
    )
):
    torchaudio.save(f"narrator_take_{i}.wav", out["tts_speech"], model.sample_rate)

Filmmaker Production Playbook

  1. Always match prompt_text verbatim to your reference WAV. Providing an accurate transcript of the 5 second actor clip improves consonant crispness and cross lingual accent fidelity.
  2. Use for fast international trailer localization. Generate English, Spanish, French, and Japanese teaser voiceovers from one approved narrator clip (with written voice talent clearance) in minutes.
  3. Pair with IndexTTS-2.5 for tight lip sync windows. Use Fun-CosyVoice 3.0 for fast narrative voiceovers and IndexTTS-2.5 whenever a line must hit an exact millisecond picture cut.

Related Reading


Sources