
Bilibili IndexTeam — IndexTTS-2.5 open source duration and emotion controlled speech synthesis
Share this post:
IndexTTS-2.5 & 2.0: Open Source Duration & Emotion TTS
IndexTTS-2.5 and IndexTTS-2.0 (index-tts/index-tts, Bilibili IndexTeam) solve the single hardest technical problem in AI film dubbing and ADR: exact timing alignment with independent emotional acting. Unlike standard autoregressive voice cloners whose pacing varies unpredictably on every take, IndexTTS lets sound editors specify the exact target duration of a translated line so it lands cleanly inside an actor's mouth flap window, while independently controlling speaker timbre (from Reference A) and emotional delivery (from Reference B or text emotion vectors).
Studio availability. Try both public Hugging Face demos below; IndexTTS-2.5 is not part of a paid AI FILMS Studio plan (use the AI Voice Generator in the voice workspace for example instead).
Why Duration Control and Emotion Disentanglement Matter for Dubbing
When localizing a 4.2 second dramatic line from English into Spanish or Japanese, standard TTS models frequently output a 5.6 second read that runs past the shot cut, forcing editors to time stretch the audio until the actor sounds robotic. Worse, cloning a calm studio reference clip makes it nearly impossible to generate a panicked scream without losing the actor's identity.
IndexTTS-2.0 and 2.5 tackle both filmmaking pain points directly:
- Exact token level duration control. Specify the exact target length (in seconds or semantic token count) for dubbing mode, or switch to free duration mode when recording natural voiceover narration.
- Disentangled timbre and emotion conditioning. Feed a neutral 6 second reference clip for speaker identity, and supply a separate angry, weeping, or whispering reference clip (or an 8 dimensional emotion vector / text emotion prompt) to drive the acting performance without swapping the actor's voice.
- Natural breath pacing in IndexTTS-2.5. Version 2.5 refines prosody smoothness, pronunciation stability on proper nouns, and bilingual code switching for international film trailers.
IndexTTS-2 dual reference design decoupling speaker timbre from emotional delivery and target duration. Credit: Bilibili IndexTeam.
By decoupling timbre from emotion, a director can record a single neutral voice sheet with an actor and later direct whispered, furious, or exhausted ADR pickups without rebooking the vocal booth.
Exported WAV dialogue stems lock directly into lip sync engines like **LongCat-Video-Avatar 1.6**.
Try IndexTTS-2.5 Live in the Browser
Test IndexTTS-2.5 directly in the official Hugging Face Space below (indexteam-indextts-2-5-demo.hf.space).
Try IndexTTS-2.0 Live in the Browser
You can also test IndexTTS-2.0 (856 likes on Hugging Face) directly below (indexteam-indextts-2-demo.hf.space).
Hardware and Parameter Matrix
| Parameter / Tier | Recommended Value | Filmmaking Role |
|---|---|---|
| VRAM Footprint | 6 GB – 12 GB VRAM (FP16) |
Runs comfortably on consumer editing workstations |
spk_audio_prompt |
5–10s clean neutral WAV | Locks character vocal timbre across scenes |
emo_audio_prompt / emo_vector |
Separate acting reference or 8 emotion sliders | Drives anger, fear, joy, sadness, or calm independently of speaker identity |
target_duration |
Exact shot window (seconds) | Fits translated dubbing lines inside existing picture cuts |
Local Setup with Python
git clone https://github.com/index-tts/index-tts.git
cd index-tts
pip install -e .
huggingface-cli download IndexTeam/IndexTTS-2 --local-dir ./checkpoints
from indextts.infer_v2 import IndexTTS2
tts = IndexTTS2(
cfg_path="checkpoints/config.yaml",
model_dir="checkpoints",
use_fp16=True,
)
tts.infer(
spk_audio_prompt="cast_detective_neutral.wav",
emo_audio_prompt="acting_ref_urgent_whisper.wav",
text="Stay behind the pillar. Do not move until the headlights pass.",
output_path="scene_14_adr_take02.wav",
emo_alpha=0.75,
)
Filmmaker Production Playbook
- Record a 30 second neutral cast voice bank on set. Capture clean, dry room reads for each principal actor (with written performer consent) so IndexTTS can cover minor script rewrites and localized trailers later.
- Use
emo_alphabetween0.6and0.85. Pushing emotion strength to1.0can occasionally bleed vocal pitch from the emotion reference clip into your speaker identity. - Compare with MIT licensed alternatives. Pair IndexTTS-2.5 with Ming-3.0-tts and Fun-CosyVoice 3.0 when building multilingual dubbing pipelines.
Related Reading
- Fun-CosyVoice 3.0: Open Source Multilingual Voice Cloning
- Ming-3.0-tts: Open Source Voice Design & Cloning
- ZONOS3-Mini & Tiny-Voca: Open Source Voice Cloning TTS
- LongCat-Video-Avatar 1.6: Open Source 10+ Min Lip Sync
Sources
Continue Reading
Video & LipSync
- Video Generator
- Text to Video
- Image to Video
- Start-End Frame to Video
- Draw to Video
- Motion Control
- Video Enhancer
- Video Upscaler
- Video Extend
- Video to Video LipSync
- Audio to Video LipSync
- Image to Video LipSync
- Video FaceSwap
- Seedance 2.5
- WAN 3.0
- Seedance 2
- FLUX 3
- Minimax H3
- Vidu Q3 Pro
- Grok Imagine Video 1.5
- Gemini Omni
- Google Veo 3.1
- Kling 3.0 Pro
- Luma Ray 3.2
- LTX 2.3
- Happy Horse 1.1
- Kling 3.0 Motion
- ByteDance Upscaler
- InfiniteTalk
- InsightFace
Image & Edit
- AI Character
- AI Actor
- Art Generator
- Text to Image
- Image to Image
- LoRA Image to Image
- Draw to Edit
- Image LoRA Training
- Remove Background
- Image Enhancer
- MidJourney 8.0
- Krea 2
- OpenAI GPT Image 2.0
- Nano Banana Pro Ultra
- Nano Banana 2 Lite
- Nano Banana Pro
- Minimax Image
- Clarity Pro
- Z-Image Turbo LoRA
- Krea 2 Turbo LoRA
- Z-Image LoRA Trainer
- Qwen Image LoRA Trainer
- Background Erase Network 2 (BEN2)


