EditorNodesPricingBlog

Haru-4B: Open Source Full Duplex Speech Dialogue AI

November 22, 2026
Haru-4B: Open Source Full Duplex Speech Dialogue AI

AI FILMS architecture diagram built from higurashi-ai Haru-4B (Apache 2.0) specs

Share this post:

Haru-4B: Open Source Full Duplex Speech Dialogue AI

Turn based voice pipelines kill the natural rhythm of dramatic dialogue. A standard ASR, LLM, and TTS chain forces one speaker to finish completely before the other starts reacting, stripping away overlapping banter, interruptions, and quick breaths. Haru-4B (higurashi-ai/Haru-4B), released in November 2026 under the Apache 2.0 license, is a native full duplex spoken dialogue foundation model running at 7.5 Hz. Every 133 milliseconds, Haru-4B listens to a frame of user speech and emits a frame of its own 24 kHz voice and text simultaneously across 9 languages. You can test this open source model free in your browser below with zero login.

Studio availability. Try the public Hugging Face demo below; Haru-4B is not part of a paid AI FILMS Studio plan (use ElevenLabs Eleven V3 Text to Speech in the audio workspace for example instead).

Try Haru-4B Live

Test simultaneous full duplex character dialogue and custom voice conditioning directly in the public Hugging Face Space below.

Architecture and Technical Design

Haru-4B eliminates separate audio transformers and discrete audio codebook tokens. Its backbone is a 32 layer Qwen3.5-4B text transformer (hidden size 2,560) paired with a causal VibeVoice acoustic tokenizer operating at 24 kHz (3,200 samples per 133 ms frame).

Latency chart comparing turn based cascaded voice pipelines against Haru-4B 133 ms full duplex ticks

Interaction step latency comparison based on higurashi-ai Haru-4B 7.5 Hz frame specs

At every 133 ms tick, the model sums three embeddings into a single transformer position: the user's incoming 64 dimensional continuous speech latent, the model's previous speech latent, and the model's previous text token. Listening and speaking never block each other.

To decode continuous speech without multi step diffusion lag, a compact 4 layer MLP audio head uses MeanFlow sampling to generate the next 64 dimensional audio latent in 1 sampling step (default guidance scale 1.5). Character persona is controlled via a structured text card, while voice timbre comes from a clean 3 second to 3 minute reference recording.

The model supports English, Spanish, Portuguese, German, French, Chinese, Japanese, Russian, and Korean. Official code ships with a PyTorch web server, a native macOS MLX app for Apple Silicon laptops, and a vLLM-Omni serving plugin.

Hardware, VRAM, and License Matrix

Deployment Option Precision / Runtime Typical Hardware Frame Rate / Latency License Intended Use
PyTorch Server (haru.server) BF16 (model.safetensors) NVIDIA GPU (~12 GB VRAM) 7.5 Hz (133 ms/frame) Apache 2.0 Live interactive character reads
Native macOS MLX App Apple Silicon MLX M series Mac (16+ GB unified RAM) Real time local duplex Apache 2.0 Portable on set table reads
vLLM-Omni Plugin BF16 serving Data center GPU Multi session streaming Apache 2.0 Interactive narrative installations

All weights (model.safetensors, config, and tokenizer) ship under Apache 2.0. Haru-4B includes no bundled voice clips, so teams supply licensed actor recordings.

Parameter Reference

Parameter Default / Range Effect on Dialogue Delivery
Frame rate 7.5 Hz (133 ms) Fixed full duplex tick covering 3,200 samples at 24 kHz
guidance_scale 1.5 (range 1.0 to 2.0) Balances natural vocal prosody against tight character adherence
Reference audio clip 3 s to 3 min clean mono/stereo Sets target character voice timbre (single speaker only)
Sampling steps (MeanFlow) 1 Single step continuous 64D latent generation
Audio hardware Wired mic + wired headphones Avoids 100+ ms Bluetooth round trip latency

Local Python and CLI Setup

Clone the official repository, launch the local duplex server on port 8998, or drive a live DuplexSession directly in Python.

git clone https://github.com/higurashi-ai/Haru && cd Haru
pip install -e ./haru
python -m haru.server --hf-repo higurashi-ai/Haru-4B
from haru import DuplexSession, HaruModel
from haru.audio import load_audio, prepare_reference

model = HaruModel.from_pretrained("higurashi-ai/Haru-4B")
voice = prepare_reference(load_audio("actor_voice_ref.wav"))

character_card = "Name: Elena\nRole: Exhausted night shift detective questioning a witness."
session = DuplexSession(model, system_prompt=character_card, reference=voice)

frame = session.start()
while not frame.done:
    # Pass 3200 samples (133 ms @ 24 kHz) of live or recorded user audio
    frame = session.step(user_audio_chunk)

Filmmaker Production Playbook

  • Run live improvised scene rehearsals with AI scene partners. Writers and directors can load a character card and a licensed temp voice clip, put on wired headphones, and improvise overlapping dialogue directly against Haru-4B to test pacing before casting.
  • Record clean voice references with zero room echo. Because Haru-4B copies acoustic room tone from the reference clip, pass raw actor samples through Inworld Voice Isolator before calling prepare_reference.
  • Keep scripted line delivery separate from improvisation. The authors note that Haru-4B is a pretrained base model built for natural conversational rhythm rather than strict verbatim reading. For exact script locked ADR lines, pair it with IndexTTS-2.5 or ElevenLabs Eleven V3 Text to Speech in AI FILMS Studio.
  • Always use wired headphones during live captures. Speaker bleed back into the microphone confuses full duplex listening, and Bluetooth headsets add 100 ms or more of lag.

Related Reading


Sources

  1. higurashi-ai Haru-4B Model Hub on Hugging Face
  2. Haru Official GitHub Repository (PyTorch, MLX, vLLM-Omni)
  3. Haru-4B Technical Blog Post
  4. Haru-4B Interactive Hugging Face Space