EditorNodesPricingBlog

HeartMuLa: Open Source Apache 2.0 Music AI

November 20, 2026
HeartMuLa: Open Source Apache 2.0 Music AI

HeartMuLa Team — heartlib Apache 2.0 music foundation model benchmarks

Share this post:

HeartMuLa: Open Source Apache 2.0 Music AI

HeartMuLa (HeartMuLa/heartlib, Apache 2.0 License) is an open source music foundation model suite combining HeartMuLa-oss-3B and HeartMuLa-oss-7B language models with HeartCodec, an ultra low frame rate neural audio codec, and built in lyric transcription tools. Designed for local lyrics to song composition, multilingual vocal synthesis, and controllable instrumental scoring, HeartMuLa gives independent filmmakers permissive commercial weights they can run on a single workstation GPU with zero subscription lock in.

Studio availability. Try the public Hugging Face demo below; HeartMuLa is not part of a paid AI FILMS Studio plan (use Suno in the music workspace for example instead).

Why HeartCodec and Apache 2.0 Matter for Indie Films

Commercial film distribution requires clean chain of title on every audio stem. Many popular open music checkpoints carry non commercial or territory restricted licenses that block festival screening and streaming platform delivery.

HeartMuLa solves both legal clearance and local compute cost:

  1. Permissive Apache 2.0 license across weights and code. Studios can generate temp scores, diegetic radio songs, and final instrumental beds locally and ship them in commercial projects with full Apache 2.0 rights.
  2. Ultra low bitrate HeartCodec tokenization. High frame rate audio codecs choke transformer context windows past 60 seconds. HeartCodec compresses 48 kHz music into compact acoustic tokens so HeartMuLa-oss-3B generates full multi minute songs rapidly on 12 GB – 16 GB GPUs.
  3. Structured lyric section tags and transcription. Filmmakers can format [Intro], [Verse], [Chorus], [Bridge], and [Outro] blocks alongside style descriptors, and run HeartMuLa's companion lyrics transcription module (HeartTranscriptor) to analyze reference songs before generating variations.
HeartMuLa architecture pairing HeartCodec low rate audio tokenizer with 3B and 7B music language models

HeartMuLa system overview integrating HeartCodec acoustic tokens, `HeartMuLa-oss` language models, and lyric alignment. Credit: HeartMuLa Team (Apache 2.0).

By compressing multi instrument stereo mixes into low frame rate HeartCodec tokens, the 3B checkpoint fits comfortably on consumer GPUs while retaining clean vocal phrasing and stereo drum transients.

Indie post teams can batch render ten scoring variations overnight on local hardware without paying per generation API fees.

Try HeartMuLa Live in the Browser

You can test HeartMuLa directly in the embedded Hugging Face Space below (mrfakename-heartmula.hf.space).

Hardware, Variants, and License Matrix

Model Variant Parameters VRAM Needed Core Strength License
HeartMuLa-oss-3B + HeartCodec 3B LM + codec 10 GB – 16 GB VRAM (BF16 / FP8) Fast consumer GPU scoring and multilingual vocals Apache 2.0
HeartMuLa-oss-7B + HeartCodec 7B LM + codec 20 GB – 24 GB VRAM (BF16) Deeper harmonic complexity and rich acoustic arrangement Apache 2.0

Parameter Reference Table

Parameter Recommended Value Musical Effect
--tags Comma separated genre, mood, and instruments Example: "cinematic, dark ambient, analog synth, slow cello, 80 bpm".
--lyrics Section tagged text or [Instrumental] Controls vocal timing or forces pure dialogue safe underscore.
--cfg_scale 1.5 – 2.5 Tightens style tag adherence without introducing metallic codec artifacts.
--temperature 0.85 – 1.0 Balances safe harmonic predictability against creative melodic turns.

Local Setup with heartlib

git clone https://github.com/HeartMuLa/heartlib.git
cd heartlib
pip install -e .

huggingface-cli download HeartMuLa/HeartMuLa-oss-3B --local-dir ./ckpt/HeartMuLa-oss-3B
huggingface-cli download HeartMuLa/HeartCodec-oss --local-dir ./ckpt/HeartCodec-oss
from heartlib import HeartMuLaGenPipeline
import torch

pipe = HeartMuLaGenPipeline.from_pretrained(
    "./ckpt",
    device=torch.device("cuda"),
    dtype=torch.bfloat16,
    version="3B",
)

pipe(
    {
        "lyrics": "[Intro]\n[Instrumental]\n[Bridge]\n[Outro]",
        "tags": "cinematic thriller underscore, pulsing low strings, distant brass swells, minor key, 90 bpm",
    },
    max_audio_length_ms=120_000,
    save_path="scene_09_thriller_bed.wav",
    cfg_scale=1.8,
    temperature=0.9,
)

Filmmaker Production Playbook

  1. Use HeartMuLa-oss-3B for fast Apache 2.0 temp tracks. Cut your rough assembly to clean 90 to 120 second instrumental beds generated locally, then refine with 7B or StepAudio-3-Music once picture locks.
  2. Separate diegetic songs from score. Write short [Verse] and [Chorus] lyric sheets for background diner radios or car stereos so you never face music clearance blocks at festivals.
  3. Normalize stems to -24 LKFS before mixing. Drop generated WAV files into your NLE and trim sub 40 Hz rumble so footsteps and dialogue stay crisp.

Related Reading


Sources