
StepFun AI — StepAudio architecture overview
Share this post:
StepAudio-3-Music: Open Source 48 kHz Film Score AI
StepAudio-3-Music (stepfun-ai/StepAudio-3-Music) is StepFun AI's long form open music foundation model engineered to compose studio grade 48 kHz audio tracks up to 5.5 minutes long across full vocal songs, orchestral instrumentals, vocal accompaniments, and cover arrangements. Its standout innovation for film composers is ABC-CoT (ABC Notation Chain of Thought), which explicitly plans key signatures, tempo shifts, chord progressions, and recurring motifs in symbolic ABC notation before synthesizing high fidelity 48 kHz audio.
Studio availability. Try the public Hugging Face demo below; StepAudio-3-Music is not part of a paid AI FILMS Studio plan (use Suno in the music workspace for example instead).
Why ABC Notation Planning Matters for Film Scoring
Most text to music generators wander structurally after 45 seconds. A tension cue starts with a tight cello ostinato, loses its tempo at 1:10, and drifts into unrelated chord changes before the scene climax arrives.
StepAudio-3-Music fixes structural drift with three composer focused capabilities:
- Symbolic ABC-CoT structural planning. Before rendering acoustic tokens, the model drafts an explicit symbolic ABC score mapping intro bars, tempo, harmonic modulation, and thematic return. Film editors can steer exact musical structure rather than rolling the dice on a vague mood tag.
- Native 48 kHz broadcast sample rate up to 5.5 minutes. Standard 48 kHz output drops directly onto DaVinci Resolve, Premiere Pro, or Avid timelines with zero sample rate conversion artifacts, and 5.5 minute maximum duration covers an entire dramatic sequence or end credit crawl in one continuous take.
- Four production modes in one engine. StepAudio-3-Music handles text to full song, pure orchestral or electronic instrumentals, vocal to accompaniment backing tracks, and style cover transformations.
StepAudio multi stage tokenization and synthesis pipeline supporting long form audio coherence. Credit: StepFun AI.
Separating symbolic musical planning from acoustic waveform decoding lets the model hold steady tempo and recurring leitmotifs across multi minute cues.
Directors can request a quiet solo piano opening that swells into brass percussion at the two minute mark without losing rhythmic lock.
Try StepAudio-3-Music Live in the Browser
You can audition StepAudio-3-Music directly in StepFun AI's official Hugging Face Space below (stepfun-ai-stepaudio-3-music.hf.space).
Preference alignment training sharpens stereo imaging, acoustic instrument realism, and vocal intelligibility across genres ranging from chamber strings to synthwave trailers.
Exported 48 kHz WAV cues pair cleanly with Foley tracks from **LiveFoley** and dialogue from **Ming-3.0-tts** or **IndexTTS-2.5**.
Post training alignment workflow improving acoustic clarity and rhythmic consistency. Credit: StepFun AI.
Capabilities and Spec Matrix
| Feature | StepAudio-3-Music Spec | Filmmaking Benefit |
|---|---|---|
| Max Duration | Up to 5.5 minutes continuous |
Covers full scenes and end credits without manual loop splicing |
| Sample Rate | Native 48 kHz stereo |
Matches standard 48 kHz film and broadcast post timelines |
| Structural Control | ABC-CoT symbolic notation planning |
Preserves key, meter, and thematic motifs across long cues |
| Generation Modes | Full song, instrumental, accompaniment, cover | Supports temp scoring, diegetic songs, and remix variations |
Parameter Reference Table
| Control | Recommended Setting | Musical Effect |
|---|---|---|
mode |
Instrumental or Full Song |
Use Instrumental for dialogue underscore; Full Song for needle drops and credits. |
abc_cot |
Enabled | Drafts bar level harmonic structure before rendering audio tokens. |
duration_sec |
60 to 330 seconds |
Matches scene edit length directly in seconds. |
style_prompt |
Instrumentation + BPM + key + mood | Explicit tempo and instrumentation (e.g., "92 BPM, D minor, muted cello ostinato, sparse felt piano") yields tighter cues. |
Filmmaker Production Playbook
- Score directly to your picture BPM. Calculate the cut rhythm of your sequence (for example,
96 BPMor120 BPM) and state the exact BPM and key signature in your style prompt so musical hits land cleanly on editorial cuts. - Use Instrumental mode under spoken dialogue. Reserve dense brass and vocal layers for montages and trailers; request sparse strings, low pads, or solo piano when characters are speaking.
- Compare with Apache 2.0 local options. If your pipeline requires permissive local commercial weights on a consumer GPU, pair StepAudio-3-Music exploration with HeartMuLa and ACE-Step 1.5.
Related Reading
- HeartMuLa: Open Source Apache 2.0 Music Foundation Model
- ACE-Step 1.5: Open Source Music Generation Guide
- MiniMax Music 3 & GGUF: Open Source Film Score AI
- YuE2: Open Source Lyrics to Song Editable Score
Sources
Continue Reading
Video & LipSync
- Video Generator
- Text to Video
- Image to Video
- Start-End Frame to Video
- Draw to Video
- Motion Control
- Video Enhancer
- Video Upscaler
- Video Extend
- Video to Video LipSync
- Audio to Video LipSync
- Image to Video LipSync
- Video FaceSwap
- Seedance 2.5
- WAN 3.0
- Seedance 2
- FLUX 3
- Minimax H3
- Vidu Q3 Pro
- Grok Imagine Video 1.5
- Gemini Omni
- Google Veo 3.1
- Kling 3.0 Pro
- Luma Ray 3.2
- LTX 2.3
- Happy Horse 1.1
- Kling 3.0 Motion
- ByteDance Upscaler
- InfiniteTalk
- InsightFace
Image & Edit
- AI Character
- AI Actor
- Art Generator
- Text to Image
- Image to Image
- LoRA Image to Image
- Draw to Edit
- Image LoRA Training
- Remove Background
- Image Enhancer
- MidJourney 8.0
- Krea 2
- OpenAI GPT Image 2.0
- Nano Banana Pro Ultra
- Nano Banana 2 Lite
- Nano Banana Pro
- Minimax Image
- Clarity Pro
- Z-Image Turbo LoRA
- Krea 2 Turbo LoRA
- Z-Image LoRA Trainer
- Qwen Image LoRA Trainer
- Background Erase Network 2 (BEN2)


