MOSS-Audio: Open Source Audio Understanding Model Under Apache 2.0

OpenMOSS published the formal technical report for MOSS-Audio on June 1, 2026, releasing it under Apache 2.0 for commercial use. The model analyzes speech, environmental sound, and music across four variants ranging from 4.6 billion to 8.6 billion parameters, with a dedicated reasoning mode that produces written explanations of its audio assessments alongside the results.

What MOSS-Audio Does

MOSS-Audio is an audio understanding model, not a generation model. It takes audio files as input and returns analysis: transcriptions with word level timestamps, speaker identification, emotion and pitch detection, environmental sound classification, music analysis, and answers to natural language questions about audio content.

The distinction matters for filmmakers. MOSS-SoundEffect v2.0 generates sound effects from text descriptions, and NAVA generates synchronized video and audio from a single prompt. MOSS-Audio does the complementary job: it tells you what is already in a piece of audio and explains why.

For audio post production workflows that require both generation and understanding, MOSS-Audio slots in as the analysis layer. It can analyze incoming dialogue recordings for quality issues, identify ambient sound elements in production audio that need to be replaced, and evaluate the emotional register of a music cue against the scene it underscores. Those tasks require understanding, and MOSS-Audio is the first Apache 2.0 model to cover all three audio categories at this benchmark level.

Architecture

MOSS-Audio couples a dedicated audio encoder with a modality adapter and a large language model. The encoder runs at 12.5 Hz temporal resolution, processing audio into tokens the language model can read.

DeepStack and Time-Aware Representation

DeepStack cross layer feature injection addresses a known limitation of audio language models: the final encoder output summarizes the audio but loses granular acoustic detail in the process. Earlier encoder layers contain information about fine spectral structure, transient events, and low level timbral characteristics that gets compressed as the encoder moves toward its final representation. DeepStack selects intermediate encoder outputs and projects them directly into the early layers of the language model, giving the LLM access to acoustic context it would otherwise never see.

The practical effect is most visible on tasks where fine acoustic detail matters. Speaker identification across noisy recordings, pitch tracking through complex polyphonic audio, and emotion detection in speech that uses subtle prosodic cues all benefit from the richer low level context DeepStack provides. On benchmark evaluations, the 8B-Thinking variant, which combines DeepStack with the reasoning mode, achieves the highest scores across all tested dimensions.

The cross layer injection design is also what makes MOSS-Audio more reliable on non standard audio than models that use only the final encoder layer. Production audio rarely consists of clean speech or isolated instrument recordings. The additional low level context helps the model correctly classify sounds that fall outside the clean training distribution, such as distorted dialogue, overlapping voices, or environmental audio recorded in uncontrolled conditions.

Time aware representation addresses a separate problem: grounding audio analysis in time. A model that can identify what is in an audio file but cannot say when it happens is limited for post production use. Time tokens inserted between frame representations during pretraining teach the model to associate analysis outputs with specific positions in the recording. The word level timestamp alignment scores, where MOSS-Audio-8B-Instruct substantially outperforms Qwen3-Omni and Gemini, demonstrate that this training approach works.

Four Variants, Two Modes

The release includes four models: MOSS-Audio-4B-Instruct (~4.6B parameters), MOSS-Audio-4B-Thinking (~4.6B), MOSS-Audio-8B-Instruct (~8.6B), and MOSS-Audio-8B-Thinking (~8.6B). All four use the Qwen3 backbone at their respective sizes.

The Instruct variants produce direct answers. The Thinking variants generate chain of thought reasoning alongside the result, working through the analysis step by step before delivering a conclusion. For a question about what emotion is present in a dialogue recording, the Thinking mode returns the reasoning behind the answer rather than a label alone. The 8B-Thinking variant achieves the highest scores across all benchmarks.

Benchmark Performance

On general audio understanding, MOSS-Audio-8B-Thinking scores 71.08 on average across four benchmarks: MMAU (77.33), MMAU-Pro (64.92), MMAR (66.53), and MMSU (75.52). These results represent a meaningful lead over publicly available alternatives at comparable scale.

Speech Captioning

On the speech captioning task, MOSS-Audio-8B-Instruct achieves an average score of 3.7252 and leads on 11 of 13 evaluated dimensions including gender, accent, pitch, volume, and emotion.

The Thinking Variants and Audio Reasoning

The Thinking variants' value is most apparent on tasks where the reason behind an analysis matters as much as the analysis itself. A music supervisor who asks "why does this reference cue feel tense?" does not only need a label. They need the model to identify which acoustic elements, whether pitch, rhythm, harmonic dissonance, or dynamic range, produce the effect so that they can describe it to a composer.

MOSS-Audio's Thinking mode outputs that breakdown. It works through the audio systematically before reaching a conclusion, naming specific elements and their perceptual effects. For sound supervision, that output is a working brief. For reference analysis, it is a detailed account of what makes a piece of audio work.

The chain of thought approach also provides a quality signal the Instruct variants cannot. When the Thinking mode produces a confident answer after a clear reasoning chain, the confidence is grounded. When it hedges or circles back during reasoning, that uncertainty is visible in the output. An Instruct variant returns a label with no indication of whether the model's confidence is high or low.

What Filmmakers Can Do With It

Automated dialogue transcription with precise timestamps removes a manual step from post production editorial prep. The word level alignment means sync points can be located exactly rather than approximated. On the sound design side, the model can analyze and tag production sound libraries at scale, returning descriptions accurate enough for search and retrieval across environmental sound, music, and voice recordings.

The Thinking variants add a layer that pure generation tools do not offer. Models like JavisDiT++ create audio content from instructions. MOSS-Audio's reasoning mode explains what is in existing audio and articulates why specific elements are present, making it useful for sound supervision and reference analysis rather than generation.

Stable Audio 3 generates original music from text. MOSS-Audio is the corresponding understanding layer: it reads existing audio with the same depth that generation models apply to writing it.

For voice and speech specifically, MisoTTS released the same week as an 8 billion parameter open weight voice model under a modified MIT license, covering speech generation from text and one shot voice cloning from a short audio sample. The OpenMOSS team followed in June 2026 with MOSS-TTS Local Transformer v1.5, a 5B parameter voice cloning model that generates 48 kHz stereo speech across 31 languages with explicit dialogue timing control under Apache 2.0. For original music generation rather than speech or audio analysis, Google's Magenta RealTime 2 also released that week under CC-BY-4.0, running as an Audio Unit plugin inside any AU compatible DAW at 200ms control latency.

For productions with existing audio libraries, MOSS-Audio can also serve as a tagging and search tool. A library of field recordings, foley, and ambient sound that has never been cataloged can be processed through MOSS-Audio's classification pipeline to generate searchable metadata. The output includes not just category labels but descriptions of acoustic characteristics, which enables more precise retrieval than keyword tags alone.

The model runs locally with Python 3.12, FFmpeg 7, and CUDA 12.8. Optional FlashAttention 2 support is available for compatible hardware. Weights for all four variants are available via HuggingFace. For filmmakers working on audio production, the sound workspace and music workspace in AI FILMS Studio cover the generation side of the workflow.

The MOSS-Audio Model Family in Context

OpenMOSS has built a complementary set of models around audio production workflows. MOSS-SoundEffect v2.0 handles environmental sound generation from text descriptions, covering the sound design creation side. MOSS-Audio handles understanding and analysis across all three audio types: speech, environmental sound, and music.

The two models address the fundamental split in audio post production work: creation and evaluation. A sound editor who generates a foley effect with MOSS-SoundEffect needs to verify it matches the scene's acoustic environment before finalizing it. MOSS-Audio's classification capabilities provide that verification layer. The models are designed to work together across the same workflow.

The June 2026 MOSS-TTS Local Transformer v1.5 release extended the family into voice synthesis: a 5B parameter model generating 48 kHz stereo speech across 31 languages under Apache 2.0. With understanding, effects generation, and speech synthesis available from the same research team, the OpenMOSS audio suite covers the three main categories of audio post production without requiring external services or restrictive licensing.

The consistency of the Apache 2.0 license across the OpenMOSS audio releases is deliberate. A production company that evaluates one tool in the suite and decides to build an internal workflow faces no licensing transition when adopting additional tools. The entire OpenMOSS audio stack can be deployed under a single legal assessment.

What Apache 2.0 Means for Production Audio Tools

Apache 2.0 is the most permissive license available for a model of this capability level. Commercial use, adaptation, redistribution, and incorporation into proprietary products are all permitted without restriction. Attribution in derivative works is required. There is no revenue threshold, no production scale limit, and no requirement to make modifications available to the public.

For a production company building internal audio analysis tools, Apache 2.0 means the MOSS-Audio weights can be integrated into a proprietary pipeline without licensing negotiations or usage fees. The model can be fine tuned on production audio specific to a particular project or studio's needs, and the fine tuned version can be retained as proprietary infrastructure.

The Apache 2.0 release also removes the enterprise legal review step that often delays evaluation of more restrictively licensed models. A model under a noncommercial or research only license requires legal sign off before even a proof of concept can be run on real production audio. Apache 2.0 permits evaluation, integration, and deployment without that review, which is why the adoption timeline for permissively licensed models is typically shorter than for restricted ones.

The license is also irrevocable for the version it applies to. A production pipeline built on MOSS-Audio's current weights is not subject to future licensing changes by OpenMOSS. That stability matters for organizations building internal tools with a multi year service life.

---

Update (August 19, 2026): Audio understanding and audio generation are separate problems. MiDashengLM-Gen covers the generation side under the same Apache 2.0 licence, released in August 2026. Full story here.

Sources

GitHub: OpenMOSS/MOSS-Audio
HuggingFace (4B-Instruct): OpenMOSS-Team/MOSS-Audio-4B-Instruct
HuggingFace (8B-Instruct): OpenMOSS-Team/MOSS-Audio-8B-Instruct
HuggingFace (8B-Thinking): OpenMOSS-Team/MOSS-Audio-8B-Thinking
arXiv: MOSS-Audio Technical Report
License: Apache 2.0 (commercial use permitted)