MOSS-Transcribe-Diarize 0.9B: Joint Speech Transcription and Speaker Identification for Filmmakers

The OpenMOSS team released MOSS-Transcribe-Diarize 0.9B on July 9, 2026 under an Apache 2.0 license. The model handles automatic speech recognition and speaker diarization jointly in a single inference pass, rather than running them as two separate sequential processes.

At the INTERSPEECH 2026 conference, the model won first place in the MLC-SLM Challenge, a benchmark specifically designed to evaluate the quality of multilingual transcription and speaker identification across real world audio conditions. The model supports more than 50 languages and handles audio files up to 90 minutes long.

For filmmakers and production teams, the tool addresses one of the most time consuming tasks in post production and documentary work: accurately transcribing recordings with multiple speakers and identifying which person said what, when.

Joint Transcription and Speaker Diarization

Most speech processing workflows treat transcription and speaker diarization as two separate problems. A transcription model converts speech to text. A diarization model segments the audio into speaker turns and labels each segment. The outputs are then merged in a post processing step, which introduces alignment errors at boundaries where speakers overlap or interrupt.

MOSS-Transcribe-Diarize solves both tasks simultaneously in a single model. The transcription and the speaker labels are generated together, using shared representations of the audio that capture both what is said and who is saying it at the same time.

The joint approach eliminates the alignment problem. Because the model learns the relationship between speech content and speaker identity during training, rather than inferring it after the fact, the output transcript already contains accurate speaker labels at the word level rather than the segment level.

The practical difference for a production team is significant. A sequential pipeline might correctly transcribe every word but mislabel speakers during overlapping speech, at turn boundaries, or in moments of crosstalk. A joint model maintains accuracy through those moments because speaker identity is part of the model's core output rather than a secondary annotation.

The MLC-SLM Challenge benchmark, which MOSS-Transcribe-Diarize won, specifically tests performance on audio from multiple speakers in real acoustic conditions, not clean studio recordings. The win at that benchmark is relevant evidence that the joint approach holds in conditions that production audio typically presents: background noise, room acoustics, and varied speaker distances from the microphone.

The situations where a sequential pipeline fails most visibly are also the situations that production teams encounter most often. When two speakers overlap at the end of a line, when a loud ambient sound cuts through dialogue, or when a speaker moves significantly closer to the microphone while speaking, the sequential pipeline misattributes the segment. A joint model is trained to recognize those boundary conditions as a unified problem rather than two separate edge cases.

For productions working with interview footage from multiple cameras, the joint architecture also reduces the amount of manual review required after transcription. A sequential system that generates errors in diarization requires an editor to listen back through every segment the system is uncertain about. A joint system produces fewer attribution errors in the first place, which means the review pass is shorter.

How the Model Works

At 0.9 billion parameters, the model is compact enough to run on consumer hardware without quantization. Comparable models that run transcription and diarization separately often require more total memory when both pipelines are active. The joint architecture trades the flexibility of modular pipelines for lower combined resource requirements.

The output format includes word level timestamps alongside speaker labels, which means the transcript can be used directly for subtitle generation and for aligning generated text to the audio waveform. Productions that need to create closed captions or generate timed subtitle files from interview footage can use the transcript output as the source document without a separate alignment step.

The 90 minute audio limit covers most common production contexts. Feature film audio tracks are typically processed by scene or reel, not as a single file. Documentary interviews rarely exceed the limit in a single session. For longer recordings, the audio can be split before processing and the transcripts merged afterward.

The model is available in the OpenMOSS-Team namespace on Hugging Face and in the OpenMOSS repository on GitHub. Both the model weights and the inference code are released under Apache 2.0, which permits commercial use without royalty requirements.

Because the weights are hosted on Hugging Face, the model can be loaded with standard Python tooling. A production team with a machine running a modern GPU can have the model running locally in under 30 minutes from first download, without needing a cloud API subscription or ongoing access fees. The Hugging Face model card includes inference examples that demonstrate the format of the output transcript with speaker labels.

The INTERSPEECH 2026 Benchmark

INTERSPEECH is the largest annual conference in speech and language processing, organized by the International Speech Communication Association. It draws research from academic institutions and industry labs across more than 70 countries. A first place finish at an INTERSPEECH challenge is a validated evaluation of model performance, not a marketing claim.

The MLC-SLM Challenge evaluated models on Multilingual Conversational Speech. The test conditions targeted what is hardest for standard transcription systems, including multiple speakers, non-native accents, overlapping speech, and varied acoustic environments. These are exactly the conditions that production audio presents, particularly in documentary, interview, and unscripted content.

MOSS-Transcribe-Diarize won the challenge across the combined metric of transcription accuracy and speaker assignment accuracy. A model can rank well on transcription while performing poorly on speaker identification. The combined metric ensures that the winner has to perform well on both problems simultaneously.

The timing of the release, nine days before the INTERSPEECH 2026 conference dates in August, suggests the team published weights and code before the full paper presentation, making the model available to practitioners ahead of the academic publication cycle.

The MLC-SLM Challenge is one of the few speech benchmarks that specifically evaluates the combined quality of transcription and speaker assignment as a single score rather than reporting them separately. That design choice matters for production contexts. A model that scores well on word error rate but poorly on speaker assignment is still a model that requires extensive manual correction before its output is usable. The combined metric selects for models that are genuinely useful in practice.

Post Production and Filmmaking Applications

The most direct use case for MOSS-Transcribe-Diarize in production is generating transcripts for Automated Dialogue Replacement sessions. ADR requires a reference transcript that shows exactly what the actor said in the original recording, word by word and moment by moment, so the ADR director can cue replacement lines accurately. A joint transcription and speaker identification model produces that reference automatically, with speaker labels that identify which character's dialogue needs replacement.

For documentary filmmakers, the tool addresses the primary post production bottleneck: transcribing hours of multi-person interview footage before the edit begins. A traditional workflow requires a transcription service or a team member to listen and type. MOSS-Transcribe-Diarize processes a 90 minute interview in a fraction of real time and returns a transcript that already labels each speaker, which the editor can use directly to find usable quotes without re-listening to the full recording.

Investigative documentary and journalism productions working under time pressure gain the most direct advantage. A producer who completes an interview at 6 PM and needs a working transcript before an editorial meeting at 9 AM no longer depends on overnight transcription services. The full labeled transcript is available within the same working session as the recording.

Productions using multiple cameras with separate audio tracks for each participant create a different problem: individual tracks are easy to transcribe but need to be reconciled into a single timeline view. A joint model can process the mixed room audio and produce a single timeline transcript with speaker labels, giving the editor a unified reference for the full conversation rather than separate transcripts per track.

Localization teams preparing content for international release use transcripts as the source document for subtitling and dubbing scripts. MOSS-Transcribe-Diarize's 50+ language support means that content produced in a language other than English can be transcribed directly, without a translation step in the pipeline. A Korean feature film's dialogue track can be transcribed in Korean, reviewed for accuracy, and then sent to translators rather than requiring a native speaker to produce the initial transcript manually.

The speaker identification capability also has editorial value beyond transcription. In a documentary with multiple interview subjects, knowing which person is speaking at every moment in the archive footage allows the editor to search for a specific subject's comments without watching the full interview. The transcript becomes a searchable database of who said what and when.

In pre-production, the model is useful for table read recordings. A table read with four or more actors generates a recording that is long, multi-speaker, and difficult to use as a reference document without a transcript. Running the recording through MOSS-Transcribe-Diarize produces a complete script breakdown showing which actor read which line, which is useful for identifying pacing issues, line delivery questions, and timing mismatches before the shoot begins.

Unscripted and reality production teams also benefit directly. Any production that generates hours of continuous recording across multiple microphones, from day in the life footage to observational documentary, faces the same core problem: too much audio to review manually before the edit. A 90 minute limit covers most shooting sessions. A team processing footage the same evening can have a complete labeled transcript of the day's recording before the following morning's shoot begins.

The AI FILMS Studio voice workspace provides access to text-to-speech synthesis and voice generation tools that work alongside transcription in a full audio pipeline.

Technical Specifications

Specification	Value
Parameters	0.9B
Max audio length	90 minutes
Languages	50+
License	Apache 2.0
Benchmark	1st place, MLC-SLM Challenge, INTERSPEECH 2026
Release date	July 9, 2026

The Apache 2.0 license permits commercial use, which is relevant for production companies evaluating whether to integrate the model into their workflows. Several speech models with competitive performance use licenses that prohibit commercial use, which rules them out for professional production contexts. MOSS-Transcribe-Diarize does not have that restriction.

The 50+ language count covers all major production languages and a significant number of regional languages used in documentary and international co-production contexts. The model's multilingual support was a specific evaluation criterion in the MLC-SLM Challenge, which distinguishes it from benchmarks that test only English or a handful of high resource languages.

Local deployment on a production machine eliminates the privacy concern associated with sending audio to a transcription API. For footage containing sensitive conversations, unpublished dialogue, or early cuts under NDA, a locally deployed model keeps the audio on the production's own hardware throughout the transcription process.

The 0.9B parameter size is also relevant for batch processing. A production running overnight transcription jobs can process multiple 90 minute sessions sequentially on a single GPU without the memory overhead required by larger models. For studios with significant archival interview footage, that means an entire catalog can be transcribed and labeled with speaker identities in a series of overnight runs rather than requiring dedicated compute infrastructure or cloud API costs per file.

The July 9 release date also marks an unusually short gap between a challenge win and a public model release. Research teams that win benchmarks at major conferences typically publish a paper at the conference and release weights several weeks later, after the peer review process is complete. MOSS-Transcribe-Diarize was available to download before the INTERSPEECH paper presentation date, which gives practitioners access to the model well ahead of the usual academic publication timeline.

The OpenMOSS Ecosystem

OpenMOSS first became known in the filmmaking community through its audio generation tools. MOSS-Transcribe-Diarize extends the team's scope from audio generation to audio analysis, covering both directions of the audio pipeline in a single open source toolkit.

MOSS-Transcribe-Diarize is the latest addition to the OpenMOSS toolkit for audio production. The team has previously released tools covering distinct stages of an audio pipeline: MOSS-TTS for text to speech synthesis, which generates voice output from text using local transformer inference, and MOSS SoundEffect v2 for sound design generation, which produces sound effects from text descriptions under an open license.

Together, the three tools cover a broad range of the audio production cycle: generating voices, generating sound effects, and now converting recorded speech back into text with speaker attribution. A production that uses all three tools can work from script to audio and from audio back to text without requiring a commercial API subscription for any step of the process.

The addition of a transcription and diarization model also opens a feedback loop. A production that generates voice with MOSS-TTS can verify the output by transcribing it with MOSS-Transcribe-Diarize and comparing the result to the input text, catching errors in pronunciation or timing that audio quality alone might not reveal. That loop between generation and verification is not possible when the two tools come from different providers with incompatible output formats.

The OpenMOSS team has now released models covering the full audio cycle in a production pipeline. That breadth is unusual for an open source team. Most open source audio work focuses on a single problem, such as speech synthesis or sound classification. OpenMOSS has instead built across the full pipeline, releasing tools in sequence that address each stage without requiring a commercial API at any point.

Filmmakers working with AI audio tools can also explore voice generation and sound synthesis capabilities in the AI FILMS Studio sound workspace.

---

Sources

GitHub: OpenMOSS/MOSS-Transcribe-Diarize Hugging Face: OpenMOSS-Team/MOSS-Transcribe-Diarize INTERSPEECH 2026 | International Speech Communication Association