Speech understanding

Speech understanding enables you to configure recognition, choose a model, and check language support.

Put recognition settings in the config object for Fast and Batch requests or in config within a Live session.update event.

Recognition configuration

Field	Type	Default	Fast	Batch	Live	Description
`language`	string	`en-US`	Yes	Yes	Yes	Recognition language. With `language_detection`, the preferred initial language; when `language_detection` is set and `language` is omitted, there is no preferred language.
`model`	string	Automatic	Yes	Yes	Yes	Recognition model. Omit when `language_detection.mode` is `once`; Scribe chooses the model from the detected language.
`language_detection`	object	—	Yes	Yes	Continuous only	Detects a language once or continuously. Live supports only `continuous`.
`interim_results`	boolean	—	No	No	Yes	Enables Captions: partial transcription results before a transcript item is completed.
`segmentation_mode`	string	`auto`	Yes	Yes	No	Use `auto` to split audio into segments at pauses in speech (voice activity detection), or `none` to disable segmentation.
`word_time_offsets`	boolean	`false`	Yes	Yes	No	Includes word-level start and end offsets.
`channel_separation`	boolean	`false`	Yes	Yes	No	Processes stereo channels independently.
`diarization`	boolean	`false`	Yes	Yes	Yes	Identifies speakers. When `false`, `speaker` is an empty string.
`profanity_filter`	boolean	`false`	Yes	Yes	Yes	Filters profanity when the selected model supports it.
`output_format`	string	`json`	Yes	Yes	No	Output format. Defaults to `json`.
`vocabulary`	object	—	Yes	Yes	Yes	Inline vocabulary for the current request, job, or session.
`vocabulary_id`	string	—	Yes	Yes	Yes	Persistent Vocabulary Asset to apply.

Unsupported combinations of language, model, execution mode, and settings return a validation error.

Select a model

Set config.model to explicitly select a recognition model. When it is omitted, Scribe selects a compatible standard model from the language, detection mode, and execution mode.

When config.language_detection.mode is once, omit config.model. Combining an explicit model with once returns HTTP 400 because Scribe must select the model after detecting the language.

Model	Coverage	Language detection
`zoom-scribe-en`	English	No
`zoom-scribe-en-pro`	English	No
`zoom-scribe-multilingual`	Ten languages; see Multilingual model languages	Continuous
`zoom-scribe-<language>`	A supported language-specific model	No
`whisper`	36 languages, including 26 locales not covered by `zoom-scribe-multilingual`	Continuous

The multilingual model IDs are the same in Fast and Live. Set config.model to the identifier below. You do not need a Live-specific prefix.

Model	Fast `config.model`	Live `config.model`
Zoom multilingual	`zoom-scribe-multilingual`	`zoom-scribe-multilingual`
Whisper	`whisper`	`whisper`

Models ending in -pro require premium-model access. Automatic selection does not choose a premium model unless you enable premium usage for the account.

zoom-scribe-en-pro is the Pro option for English, Scribe API 3B Pro High-Accuracy, for applications that need higher accuracy. Use the model ID, not the product name, in config.model.

The response's top-level model field identifies the model that processed the input.

{
    "file": "https://example.com/audio.wav",
    "config": {
        "language": "en-US",
        "model": "zoom-scribe-en",
        "word_time_offsets": true,
        "diarization": true
    }
}

Detect languages

Use config.language_detection to detect the input language.

Mode	Behavior
`once`	Detects one language, then recognizes the rest of the input with that language.
`continuous`	Detects language changes as the input is processed. Each segment contains one detected language.

Live supports only continuous.

For once, omit model so Scribe can select it based on the detected language:

{
    "config": {
        "language_detection": {
            "mode": "once",
            "candidate_languages": ["en-US", "fr-FR"]
        }
    }
}

Use candidate_languages to restrict detection. When both language and candidate_languages are present, the preferred language must be included in the candidate list.

For continuous, you can explicitly select a model that supports continuous detection:

{
    "config": {
        "language": "en-US",
        "model": "zoom-scribe-multilingual",
        "language_detection": {
            "mode": "continuous",
            "candidate_languages": ["en-US", "es-ES", "fr-FR"]
        }
    }
}

For Fast and Batch continuous detection, result.language and result.segments[].language are omitted because the input can contain multiple languages. Do not expect per-segment language labels. With once, result.language may also be absent; clients should handle its omission.

Live continuous detection returns the detected language in transcription.completed.language, such as fr for French.

Multilingual model languages

The zoom-scribe-multilingual model supports these languages and locales:

Language	Locale codes
English	`en-US`
Arabic	`ar-SA`, `ar-AE`
Chinese (Simplified)	`zh-CN`
French	`fr-FR`
German	`de-DE`
Italian	`it-IT`
Japanese	`ja-JP`
Korean	`ko-KR`
Portuguese	`pt-BR`, `pt-PT`
Spanish	`es-ES`

Broader language coverage

The whisper model recognizes 36 languages, with integrated language detection. These include the ten languages of the multilingual model and the following 26 locales:

Language	Locale code	Language	Locale code
Bengali	`bn-IN`	Malay	`ms-MY`
Cantonese	`zh-HK`	Persian (Farsi)	`fa-IR`
Catalan	`ca-ES`	Polish	`pl-PL`
Czech	`cs-CZ`	Romanian	`ro-RO`
Danish	`da-DK`	Russian	`ru-RU`
Dutch	`nl-NL`	Swedish	`sv-SE`
Estonian	`et-EE`	Tagalog (Filipino)	`tl-PH`
Finnish	`fi-FI`	Tamil	`ta-IN`
Greek	`el-GR`	Telugu	`te-IN`
Hebrew	`he-IL`	Thai	`th-TH`
Hindi	`hi-IN`	Turkish	`tr-TR`
Hungarian	`hu-HU`	Ukrainian	`uk-UA`
Indonesian	`id-ID`	Vietnamese	`vi-VN`

Transcript response

Fast returns the result inline. Batch writes the same result schema to the configured storage destination.

{
    "request_id": "3f2b8c1e-7a4d-4e9b-b6f0-1c5d8a2e9b47",
    "duration_sec": 10.99,
    "result": {
        "text_display": "Welcome everyone. Let's get started.",
        "segments": [
            {
                "id": "utt_0000",
                "start": 0.0,
                "end": 2.4,
                "channel": 0,
                "speaker": "Speaker 1",
                "text": "Welcome everyone.",
                "words": [{ "word": "Welcome", "start": 0.0, "end": 0.4 }]
            }
        ]
    },
    "model": "zoom-scribe-en",
    "usage": {
        "input_units": 10.99,
        "output_units": 0,
        "unit_type": "seconds"
    }
}

Process multiple channels

Fast and Batch accept mono and stereo media. Set channel_separation to true to recognize each stereo channel independently.

{
    "config": {
        "language": "en-US",
        "channel_separation": true,
        "diarization": false
    }
}

Every segment includes channel. Channel 0 is the left channel and channel 1 is the right channel. When separation is disabled or the input is mono, the value is 0.

Recognition configuration#

Select a model#

Detect languages#

Multilingual model languages#

Broader language coverage#

Transcript response#

Process multiple channels#