Failover and reconnection

View as Markdown

Realtime Media Streams (RTMS) uses a signaling connection that manages the stream lifecycle, and media connections that carry audio and transcript data. Depending on which connection is interrupted and why, the recovery steps differ.

For error codes, webhook issues, and other common problems, see Troubleshooting RTMS.

ScenarioTriggerWhat to reconnect
RTMS server failurephone.call_rtms_started arrives for an already-active streamSignaling + media connections
Signaling connection droppedphone.call_rtms_interrupted webhookSignaling + media connections
Media connection droppedMEDIA_CONNECTION_INTERRUPTED signal eventAffected media connection only
Stream terminatedphone.call_rtms_stopped with stop_reason 10Full RTMS connection restart

Store the connection details

Store the call_id, rtms_stream_id, and server_urls from the phone.call_rtms_started event for each stream, along with the media server URLs from the signaling handshake response. You need these values to reconnect. The phone.call_rtms_interrupted event doesn't include server_urls, so your app reconnects with the values it stored.

const RECONNECT_BASE_DELAY_MS = 3000;
const RECONNECT_BACKOFF_FACTOR = 2;
const RECONNECT_MAX_DELAY_MS = 30000;

Exponential backoff

Scenarios 2 and 3 use this delay function. Reconnection attempts use exponential backoff, starting at 3 seconds and doubling after each attempt until the delay reaches 30 seconds. Reset reconnectAttempts to 0 after each successful handshake so the backoff counter doesn't carry over into future disconnections.

/**
 * Calculate reconnection delay with exponential backoff.
 * Attempt 0: 3s, Attempt 1: 6s, Attempt 2: 12s, Attempt 3: 24s, then capped at 30s.
 */
function getReconnectDelay(attempts) {
    return Math.min(
        RECONNECT_BASE_DELAY_MS * Math.pow(RECONNECT_BACKOFF_FACTOR, attempts),
        RECONNECT_MAX_DELAY_MS,
    );
}

Scenario 1: RTMS server failure

RTMS server failures require a full reconnection between your app and the RTMS server.

Problem

A phone.call_rtms_started event arrives with an rtms_stream_id that's already an active stream, indicating an RTMS server failure.

Solution

Complete the entire connection process to create a new connection to the server.

To detect that the stream is active

Use an if/then statement to determine whether a stream is already active. Add this check to the phone.call_rtms_started handler in your webhook receiver.

if (sessions.has(rtms_stream_id)) {
    // Scenario 1: RTMS server failed and restarted. Reconnect with new URLs.
    handleServerFailureReconnect(call_id, rtms_stream_id, server_urls);
}

To create the new connections

Use the handleServerFailureReconnect function to reestablish all the connections.

/**
 * SCENARIO 1: RTMS Server Failure
 *
 * What happened:
 *   The Zoom RTMS server went down. A new RTMS server has started and sent
 *   a fresh phone.call_rtms_started webhook with (possibly new) server_urls.
 *
 * What to do:
 *   Tear down all existing connections and start fresh with the new URLs.
 *   The call_id and rtms_stream_id stay the same.
 *
 * Trigger:
 *   phone.call_rtms_started webhook arrives for a rtms_stream_id we already have.
 *
 * See: Failover and reconnection > Scenario 1: RTMS server failure
 */
function handleServerFailureReconnect(callId, streamId, serverUrls) {
    console.log(
        `SCENARIO 1: RTMS server failure for stream ${streamId}. Reconnecting.`,
    );

    // Close any existing sockets and discard the old state
    closeSession(streamId);

    // Create a fresh connection state with the new server URLs
    const session = {
        callId,
        streamId,
        serverUrls,
        signalingWs: null,
        media: {},
        reconnectAttempts: 0,
    };
    sessions.set(streamId, session);

    // Connect immediately: the new server is ready for us
    connectToSignalingWebSocket(session, serverUrls);
}

Scenario 2: Signaling connection dropped

If the signaling connection is dropped, the signaling and media connections between your app and the RTMS server need to be reestablished.

Problem

A phone.call_rtms_interrupted event arrives, indicating that the signaling connection was dropped.

Solution

Close all connections, schedule connectToSignalingWebSocket() after an exponential backoff delay, and create a new connection to the server using the server_urls you stored. When the signaling handshake succeeds, reconnect the media connections as the handshake response describes.

To detect a phone.call_rtms_interrupted event

Use a case statement to detect a signaling connection issue.

// ---------------------------------------------------------------
// phone.call_rtms_interrupted
//
// SCENARIO 2: Our signaling connection dropped. The server interrupted
// both signaling and media. We must reconnect both.
// ---------------------------------------------------------------
case 'phone.call_rtms_interrupted': {
  const { call_id, rtms_stream_id } = payload;
  console.log(`phone.call_rtms_interrupted for call ${call_id}`);
  handleSignalingInterruptedReconnect(rtms_stream_id);
  break;
}

To create the new connections

Use the handleSignalingInterruptedReconnect function to reestablish all the connections.

/**
 * SCENARIO 2: Signaling connection dropped
 *
 * What happened:
 *   Our app's signaling WebSocket dropped (network issue, for example).
 *   Since signaling controls the stream, the RTMS server interrupted BOTH
 *   the signaling and media connections.
 *
 * What to do:
 *   Re-establish both signaling and media connections, using the
 *   server_urls stored from phone.call_rtms_started.
 *
 * Trigger:
 *   phone.call_rtms_interrupted webhook
 *
 * See: Failover and reconnection > Scenario 2: Signaling connection dropped
 */
function handleSignalingInterruptedReconnect(streamId) {
    const session = sessions.get(streamId);
    if (!session) {
        // We lost track of this stream, so there's nothing to reconnect with
        console.log(
            `No stored state for stream ${streamId}. Ignoring interruption.`,
        );
        return;
    }

    // Close any lingering sockets
    closeSockets(session);
    session.state = 'RECONNECTING';

    const delay = getReconnectDelay(session.reconnectAttempts);
    session.reconnectAttempts++;
    console.log(
        `Reconnecting in ${delay}ms (attempt #${session.reconnectAttempts})...`,
    );

    setTimeout(() => {
        if (sessions.get(streamId) !== session) return;
        connectToSignalingWebSocket(session, session.serverUrls);
    }, delay);
}

function closeSockets(session) {
    session.signalingWs?.close();
    Object.values(session.media).forEach((ws) => ws.close());
    session.media = {};
}

Scenario 3: A media connection dropped

If a media connection is dropped, only the affected media connection between your app and the RTMS server needs to be reestablished. The signaling connection remains active.

Problem

The signaling connection delivers an EVENT_UPDATE message with event_code 2, indicating which media connection was dropped. For the full event code mapping, see RTMS_EVENT_CODE.

Solution

Close the affected media connection, schedule connectToMediaWebSocket() after an exponential backoff delay, and create a new connection to the server using the media URL you stored for that media type. Use the media type from the event to identify which connection was dropped.

To detect the dropped media connection

Use a case statement to detect a media connection issue.

// ---------------------------------------------------------------
// RECONNECTION SCENARIO 3: Media connection interrupted
//
// The signaling connection is still alive, but a media socket went down.
// The server notifies us through the signaling channel.
//
// Action: Reconnect ONLY the media WebSocket. Signaling stays up.
// See: Failover and reconnection > Scenario 3: A media connection dropped
// ---------------------------------------------------------------
if (message.msg_type === 6 && message.event.event_code === 2) {
    handleMediaOnlyReconnect(session, message.event.media_type);
}

To create the new connection

Use the handleMediaOnlyReconnect function to reestablish the affected media connection.

/**
 * SCENARIO 3: Media connection dropped
 *
 * What happened:
 *   Only one media WebSocket dropped. The signaling connection is still alive.
 *
 * What to do:
 *   Reconnect ONLY that media WebSocket. Signaling stays up.
 *
 * See: Failover and reconnection > Scenario 3: A media connection dropped
 */
function handleMediaOnlyReconnect(session, mediaType) {
    console.log(
        `Closing media socket ${mediaType} and scheduling reconnection...`,
    );

    session.media[mediaType]?.close();
    delete session.media[mediaType];

    const delay = getReconnectDelay(session.reconnectAttempts);
    session.reconnectAttempts++;
    console.log(
        `Reconnecting media ${mediaType} in ${delay}ms (attempt #${session.reconnectAttempts})...`,
    );

    setTimeout(() => {
        if (sessions.get(session.streamId) !== session) return;
        connectToMediaWebSocket(
            session,
            session.mediaUrls[mediaType],
            mediaType,
        );
    }, delay);
}

Scenario 4: Stream terminated

If a connection error can't be recovered, the RTMS server ends the stream and sends a phone.call_rtms_stopped webhook with stop_reason 10. The stream can't be recovered, so you need to restart it.

Problem

A phone.call_rtms_stopped webhook arrives with a stop_reason of 10, which means a connection error ended the stream. A stop_reason of 2 means the user stopped RTMS, so don't restart it.

Solution

Close all connections for the stream and stop tracking it. For stop_reason 10, if the call is still active, restart the entire RTMS connection process from the beginning to resume receiving complete data.

function handleStreamStopped(streamId) {
    closeSession(streamId);
}

If reconnection fails

If reconnect attempts are exhausted, for example after the backoff delay has reached its 30-second cap and attempts keep failing, treat the stream as unrecoverable. Close all open connections, stop tracking the stream, and surface the failure to your application. Don't retry indefinitely.