# Getting Zoom Phone audio and transcripts from Realtime Media Streams using WebSockets

<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->

In this guide, you'll build a server that uses WebSockets to get live audio and transcripts from Zoom Phone calls using Realtime Media Streams (RTMS).

This [sample app](https://github.com/zoom/phone-rtms-auto) is available on _GitHub_.

The server will:

1. Listen for the webhook events `phone.call_rtms_started`, `phone.call_rtms_stopped`, and `phone.call_rtms_interrupted`
2. Generate a signature for handshake requests
3. Connect to the signaling WebSocket for the call, then to the media WebSockets
4. Receive audio and transcripts in real time

## Prerequisites

1. Create a general app in the [Zoom Marketplace](https://marketplace.zoom.us/develop/create).
2. Add the [Zoom Phone RTMS features](/docs/rtms/phone/add-features/) to the app, including the events, scopes, and the per-user setting.

## Add the app for other users

To add the app for other users in your account, as an admin you have to:

1. Sign into the [Zoom App Marketplace](https://marketplace.zoom.us/).
2. To go to the app, in the upper-right of the screen, choose **Manage**.
3. Select your app from the list.
4. Choose **Preview Your App Listing**.
5. Choose **Add for Others**.
6. Select the users you want to add the app for.
7. Choose **Allow** to grant the app the requested permissions on their behalf.

## Get started

Create a new Node.js project and install [express](https://expressjs.com/), [dotenv](https://www.npmjs.com/package/dotenv), and [ws](https://www.npmjs.com/package/ws) as dependencies.

```bash
npm init -y
```

```bash
npm install express dotenv ws
```

Create a basic server on `localhost:3000`. Create a new file named `index.js` and add the following code to it:

```javascript
import express from "express";
import dotenv from "dotenv";
import WebSocket from "ws";
import crypto from "crypto";

// Load environment variables from .env
dotenv.config();

const app = express();

// Enable JSON body parsing
app.use(express.json());

// Basic root route for testing
app.get("/", (req, res) => {
    res.send("Zoom RTMS Server is up and running.");
});

// Listen on localhost:3000
const PORT = 3000;
app.listen(PORT, () => {
    console.log(`Server is listening on http://localhost:${PORT}`);
});
```

### Set up environment variables

Create a `.env` file in your project root. Add the following environment variables:

```javascript
ZOOM_APP_CLIENT_ID = your_client_id;
ZOOM_APP_CLIENT_SECRET = your_client_secret;
PORT = 3000;
```

Get your `ZOOM_APP_CLIENT_ID` and `ZOOM_APP_CLIENT_SECRET` from the **Basic Information** page of your app.

## Build the webhook receiver

When an RTMS stream starts, stops, or is interrupted, your app receives a webhook event. The `phone.call_rtms_started` event contains the `server_urls` you use to connect to the signaling server.

When a stream starts: `phone.call_rtms_started`

```json
{
    "event": "phone.call_rtms_started",
    "event_ts": 1626230691572,
    "payload": {
        "call_id": "CALL_ID",
        "rtms_stream_id": "RTMS_STREAM_ID",
        "server_urls": "wss://SIGNALING_SERVER_URL"
    }
}
```

When a stream stops: `phone.call_rtms_stopped`

```json
{
    "event": "phone.call_rtms_stopped",
    "event_ts": 1626230691572,
    "payload": {
        "call_id": "CALL_ID",
        "rtms_stream_id": "RTMS_STREAM_ID",
        "stop_reason": 2
    }
}
```

For all `stop_reason` values, see [RTMS_STOP_REASON](/docs/rtms/data-types/#rtms_stop_reason).

To connect to the stream, your app needs the `call_id`, `rtms_stream_id`, and `server_urls` from the payload. Add a `/webhook` route to receive the POST requests from your event subscription. Add the following code to `index.js`:

```javascript
// Keep track of active RTMS sessions, keyed by rtms_stream_id
const sessions = new Map();

app.post("/webhook", (req, res) => {
    const { event, payload } = req.body;

    if (event === "phone.call_rtms_started") {
        const { call_id, rtms_stream_id, server_urls } = payload;
        console.log(`Starting RTMS for call ${call_id}`);

        const session = {
            callId: call_id,
            streamId: rtms_stream_id,
            signalingWs: null,
            media: {},
            mediaUrls: {},
            channelPaths: new Map(),
            sessionDir: `recordings/${call_id}`,
            reconnectAttempts: 0,
        };
        sessions.set(rtms_stream_id, session);
        connectToSignalingWebSocket(session, server_urls);
    } else if (event === "phone.call_rtms_stopped") {
        const { call_id, rtms_stream_id, stop_reason } = payload;
        console.log(
            `Stopping RTMS for call ${call_id}, stop reason ${stop_reason}`,
        );
        closeSession(rtms_stream_id);
    } else if (event === "phone.call_rtms_interrupted") {
        const { call_id, rtms_stream_id } = payload;
        console.log(
            `RTMS interrupted for call ${call_id}, stream ${rtms_stream_id}`,
        );
    } else {
        console.log("Unknown event:", event);
    }

    res.sendStatus(200);
});
```

## Create the signature generator

Next, create a function to generate the signature for the handshake requests using HMAC SHA256. The signature signs the client ID, call ID, and RTMS stream ID, joined by commas.

Add the following code to `index.js`:

```javascript
function generateSignature(callId, streamId) {
    const message = `${CLIENT_ID},${callId},${streamId}`;
    return crypto
        .createHmac("sha256", CLIENT_SECRET)
        .update(message)
        .digest("hex");
}
```

Read the client ID and secret from the environment variables you set earlier. Add the following lines near the top of `index.js`, after `dotenv.config()`:

```javascript
const CLIENT_ID = process.env.ZOOM_APP_CLIENT_ID;
const CLIENT_SECRET = process.env.ZOOM_APP_CLIENT_SECRET;
```

## Connect to the signaling server with WebSockets

Once the `phone.call_rtms_started` event arrives, connect to the signaling server using the `server_urls` value from the payload. The value can contain more than one URL, separated by commas, so the helper below selects the `wss://` URL. Send a handshake request when the connection opens.

Add the following code to `index.js`:

```javascript
function selectWebSocketUrl(serverUrls) {
    return serverUrls
        .split(",")
        .map((url) => url.trim())
        .find((url) => url.startsWith("wss://"));
}

function send(ws, message) {
    ws.send(JSON.stringify(message));
}

function connectToSignalingWebSocket(session, serverUrl) {
    const ws = new WebSocket(selectWebSocketUrl(serverUrl));
    session.signalingWs = ws;

    ws.on("open", () => {
        console.log(`Signaling WebSocket opened for call ${session.callId}`);

        send(ws, {
            msg_type: 1, // SIGNALING_HAND_SHAKE_REQ
            protocol_version: 2, // Zoom Phone RTMS requires protocol version 2
            sequence: 0,
            call_id: session.callId,
            rtms_stream_id: session.streamId,
            signature: generateSignature(session.callId, session.streamId),
        });
    });

    ws.on("message", (data) => {
        const message = JSON.parse(data.toString());

        if (message.msg_type === 2 && message.status_code === 0) {
            // Successful handshake response with media server URLs
            const urls = message.media_server.server_urls;
            session.mediaUrls = { 1: urls.audio, 8: urls.transcript };
            session.reconnectAttempts = 0; // Reset backoff after a successful handshake
            connectToMediaWebSocket(session, urls.audio, 1); // AUDIO
            connectToMediaWebSocket(session, urls.transcript, 8); // TRANSCRIPT
        } else if (message.msg_type === 12) {
            // KEEP_ALIVE_REQ: respond on the same socket, echoing the timestamp
            send(ws, { msg_type: 13, timestamp: message.timestamp });
        }
    });

    return ws;
}
```

Store a reference to the signaling WebSocket, because you need it later to send the `CLIENT_READY_ACK` after the media handshake completes. Failing to respond to keep-alive requests causes the server to close the connection.

## Connect to the media server with WebSockets

When the signaling handshake succeeds, the signaling server responds with the media server URLs in `media_server.server_urls`:

```json
{
    "msg_type": 2,
    "protocol_version": 2,
    "sequence": 0,
    "media_server": {
        "server_urls": {
            "audio": "wss://...",
            "transcript": "wss://..."
        }
    }
}
```

Your app opens two separate WebSocket connections: one for audio using the `audio` URL, and one for transcripts using the `transcript` URL. Each of these values can also hold several URLs separated by commas, so the helper selects the `wss://` URL for each connection. Each connection uses its own `media_type` value: 1 for audio and 8 for transcript.

Add the following code to `index.js`:

```javascript
function connectToMediaWebSocket(session, mediaUrl, mediaType) {
    const ws = new WebSocket(selectWebSocketUrl(mediaUrl));
    session.media[mediaType] = ws;

    ws.on("open", () => {
        send(ws, {
            msg_type: 3, // DATA_HAND_SHAKE_REQ
            protocol_version: 2,
            call_id: session.callId,
            rtms_stream_id: session.streamId,
            signature: generateSignature(session.callId, session.streamId),
            media_type: mediaType, // 1 = AUDIO, 8 = TRANSCRIPT
            payload_encryption: false,
        });
    });

    ws.on("message", (data) => {
        const message = JSON.parse(data.toString());

        if (message.msg_type === 4 && message.status_code === 0) {
            // Handshake succeeded: send the client ready ACK on the signaling socket
            send(session.signalingWs, {
                msg_type: 7, // CLIENT_READY_ACK
                rtms_stream_id: session.streamId,
            });
        } else if (message.msg_type === 12) {
            // KEEP_ALIVE_REQ: respond on the same socket, echoing the timestamp
            send(ws, { msg_type: 13, timestamp: message.timestamp });
        } else if (message.msg_type === 14) {
            // MEDIA_DATA_AUDIO: raw PCM audio
            const audioBuffer = Buffer.from(message.content.data, "base64");
            // Convert the PCM data to a playable format before you use it.
        } else if (message.msg_type === 17) {
            // MEDIA_DATA_TRANSCRIPT
            console.log("Received transcript:", message.content);
        }
    });

    return ws;
}
```

The `CLIENT_READY_ACK` tells the RTMS server that your app is ready to receive media. After you send it, the RTMS server begins streaming data on the audio and transcript WebSockets. Audio packets arrive with `msg_type 14`, and transcript chunks arrive with `msg_type 17`.

Audio data is received as raw PCM. Convert it to a playable format before using it.

## Close the connections when the stream stops

When your app receives the `phone.call_rtms_stopped` event, close the WebSocket connections for that stream. Add the following code to `index.js`:

```javascript
function closeSession(streamId) {
    const session = sessions.get(streamId);
    if (!session) return;

    session.signalingWs?.close();
    Object.values(session.media).forEach((ws) => ws.close());
    sessions.delete(streamId);
}
```

## Start the app to receive data

Start your server with `node index.js`, and make a test call with a user who has RTMS turned on. Your app logs the webhook events as they arrive, and it logs transcript chunks as the call progresses. Audio packets arrive on the audio WebSocket, and your app converts them to a playable format.

## Troubleshooting

If you encounter errors or issues while setting up or using RTMS, see [Troubleshooting RTMS](/docs/rtms/troubleshooting/) for common error codes, webhook issues, and solutions.
