Use raw data

Access raw video and raw audio data through raw streaming. When accessing raw data through raw streaming, the host gives the meeting participant associated with the SDK app live streaming permission.

This enables more flexibility than live streaming for video and audio streams. In addition to session-wide audio and video access, this enables access to raw audio and video data for each participant unless the participant's audio or video are muted.

Use of live streaming tokens to access raw data is a controlled feature. Before using this feature, contact our support team  to enable it.

For raw streaming, if the SDK user is also the meeting host, they can start both raw streaming and an actual live streaming service. If the SDK user is not the meeting host, they must request raw streaming permission from the meeting host. In a meeting, the host can perform real external live streaming, with one or more SDK clients doing raw streaming locally.

The SDKs prompt users with a dialog to let them know when recording has started, as in the Zoom client.

Raw data formats#

• Video is in YUV420p format
• Video is in CVPixelBufferRef
• Audio is in PCM format

Before you start#

To work with raw data in Meeting SDK for iOS, you must have:

• Meeting SDK for iOS version 5.11.0 or later
  • Raw streaming by meeting host: Meeting SDK for iOS version 5.11.0 or later
  • Raw streaming by non-host participant: Meeting SDK for iOS version 5.12.8 or later
• Raw live streaming permission for the current user
• The meeting host has a Pro, Business, Education, or Enterprise account
• The meeting host is licensed

Streaming sequence diagram#

This sequence diagram shows the raw streaming flow.

Usage#

• Set up video raw data callbacks
• Set up audio raw data callbacks
• Request RawLiveStream permission
• Call StartRawLiveStream to start streaming raw data
• Call StopRawLiveStream to stop streaming raw data

Methods#

• Video raw data callbacks: onMobileRTCRender — triggered for each video frame
• Audio raw data callbacks
  • onMobileRTCOneWayAudioRawData  — per-user audio (no support for telephone participants)
  • onMobileRTCMixedAudioRawData  — session-wide audio (what meeting participants hear)
• StartRawLiveStream  — start raw streaming
• StopRawLiveStream  — stop raw streaming

Stream raw data from within the current meeting#

Follow these steps to stream either audio or video from inside a meeting.

1. Implement an instance of MobileRTCVideoRawDataDelegate.

   • In this class, respond to onMobileRTCRender to process the raw video data stored in the YUVRawDataI420 object.
   • Additionally, set up an MobileRTCAudioRawDataDelegate for the raw audio data.
   • Use one of two different callbacks to receive raw data onMobileRTCOneWayAudioRawData for per-user audio, or onMobileRTCMixedAudioRawData for meeting-wide audio.

   Note: Do not perform heavy operations from within the raw data callbacks. This may cause delayed or lost data.

2. Call StartRawLiveStream in MobileRTCMeetingService to trigger the live streaming consent dialog for all users, if the current user has permission to start the live streaming.

   • Listen for the onUserRawLiveStreamingStatusChanged callback to determine when the raw live streaming successfully starts.

3. Once the streaming starts, pass the MobileRTCVideoRawDataDelegate into the initWithDelegate method in MobileRTCRenderer.

   • For audio, pass your delegate to the MobileRTCAudioRawDataHelper class's initWithDelegate function.

4. To end the recording, call StopRawLiveStream in MobileRTCMeetingService.

   • Unsubscribe in any MobileRTCRenderer or MobileRTCAudioRawDataHelper instance you used.

Notify users when beginning a live stream#

When beginning a live stream inside a current meeting, the Meeting SDK displays a notice telling users that the following portion of the meeting will be streamed to another platform. This gives users an informed chance to opt out of being streamed on that new platform.

• MobileRTCRawLiveStreamInfo.userId - the name of the app opening the new streaming session, such as "AI Note taker by XYZ"
• MobileRTCRawLiveStreamInfo.broadcastUrl - URL of a landing page hosted by the app developer, OR the URL of the app's Zoom Marketplace page, such as https://example.com/description-page or https://marketplace.zoom.us/apps/XXXXXXexampleXXXX.

Request live streaming permission#

If you do not have live streaming permission, you can either request this permission from the meeting host during the meeting or request live-streaming permission through the OAuth app, so that the SDK client can start and stop raw streaming.

Request live streaming permission from the host#

To request live streaming permission from host during the meeting:

1. Call requestRawLiveStream.
2. Receive permission by checking callback onRawLiveStreamPrivilegeChanged.

Request live-streaming permission through an OAuth app#

To successfully request live-streaming permission through an OAuth app, your app must be:

• a public OAuth app on the Zoom Marketplace
• installed by the meeting host or by their account admin

1. Your OAuth app will request one of the two scopes:
   • Meeting_token:read:admin:live_streaming - account level, installed by admin
   • Meeting_token:read:live_streaming - user level, installed by host
2. Once your SDK app has the meeting ID it needs to join, it calls the "Get a meeting's join token for live streaming" REST API endpoint
3. Have the OAuth app pass the returned token to your Meeting SDK app.
4. Have the Meeting SDK app use the returned token to join the meeting with the app_privilege_token  parameter.

Notes#

• The usage or actual handling of the raw data may vary based on your use case.
• The SDK only provides the raw data. Implement only the raw data video or audio callbacks that you need.
• Performing any heavy handling within the callback delays the callback being triggered, which may result in loss of data.