Video

You can control user video for an ongoing session using the methods provided by the Video SDK. Using the SDK functions, users in a session can be provided with controls for managing their video. Session hosts can manage their own video as well the video of other users in the session.

Some common features that you might want to include in your app could be an option to enable or disable a user's video. Prior to changing the video control, you may also need to obtain data about the state of each users' present it in the UI of your app.

Start video

If the user's video is not already enabled, you can start video using startVideo, provided by the ZMVideoSDKVideoHelper:

// Get the ZMVideoSDKVideoHelper to perform video actions.
if let videoHelper = ZMVideoSDK.shared()?.getVideoHelper() {
    // Start local User\'s video.
    videoHelper.startVideo()
}

// Get the ZMVideoSDKVideoHelper to perform video actions.
ZMVideoSDKVideoHelper *videoHelper = [[ZMVideoSDK sharedZMVideoSDK] getVideoHelper];
if (videoHelper) {
    // Start local User\'s video.
    [videoHelper startVideo];
}

Stop video

To stop displaying the video of the user, use stopVideo:

// Get the ZMVideoSDKVideoHelper to perform video actions.
if let videoHelper = ZMVideoSDK.shared()?.getVideoHelper() {
    // Stop local User's video.
    videoHelper.stopVideo()
}

// Get the ZMVideoSDKVideoHelper to perform video actions.
ZMVideoSDKVideoHelper *videoHelper = [[ZMVideoSDK sharedZMVideoSDK] getVideoHelper];
if (videoHelper) {
    // Stop local User's video.
    [videoHelper stopVideo];
}

Render video

The Video SDK allows you to render the video of each user who joins a session.

  1. Retrieve the ZMVideoSDKUser object related to each user as they join the session.
  2. Obtain the ZMVideoSDKRawDataPipe from each user whose stream you'd like to render.
  3. Subscribe to their video pipe.
  4. Listen for the user's video frames.

Retrieve user

The ZMVideoSDKUserHelper helps with retrieval of user information. See Sessions for more details on user management.

To be notified when users join a session, you can use onUserJoin within ZMVideoSDKDelegate:

func onUserJoin(_ userHelper: ZMVideoSDKUserHelper!, userList userArray: [ZMVideoSDKUser]!) {
    // UserArray contains the new users.
    // Use helper to perform actions on a user.
    if let userArray = userArray {
        for user in userArray {
            print(user)
        }
    }
}

- (void)onUserJoin:(ZMVideoSDKUserHelper*)userHelper userList:(NSArray<ZMVideoSDKUser >)userArray {
    // UserArray contains the new users.
    // Use helper to perform actions on a user.
    for (int i = 0; i < userArray.count; i++) {
        NSLog(@"%@", userArray[i]);
    }
}

Subscribe to a user's video pipe

Now that you have access to each user represented by an ZMVideoSDKUser object, you must retrieve the ZMVideoSDKRawDataPipe of each user whose video you would like to render and to subscribe to that user's video pipe with your implementation of ZMVideoSDKRawDataPipeDelegate.

func onRawDataFrameReceived(_ rawData: ZMVideoSDKYUVRawDataI420!) {
    // Access the video data here.
}
let pipe = user.getVideoPipe()
pipe?.subscribe(ZMVideoSDKResolution_720P, listener: self);

- (void) onRawDataFrameRecieved: (ZMVideoSDKYUVRawDataI420 *) rawData {
    // Access the video data here.
}
ZMVideoSDKRawDataPipe pipe = [user getVideoPipe];
[pipe subscribe:ZMVideoSDKResolution_720P listener:self];

Send raw video data

You may also choose to send video data after modifying it (for example, adding a virtual background or filter) by using sendVideoFrame within ZMVideoSDKVideoSender. To obtain an ZMVideoSDKVideoSender you must assign a ZMVideoSDKVideoSource. The onInitialize within ZMVideoSDKVideoSource will provide an ZMVideoSDKVideoSender.

func onInitialize(_ sender: ZMVideoSDKVideoSender!, supportedCapbilityList supportedCapList: [ZMVideoSDKVideoCapability]!, suggestCapbility suggestCap: ZMVideoSDKVideoCapability!) {
    // Store video rawdata sender.
    self.rawDataSender = rawDataSender
}
// Call sendVideoFrame to send a frame buffer of raw data.
self.rawDataSender?.sendVideoFrame(frameBuffer, width: width, height: height, frameLength: frameLength, rotation: rotation)

- (void) onInitialize:(ZMVideoSDKVideoSender *)sender supportedCapbilityList:(NSArray<ZMVideoSDKVideoCapability *> *)supportedCapList suggestCapbility:(ZMVideoSDKVideoCapability *)suggestCap {
    // Store video rawdata sender.
    self.videoRawdataSender = rawDataSender;
}
//Call sendVideoFrame to send a frame buffer of raw data.
[self.videoRawdataSender sendVideoFrame:frameBuffer width:width height:height frameLength:frameLength rotation:rotation];

Preprocess raw video

You can preprocess raw video data using onPreProcessRawData within ZMVideoSDKVideoSourcePreProcessor.

func onPreProcessRawData(_ rawData: ZMVideoSDKYUVProcessDataI420!) {
    // Perform preprocess actions here.
}

- (void) onPreProcessRawData:(ZMVideoSDKYUVProcessDataI420 *)rawData {
    // Perform preprocess actions here.
}

Check the video status of a user

Similar to audio, you can control video on a per-user basis. First check the current video status of the user:

// Get videoStatus for User.
let videoStatus = user.videoStatus()
// Check if User's video is on.
let isVideoOn = user.getVideoPipe().getVideoStatus().isOn()

// Get videoStatus for User.
ZMVideoSDKVideoStatus *videoStatus = [user videoStatus];
// Check if User's video is on.
BOOL isVideoOn = [[[user getVideoPipe] getVideoStatus] isOn];

Use virtual background

Use ZMVideoSDKVideoHelper to add, get, set, and remove virtual backgrounds.

To add virtual backgrounds:

  1. Use addVirtualBackgroundItem to add a virtual background item.
  2. Use getVirtualBackgroundItemList to get a list of available virtual background items.
  3. Choose a virtual background item from the list and use setVirtualBackgroundItem to set the virtual background.

Use removeVirtualBackgroundItem to remove the virtual background or set the virtual background to None. Use getSelectedVirtualBackgroundItem to get the current selected background.

// Add virtual background item
func addVirtualBackgroundItem(_ imagePath: String?, imageItem: ZMVideoSDKVirtualBackgroundItem?) -> ZMVideoSDKErrors {
}
// Remove virtual background item
func remove(_ imageItem: ZMVideoSDKVirtualBackgroundItem?) -> ZMVideoSDKErrors {
}
// Get virtual background item list
func getVirtualBackgroundItemList() -> [ZMVideoSDKVirtualBackgroundItem]? {
}
// Set virtual background item
func setVirtualBackgroundItem(_ imageItem: ZMVideoSDKVirtualBackgroundItem?) -> ZMVideoSDKErrors {
}
// Get selected virtual background
func getSelectedVirtualBackgroundItem() -> ZMVideoSDKVirtualBackgroundItem? {
}

// Add virtual background item
- (ZMVideoSDKErrors) addVirtualBackgroundItem: (NSString *) imagePath
imageItem: (ZMVideoSDKVirtualBackgroundItem *_Nullable *_Nonnull) imageItem
// Remove virtual background item
- (ZMVideoSDKErrors) removeVirtualBackgroundItem: (ZMVideoSDKVirtualBackgroundItem *) imageItem
// Get virtual background item list
- (NSArray< ZMVideoSDKVirtualBackgroundItem * > *) getVirtualBackgroundItemList
// Set virtual background item
- (ZMVideoSDKErrors) setVirtualBackgroundItem: (ZMVideoSDKVirtualBackgroundItem *) imageItem
// Get selected virtual background
- (ZMVideoSDKVirtualBackgroundItem *) getSelectedVirtualBackgroundItem

See ZMVideoSDKVideoHelper Class Reference for details.

Video quality preference

When network bandwidth is limited, you can adjust video quality preferences between resolution and frame rate. For example, you can choose to preserve the video sharpness or smoothness. If bandwidth is not a concern, you can receive the best quality video using a high video resolution and maximum frame rate.

Set your video quality preferences with the ZMVideoSDKPreferenceSetting.

Video preference modes

Choose from the following modes, depending on what you'd like to prioritize:

  • Balance mode. Zoom will do what is best under the current bandwidth situation and make adjustments as needed. You don't need to set any additional parameters for this mode. This mode is suitable for video conference usage. This is the default preference.
  • Smoothness mode. Preserves the frame rate as much as possible. If network bandwidth degrades, Zoom will sacrifice video resolution to preserve the frame rate. This prioritizes a smooth video frame transition.
  • Sharpness mode. Preserves the resolution as much as possible. If network bandwidth degrades, Zoom will sacrifice the frame rate to preserve video resolution. This prioritizes a sharp video image.
  • Custom mode. Allows you to provide the minimum and maximum frame rate. Use this mode if you have an understanding of your network behavior and a clear idea of how to adjust the frame rate to achieve the desired video quality. You can also use this mode to influence bandwidth usage by increasing or decreasing the maximum frame rate setting. Given a resolution, a lower maximum frame rate results in less bandwidth usage. Note that if the bandwidth cannot be sustained by following the minimum and maximum frame rates, the system will drop down to the next lower resolution.

See the reference documentation for details.

Use cases

Video quality preferences are useful when the primary focus is not the image of a person attending the meeting, but on other video images where smoothness or sharpness is preferred.

Aside from video conferencing, you may want to set video quality preferences for:

  • Transmission of medical X-ray images.
  • Transmission of video captured by endoscopic cameras during a medical operation.
  • Live sporting event broadcasts, such as co-watching or broadcasting a football game.

Callbacks

The following is an example video callback.

Get notified when a user's video status has changed

func onUserVideoStatusChanged(_ videoHelper: ZMVideoSDKVideoHelper!, userList userArray: [ZMVideoSDKUser]!) {
    // UserArray contains the users that had a video status change.
    // Use helper to perform actions on a user.
    if let userArray = userArray {
        for user in userArray {
            // Use to check if a user\'s video is on or off.
            if ((user.getVideoStatus()?.isOn) == true) {
                // User's video is on.
            }
        }
    }
}

- (void)onUserVideoStatusChanged:(ZMVideoSDKVideoHelper*)videoHelper userList:(NSArray*)userArray {
    // UserArray contains the users that had a video status change.
    // Use helper to perform actions on a user.
    for (int i = 0; i < userArray.count; i++) {
        if (userArray[i]) {
            // Use to check if a user\'s video is on or off.
            if ([userArray[i] videoStatus].isOn) {
                // User's video is on.
            }
        }
    }
}