VisionOS RTC support using Agora video SDK v4.2.6, visionOS special support

Full Changelog: https://github.com/AgoraIO/AgoraRtcEngine_iOS/compare/4.2.6...4.3.0

test for 4.3.0

4.2.6 release

4.2.3 release

v4.2.2 was released on July 27, 2023.

New features

1. Wildcard token

   This release introduces wildcard tokens. Agora supports setting the channel name used for generating a token as a wildcard character. The token generated can be used to join any channel if you use the same user id. In scenarios involving multiple channels, such as switching between different channels, using a wildcard token can avoid repeated application of tokens every time users joining a new channel, which reduces the pressure on your token server.

All 4.x SDKs support using wildcard tokens.

2. Preloading channels

   This release adds preloadchannel(bytoken:channelid:uid:) and preloadchannel(bytoken:channelid:useraccount:)) methods, which allows a user whose role is set as audience to preload channels before joining one. Calling the method can help shortening the time of joining a channel, thus reducing the time it takes for audience members to hear and see the host.

   When preloading more than one channels, Agora recommends that you use a wildcard token for preloading to avoid repeated application of tokens every time you joining a new channel, thus saving the time for switching between channels.

3. Customized background color of video canvas

   In this release, the backgroundColor member has been added to AgoraRtcVideoCanvas, which allows you to customize the background color of the video canvas when setting the properties of local or remote video display.

4. Video source types for local preview

   To allow users in selecting different types of video sources for local video preview, this release introduces startPreview(_:) and stopPreview(_:). You can call startPreview(_:) and specify the type of video source to be previewed by setting the sourceType parameter, and call stopPreview(_:) to stop the video preview.

The video source type specified in this method must match the video source type set in the AgoraRtcVideoCanvas of the setupLocalVideo(_:) method.

Improvements

1. Improved camera capture effect

   This release has improved camera capture effect in the following aspects:

   1. Support for camera exposure adjustment

      This release adds isCameraExposureSupported() to query whether the device supports exposure adjustment and setCameraExposureFactor(_:) to set the exposure ratio of the camera.

   2. Optimization of default camera selection

      Since this release, the default camera selection behavior of the SDK is aligned with that of the iOS system camera. If the device has multiple rear cameras, better shooting perspectives, zooming capabilities, low-light performance, and depth sensing can be achieved during video capture, thereby improving the quality of video capture.

2. Virtual Background Algorithm Upgrade

   This version has upgraded the portrait segmentation algorithm of the virtual background, which comprehensively improves the accuracy of portrait segmentation, the smoothness of the portrait edge with the virtual background, and the fit of the edge when the person moves. In addition, it optimizes the precision of the person's edge in scenarios such as meetings, offices, homes, and under backlight or weak light conditions.

3. Channel media relay

   The number of target channels for media relay has been increased to 6. When calling startOrUpdateChannelMediaRelay(_:) and startOrUpdateChannelMediaRelayEx(_:connection:), you can specify up to 6 target channels.

4. Enhancement in video codec query capability

   To improve the video codec query capability, this release adds the codecCapLevels member in AgoraVideoCodecCapInfo. After successfully calling queryCodecCapability(), you can obtain the hardware and software decoding capability levels of the device for H.264 and H.265 video formats through AgoraVideoCodecCapLevels.

This release includes the following additional improvements:

1. The SDK automacially adjusts the frame rate of the sending end based on the screen sharing scenario. Especially in document sharing scenarios, this feature avoids exceeding the expected video bitrate on the sending end to improve transmission efficiency and reduce network burden.
2. To help users understand the reasons for more types of remote video state changes, the AgoraVideoRemoteReason.codecNotSupport enumeration has been added to the remoteVideoStateChangedOfUid callback, indicating that the local video decoder does not support decoding the received remote video stream.

Issues fixed

This release fixed the following issues:

• Slow channel reconnection after the connection was interrupted due to network reasons.
• In screen sharing scenarios, the delay of seeing the shared screen was occasionally higher than expected on some devices.
• In custom video capturing scenarios, setBeautyEffectOptions, setLowlightEnhanceOptions, setVideoDenoiserOptions, and setColorEnhanceOptions could not load extensions automatically.

Improvements

This version improves the network transmission strategy, enhancing the smoothness of audio and video interactions.

To see more, check out the full release notes.

special build for activefence test

What’s new – key features

• Public API for AI Noise Suppression - to enable AI Noise Suppression feature and set different noise suppression modes to improve audio quality and performance.
• Use-case based setting for video conferencing - based on best practice, to help customers simplify and optimize their parameter settings to achieve the best video performance/end-user experience.
• Dynamic virtual background - a new sub-feature of Virtual Background, the dynamic virtual background feature adds more fun and provides higher flexibility for customers and their end users.
• Remote device synchronization - leverages the Network Time Protocol (NTP) to synchronize system clocks across a computer network, ensuring that all remote devices are synchronized. This is essential for use cases with high synchronization requirements, such as online chorus, lyrics progress synchronization, etc., providing users with a better collaborative experience.
• Screen sharing - this version mainly focuses on mobile platforms (Android/iOS). Improving horizontal and vertical screens to ensure a smooth switching experience for a better audience experience.

Other improvements

• Voice changer – more diverse effects are available now
• Custom audio and capture synchronization capabilities - accurately control the audio and video timing and ensure audio and video synchronization.

Release 4.0.1 fixes some bugs in the 4.0.0 release. See full release notes here:

https://docs.agora.io/en/video-calling/reference/release-notes?platform=ios#v

This version is identical to the 4.0.0-r.4 build.

Identical to 3.5.1 build. Frameworks are just hosted on GitHub release servers.