Skip to main content

Introduction

This section provides information on how to integrate the DeepAR SDK with popular frameworks for video-calling and streaming.

Use cases​

  • Background blur.
  • Background replacement (green screen).
  • Beautification.
  • Adding AR effects.
  • Camera preview postprocessing.

Overview​

In video-calling and streaming apps, DeepAR SDK serves as a video processing layer. Before camera frames are sent to the video-calling API for streaming, DeepAR processes them and outputs composited AR frames. DeepAR is only applied to the local (outgoing) video stream. Remote (incoming) video streams that also have AR applied are not processed. AR processing for remote streams is done locally on the device that sends out that particular stream. The local stream is presented on-screen either by DeepAR or by the streaming framework, that is an application implementation choice.

streaming workflow

Video resolutions​

When implementing a streaming or video-calling application, 3 different video resolutions need to be considered.

Camera resolution.​

  • Choose this resolution when initializing the camera.
  • This one is the most important for the performance since DeepAR takes more time to analyze larger images. The smaller the resolution, the better performance of the app will be.
  • It is strongly suggest you use 640x360 camera resolution.

DeepAR output resolution.​

  • This is the resolution of DeepAR output (camera + AR effect):
    • iOS: Resolution of the AR view.
    • Android: Resolution of the output surface.
    • Web: Resolution of the canvas.
  • This resolution impacts the final quality of the video stream the most.
  • It needs to be also 16:9 since that is what most streaming API expect.
  • The impact of this resolution on the performance is low.
  • It is suggested to use 1920x1080 DeepAR output.

Streaming resolution.​

  • This is the resolution of the video being streamed to other users.
  • The default for most APIs is 1280x720 and that is also the recommended resolution here,

Integrate DeepAR with streaming SDKa​

Amazon IVS with DeepAR

Amazon IVS with DeepAR

ACS with DeepAR

Guides for each platform​

iOS Offscreen rendering guide

Android Offscreen rendering guide

Web streaming guide

Demo apps​

These are the demo apps showcasing DeepAR SDK with video-calling and streaming frameworks.

iOS​

Android​

Web​