Skip to main content

DeepAR AR Entertainment

About DeepAR​

DeepAR SDK​

DeepAR’s all-in-one AR SDK is built for Web, mobile (iOS and Android), and desktop (macOS currently, Windows coming soon) platforms. Our powerful SDK supports highly detailed face tracking, body tracking, and animation for hyper-realistic 3D face filters, masks, and effects.

The SDK can seamlessly integrate with popular live-streaming video stacks like Amazon Interactive Video Service (AWS IVS), Agora, and Vonage.

DeepAR Studio​

DeepAR Studio is our tool for AR asset creation within the DeepAR SDK. It allows creators to produce massive effects driven by facial movements, body movements, emotions, and even hair shapes.

DeepAR for Entertainment​

DeepAR combines machine vision and deep learning tracking with video game quality rendering technology to provide realistic and fun AR face filters, masks and effects for any website or app.

Our team brings together some of the best experiences from 3D gaming, design, film, and animation to create and power your chosen AR entertainment solution.

👀 73% of consumers show interest in experiencing AR at events and conferences. (Source)

🎸 79% of consumers are interested in AR for entertainment such as concerts and sports events. (Source)

😊 8 in 10 consumers agree AR experiences make them feel happy. (Source)

Impact and Benefits of DeepAR’s Augmented Reality Technology​

The global augmented reality market size is anticipated to reach USD 597.54 billion by 2030, expanding at a CAGR of 40.9% from 2022 to 2030. (Source)

  • DeepAR gives audiences a new dimension to entertainment (Gaming, Film, Travel, Live Music, TV and other industries)

  • Our solution is scalable across 50,000+ SKUs and millions of end users

  • Our technology is straightforward to integrate and manage

  • We offer a highly competitive pricing model based on usage similar to AWS and not old-school enterprise per-seat or per-platform licensing models

Composable, Compatible, Layerable​

DeepAR's effects and components can integrate and work together so you can create a wide variety of fun and original AR experiences.

Our powerful technology allows you to track multiple faces at the same time and add an array of AR effects in real-time.

Face Tracking​

The Reference Mask Model is the underlying model that is driven by DeepAR Face Tracking technology. Users of the SDK use it as a starting point in creating their own assets in 3D modeling tools (like Blender or Maya).

Geometry (vertices) and deformators (bones and blend shapes) are used to drive the user-created masks. The reference model has a geometry that consists of 730 vertices, all of which are driven by the underlying face-tracking technology.

Next Generation Face Tracking - End of November 2022​

  • We track 22 points on the face in our face tracking. This is a hybrid tracking technology that is super efficient and optimized, but with an extra level of detail that blend-shapes provide.

  • Recognition: Can recognize 52 facial micro-expressions and recognize how intense they are. These expressions increase the accuracy of DeepAR face tracking and also provide developers with a range of triggers to use with scripting.

Face Morphing/ Face Distortion​

Face Morphing effects utilize blend shape as geometry deformators so users can create amusing effects.


Platform Compatibility​

AndroidiOSmacOSWeb
checkcheckcheckcheck

Device Compatibility​

Devices starting from iOS 10.0+ / Android 6.0 or higher with a 1280x720 camera (recommended).

Concepts​

  • Blend Shapes

  • Shading - Camera Multiply Shader, Matcap Shader

  • Skin Sampling

  • Rigging

Beautification - Beauty & Makeup​

Users can add beautifying and makeup filters, lenses, and effects. These include skin smoothing, face shaping, and face makeup such as lipstick, eyeshadow, and foundation.


Platform Compatibility - Beauty & Makeup​

AndroidiOSmacOSWeb
checkcheckcheckcheck

Device Compatibility - Beauty & Makeup​

Devices starting from iOS 10.0+ / Android 6.0 or higher with a 1280x720 camera (recommended).

Concepts - Beauty & Makeup​

  • 2D Tracking - Eyes, Lips

  • Shading - Unlit Shader

  • Beauty Shader - Skin Smoothing

  • Blend Shapes - Face Morpher

Animation​

This type of AR filters and effects are triggered by specific movements on parts of the users face such as open mouth, smile, raised eyebrows, blinking.


Platform Compatibility - Animation​

AndroidiOSmacOSWeb
checkcheckcheckcheck

Device Compatibility - Animation​

Devices starting from iOS 10.0+ / Android 6.0 or higher with a 1280x720 camera (recommended).

Concepts - Animation​

  • Animation State / Triggers

  • Video Textures

  • Physics

Avatars​

Avatars are creative 3D whole face masks with responsive emotions and facial movements.

Our technology tracks each eyebrow movement, jaw and mouth movement, and nose movement.


Platform Compatibility - Avatars​

AndroidiOSmacOSWeb
checkcheck✕✕

Device Compatibility - Avatars​

Devices starting from iOS 10.0+ / Android 6.0 or higher with a 1280x720 camera (recommended).

Concepts - Avatars​

  • Blend Shapes

  • Emotions

  • Rigging

  • Shading - PBR, Unlit, MATCAP

3D Objects​

Users can add fun 3D objects of all kinds on or around their faces and heads such as NFL helmets, sparkly glasses, bunny ears and anything else they can imagine!


Platform Compatibility - 3D Objects​

AndroidiOSmacOSWeb
checkcheckcheckcheck

Device Compatibility - 3D Objects​

Devices starting from iOS 10.0+ / Android 6.0 or higher with a 1280x720 camera (recommended).

Concepts - 3D Objects​

  • Occluder

  • Shading - Camera Multiply Shader

  • Pendulum / Physics

Virtual Try-On​

This asset takes the online shopping experience to the next level. Users can try-on ultra-realistic virtual makeup, hair color, shoes, watches, wrist jewelry, and glasses in real-time.


Platform Compatibility - Virtual Try-On​

AndroidiOSmacOSWeb
checkcheckcheckcheck

Device Compatibility - Virtual Try-On​

Devices starting from iOS 10.0+ / Android 6.0 or higher with a 1280x720 camera (recommended).

Concepts- Virtual Try-On​

  • Face Tracking, Body Tracking, Foot Tracking, Wrist Tracking

  • Occluder

  • Shading - Camera Multiply Shader

AR Mini-Game Filters​

Users can play interactive AR mini-games and easily control the game with their facial expressions or tapping the phone screen.

Platform Compatibility - AR Mini-Game Filters​

AndroidiOSmacOSWeb
checkcheckcheckcheck

Device Compatibility - AR Mini-Game Filters​

Devices starting from iOS 10.0+ / Android 6.0 or higher with a 1280x720 camera (recommended).

Concepts - AR Mini-Game Filters​

  • Physics

  • Rigging

  • Scripting

  • Animation editor

  • Colliders

  • Particles

Face Swap & Face Insertion​

Users can insert their faces into all kinds of objects or swap between each other's faces.


Platform Compatibility - Face Swap & Face Insertion​

AndroidiOSmacOSWeb
checkcheckcheckcheck

Device Compatibility - Face Swap & Face Insertion​

Devices starting from iOS 10.0+ / Android 6.0 or higher with a 1280x720 camera (recommended).

Concepts - Face Swap & Face Insertion​

  • Rigging

  • Shading

  • Video Textures

  • Animation

  • Face Position Component / Face ID

Segmentation​

Hair Color​

Our hair detection allows users to change their hair to any funky color combo. DeepAR Hair Color works on any hair color, length, and shape.


Platform Compatibility - Hair Color​

AndroidiOSmacOSWeb
Coming Q1 2023check✕Coming Q1 2023

Device Compatibility - Hair Color​

Devices starting from iOS 10.0+ / Android 6.0 or higher with a 1280x720 camera (recommended).

Concepts - Hair Color​

  • Hair Mask

  • LUTs (Overlay Filters)

Background Replacement & Background Blur​

This tool includes background removal, background replacement, and background blur to drive engaging and private video chats and video streaming.


Platform Compatibility - Background Replacement & Background Blur​

AndroidiOSmacOSWeb
checkcheckcheckcheck

Device Compatibility - Background Replacement & Background Blur​

Devices starting from iOS 10.0+ / Android 6.0 or higher with a 1280x720 camera (recommended).

Concepts - Background Replacement & Background Blur​

  • Foreground Mask Texture

  • LUTs (Overlay Filters)

  • Video Textures

  • Camera Quads (fixed assets in camera space for underlay layer)

Wishlist​

  • Skin Segmentation

Post Processing​

Post-processing effects are applied over the entire frame (including all the objects rendered).

The existing media could be processed with LUTs or any other post processing shader. Anything from black and white or sepia filters to other snazzy combinations.


Platform Compatibility - Post Processing​

AndroidiOSmacOSWeb
checkcheckcheckcheck

Device Compatibility - Post Processing​

Devices starting from iOS 10.0+ / Android 6.0 or higher with a 1280x720 camera (recommended).

Concepts - Post Processing​

  • Video Texture
  • Blending Modes
  • Camera Quad
  • Render Textures
  • Particles
  • Overlay Layers
  • Post Processing Layer

Overlays​

Video Overlays​

Users can add a video layer in camera space to play over the top of the scene. This can be used to create effects like snowfall, rain, confetti and much more.


Platform Compatibility - Video Overlays​

AndroidiOSmacOSWeb
checkcheckcheckcheck

Device Compatibility - Video Overlays​

Devices starting from iOS 10.0+ / Android 6.0 or higher with a 1280x720 camera (recommended).

Concepts - Video Overlays​

  • Video Texture
  • Blending Modes
  • Camera Quad
  • Overlay Layers
  • Post Processing Layer

Static Overlays​

Users can add an image in camera space over the top of the scene. This can be used to add a frame or other similar effects.

Static Overlay

Platform Compatibility - Static Overlays​

AndroidiOSmacOSWeb
checkcheckcheckcheck

Concepts - Static Overlays​

  • Blending Modes

  • Camera Quad

  • Overlay Layers

  • Post Processing Layer