DeepAR Web is an augmented reality SDK that allows users to integrate advanced, Snapchat-like face lenses in the browser environment.
DeepAR Web supports:
In order to use the DeepAR Web SDK you need to set up a license key for your web app on developer.deepar.ai.
⚠️ The license key property is required both in a production and development (localhost) environment. Development sessions will not count towards your monthly active usage.
There are two main ways to get deepar.js in your JavaScript project: via script tags or by installing it from NPM and using a build tool like Parcel, WebPack, or Rollup.
Add the following code to an HTML file:
<html>
<head>
<!-- Ensure compatibility and performance on mobile -->
<meta name="viewport" content="width=device-width, initial-scale=1 "/>
<!-- Load deepar.js -->
<script src='https://cdn.jsdelivr.net/npm/deepar/js/deepar.js'> </script>
</head>
<body>
<!-- Div element where AR preview will be inserted -->
<div style='width: 640px; height: 360px' id='deepar-div'></div>
<!-- Initialize DeepAR and load AR effect/filter -->
<script>
(async function() {
const deepAR = await deepar.initialize({
licenseKey: 'your_license_key_here',
previewElement: document.querySelector('#deepar-div'),
effect: 'https://cdn.jsdelivr.net/npm/deepar/effects/aviators'
});
})();
</script>
</body>
</html>
Alternatively, you can import DeepAR as an ES6 module.
Via <script type='module'>.
<script type='module'>
import * as deepar from 'https://cdn.jsdelivr.net/npm/deepar/js/deepar.esm.js';
</script>
Or via dynamic import.
const deepar = await import('https://cdn.jsdelivr.net/npm/deepar/js/deepar.esm.js');
Add deepar.js to your project using yarn or npm.
Note: Because we use ES2017 syntax (such as import), this workflow assumes you are using a modern browser or a bundler/transpiler to convert your code to something older browsers understand. However, you are free to use any build tool that you prefer.
import * as deepar from 'deepar';
const deepAR = await deepar.initialize({
licenseKey: 'your_license_key_here',
previewElement: document.querySelector('#deepar-canvas'),
effect: 'https://cdn.jsdelivr.net/npm/deepar/effects/aviators'
});
AR filters are represented by effect files in DeepAR. You can load them to preview the effect.
Places you can get DeepAR effects:
Load an effect using the switchEffect method:
await deepAR.switchEffect('path/to/effect/alien');
Take a screenshot.
// The image is a data url.
const image = await deepAR.takeScreenshot();
Record a video.
await deepAR.startVideoRecording();
// Video is now recording...
// When user is ready to stop recording.
const video = await deepAR.finishVideoRecording();
Enable background blur with strength 5.
await deepAR.backgroundBlur(true, 5)
This is also known as background removal or green screen effect.
Enable background replacement with an image of a sunny beach.
await deepAR.backgroundReplacement(true, 'images/sunny_beach.png')
DeepAR has some callbacks you can implement for additional informations. For example, to check if face is detected in the scene.
await deepAR.switchEffect('path/to/face-filter');
deepAR.callbacks.onFaceTracked = function(faceDataArr) {
if (faceDataArr[0].detected) {
console.log("Face is detected!");
}
};
See all available callbacks here DeepARCallbacks.
To remove callback if you don't need it anymore.
delete deepAR.callbacks.onFaceTracked;
DeepAR will by default use the user facing camera. It will also ask the camera permission from the user.
Use the back camera on the phones:
const deepAR = await deepar.initialize({
// ...
additionalOptions: {
cameraConfig: {
facingMode: 'environment'
}
}
});
Set up your own camera or custom video source:
const deepAR = await deepar.initialize({
// ...
additionalOptions: {
cameraConfig: {
disableDefaultCamera: true
}
}
});
// HTMLVideoElement that can contain camera or any video.
const video = ...;
deepAR.setVideoElement(video, true);
Initialize DeepAR but start the camera later. This is used when you do not want to ask the camera permission right away.
const deepAR = await deepar.initialize({
// ...
additionalOptions: {
cameraConfig: {
disableDefaultCamera: true
}
}
});
// When you want to ask the camera permission and start the camera.
await deepAR.startCamera();
In cases when the camera aspect ratio does not match the screen aspec ratio it is possible to rotate the camera input to achieve better camera quality. This is usually the case for in-store AR screens which have portrait screens, but use standard landscape webcams. To improve camera quality in those cases, rotate the camera physically 90° and call the next API.
const deepAR = await deepar.initialize({
// ...
additionalOptions: {
cameraConfig: {
rotation: 90 // or 270
}
}
});
If you are not using the default DeepAR camera, but want to provide your own video source and still want to apply the rotation.
deepAR.setVideoRotation(90) // or 270
In some cases, it is needed to have face tracking at longer distances, not just at hand-held distances. For that use-case, DeepAR comes with upgraded face tracking model which works better at distances and is more accurate in general. It can also be used if more accurate beauty effects are needed.
⚠️ This face tracking model works only for single person face tracking, ie. it cannot track multiple faces.
To enable this face tracking mode, pass enableFaceTrackingCnn hint at initialization.
const deepAR = await deepar.initialize({
// ...
additionalOptions: {
hint: "enableFaceTrackingCnn"
}
});
Create canvas directly in the HTML:
<canvas width='1280' height='720'></canvas>
Or you can create it in Javascript.
const canvas = document.createElement('canvas');
// Set canvas size, accounting screen resolution (to make it look 🤌 even on high definition displays)
const canvasSize = { width: 640, height: 360 };
const dpr = window.devicePixelRatio || 1;
canvas.style.maxWidth = `${canvasSize.width}px`;
canvas.style.maxHeight = `${canvasSize.height}px`;
canvas.width = Math.floor(canvasSize.width * dpr);
canvas.height = Math.floor(canvasSize.height * dpr);
⚠️ Note: Be sure to set
widthandheightproperties of thecanvas!
You can always change the canvas dimensions and they don't have to match the input video resolution. DeepAR will fit the input camera/video stream correctly to any canvas size.
You pass the canvas when initializing DeepAR.
await deepar.initialize({
canvas: canvas,
// ...
});
Apart from the main deepar.js file and AR effect files, DeepAR uses additional files like:
Some of them are required and will be downloaded every time. The others will be lazy downloaded when/if needed.
⚠️ DeepAR will by default automatically fetch all additional resources from the JsDelivr CDN.
Fetching files from JsDelivr is not recommended if you care about download speeds of DeepAR Web resources. This is because the files on JsDelivr are not compressed.
To optimize download speeds, all the DeepAR files should be compressed. It is recommended to serve DeepAR files from your own server or some CDN which supports file compression.
If it is supported, you should use GZip or Brotli compression on all DeepAR files which will significantly reduce the SDK size. Check out your server/CDN options for compressing files.
DeepAR Web can be downloaded from DeepAR Developer Portal. But since most users will install DeepAR through NPM, follow the next instructions.
It is recommended to copy ./node_modules/deepar directory which contains all the DeepAR
files into your distribution (dist) folder.
You can use rootPath to set a path to the custom root of the DeepAR SDK. All additional files
that need to be fetched by DeepAR will be resolved against the given rootPath.
const deepAR = await deepar.initialize({
// ...
rootPath: 'path/to/root/deepar/directory'
});
It is recommended to setup the copying of the DeepAR directory as part of you bundler build scripts, in case you ever need to updated to a newer version of DeepAR.
Another option, instead of using the DeepAR directory and copying it as-is, is to specify a path to each file. The advantage of this is that you can use the bundler to keep track of your files.
For example, if using Webpack, you can use it's asset modules to import all the files needed.
Pass the file paths in additionalOptions.
const deepAR = await deepar.initialize({
// ...
additinalOptions: {
faceTrackingConfig: {
modelPath: 'path/to/deepar/models/face/models-68-extreme.bin'
},
segmentationConfig: {
modelPath: 'path/to/deepar/models/segmentation/segmentation-160x160-opt.bin'
},
deeparWasmPath: 'path/to/deepar/wasm/deepar.wasm'
}
});
Generated using TypeDoc