Skip to main content

Getting Started

Create your first AR Web experience using DeepAR

In this tutorial we will cover how to set up a simple web app from scratch to implement your first AR experience on the Web.

You will use DeepAR Web SDK to:

  • Show the camera preview.
  • Apply various AR effects:
    • Face masks
    • Filters
    • Beautification
    • Background removal

Prerequisites

To follow this tutorial you should have basic knowledge of creating web apps with Javascript and it's ecosystem. In this tutorial we are going to use:

  • NPM - Javascript package manager.
  • Webpack - Bundler for your javascript apps.

Don't worry if you don't have a lot of experience with these (or any experience at all). It is pretty straightforward to use and we will explain everything along.

Install node.js to be able to use NPM and Webpack if you haven't already.

Set up DeepAR account

DeepAR developer portal is used to:

  • Download DeepAR SDK.
  • Access documentation and examples.
  • Manage your projects and app license.

You need to set up a DeepAR account here https://developer.deepar.ai/.

DeepAR License Key

To deploy your DeepAR web apps it is required to use DeepAR license key.

🔥 DeepAR offers unlimited FREE licences for testing and small projects. 🔥

  1. On developer portal, go to projects and click + New Project.
  2. Enter the name of your project. For example "My First AR App".
  3. Choose a FREE plan.
  4. Under Web App choose + Add App.
  5. Enter the domain name under which you plan to deploy your app.

    ⚠️ IMPORTANT ⚠️
    You need to enter the DOMAIN NAME, not the URL where your app is running.
    For example, if your app is running on
    https://www.domain.com/try-on/glasses
    the domain you need to enter is www.domain.com.
    Be sure to leave out any protocol name (like https) and additional paths (like try-on/glasses).

Now you have a valid license key for your DeepAR web app. Note that you can delete existing or create new unlimited free projects.

Quickstart demo

If you just want to quickly test DeepAR tech and go through the implementation without coding, you can check out our quickstart example on GitHub: https://github.com/DeepARSDK/quickstart-web-js-npm. It is very similar, but more advanced, to this tutorial in the next sections.

Creating an AR web app from scratch

Let's get started with the business. Open a can of Coke and get ready for some coding.

Project setup

First, create a directory for the project.

mkdir ar-web-app
cd ar-web-app

Create a package.json file inside the directory.

{
"name": "ar-web-app",
"version": "1.0.0"
}

Install deepar npm package. Run next command.

npm install deepar

We also need to install Webpack and Webpack dev server.

npm install webpack webpack-cli webpack-dev-server --save-dev

Create a src directory in which we will put all the source code. And then create index.js which will be an entry point of the web app.

mkdir src
touch src/index.js

Let's create a configuration file for Webpack. Create webpack.config.js file and put next code.

const path = require('path');

module.exports = {
mode: 'production',
entry: './src/index.js',
output: {
filename: 'main.js',
path: path.resolve(__dirname, 'dist'),
clean: true,
},
target: 'web',
};

In the config we declared the src/index.js as entry point. The bundled code will be emitted in dist/main.js.

Now let's create an index.html to define the look of our simple web app. Place it in the public folder.

mkdir public
touch public/index.html

After all this is done, your project directory should look like this:

.
├── node_modules
├── public
│ └── index.thml
├── src
│ └── index.js
├── package-lock.json
├── package.json
└── webpack.config.js

One last thing we need to do is modify package.json. Let's put in the two scripts we are going to need for development.

"scripts": {
"build": "webpack && cp -r public/* dist",
"dev": "webpack serve --mode development --open --port 8888"
}

Now when you run npm run build command it will bundle the app with the Webpack and copy the contents of public directory into dist directory. When you run npm run dev it will start a local server for testing the web app. It will serv all the files from the dist directory.

Your final package.json should look like this.

{
"name": "ar-web-app",
"version": "1.0.0",
"scripts": {
"build": "webpack && cp -r public/* dist",
"dev": "webpack serve --mode development --open --port 8888"
},
"dependencies": {
"deepar": "^4.0.0"
},
"devDependencies": {
"webpack": "^5.74.0",
"webpack-cli": "^4.10.0",
"webpack-dev-server": "^4.11.1"
}
}

Camera preview

The fist thing we want to do is just show a plain camera preview in the web page. After that we can add cool AR masks and filters. Here is how to do it using DeepAR:

  1. Create a canvas element in HTML or Javascript.
  2. Initialize DeepAR object in Javascript
  3. Call startVideo method on DeepAR object.

To create a canvas and call our bundled script add this to public/index.html.

<!DOCTYPE html>

<html>
<head>
<title>AR Web App</title>
</head>
<body>
<canvas id="deepar-canvas" width="640" height="480"></canvas>
<script type="text/javascript" src="main.js"></script>
</body>
</html>

In the src/index.js import DeepAR class and DeepAR WebAssembly file.

import { DeepAR } from 'deepar';
import deeparWasm from 'deepar/wasm/deepar.wasm';
  • DeepAR is the class for interacting with the SDK. It can be used to:
    • Initialize the SDK.
    • Start camera preview.
    • Load different AR mask, filters and effects.
    • Take screenshots.
    • Record videos.
  • DeepAR SDK comes with it's WebAssembly file called deepar.wasm.
    • WebAssembly is the technology that allows DeepAR to run very fast in web apps.
    • It is mandatory to load this file and pass it to the SDK.

Now let's initialize DeepAR and start camera preview.

const canvas = document.getElementById('deepar-canvas');
const deepAR = new DeepAR({
licenseKey: 'your_license_key_here',
canvas: canvas,
deeparWasmPath: deeparWasm,
callbacks: {
onInitialize: () => {
deepAR.startVideo(true);
}
}
});
  • First we get the canvas element that we created in public/index.html.
  • Crete DeepAR object with new DeepAR({ ... }).
    • licenseKey - Pass the license key that you created earlier.
    • canvas - The canvas element where DeepAR will preview the camera and AR effects.
    • deeparWasmPath - Path to the deepar.wasm WebAssembly file.
    • callbacks - Here you define the callbacks called by DeepAR. See full list of callbacks here.
      • onInitialize - Callback called when DeepAR SDK is fully initialized.
        • deepAR.startVideo(true) - Start the camera preview by DeepAR and mirror the preview.

There is one last thing we need to do before testing the camera preview. We need to configure the Webpack so it knows how to load .wasm files. We are going to use asset modules feature with the type of resource asset/resource. Add the following to your webpack.config.js.

const path = require('path');

module.exports = {
// ...
module: {
rules: [
{
test: /\.(wasm)|(bin)|(obj)$/i,
include: [
path.resolve(__dirname, 'node_modules/deepar/'),
],
type: 'asset/resource',
},
],
},
};
  • Load only .wasm, .bin or .obj files.
  • They are loaded from the deepar npm node_modules folder.
  • Type asset/resource copes the imported file ino dist folder and gives the full URL path in JS code. So when you call import deeparWasm from 'deepar/wasm/deepar.wasm', the deepar.wasm file will be coped to dist folder and deeparWasm will be the path to that file.

Now you can finally test your web app. Run next commands.

npm run build
npm run dev

This should bundle your web app and run it on a local http server on port 8888. Open http://localhost:8888/ if it didn't open automatically.

You should see a camera preview in the browser (if you have a webcam).

Coding AR

It's time to implement some Augmented Reality into our app. One thing you need to know about AR masks, filters, effects and DeepAR is that all of it is encapsulated in effect files.

This is how DeepAR ecosystem works:

  1. With DeepAR Studio the AR effects are created. When the effect is finished it is exported as an effect file. You can also get effects from our free filter pack or buy them from DeepAR Store. You can modify or create completely new effects with DeepAR Studio.
  2. When you have an effect file (or as many of them as you like). Your web app can load an preview those effects with the DeepAR SDK.

For our web app let's use the free filter pack. Download and extract the zip file.

In your project directory create a folder where we are going to keep all our effects.

mkdir public/effects

Copy following files from the free filter pack to the public/effects directory.

  • Devil Neon Horns/Neon_Devil_horns.deepar
  • Galaxy Bacground/galaxy_background.deepar
  • Hope/Hope.deepar
  • Makeup Look Simple/MakeupLook.deepar
  • Ping Pong Minigame/Ping_Pong.deepar

DeepAR uses machine learning models for things like face tracking and background removal. We need to load those models into DeepAR SDK so it can use them to properly preview the effects. Add next imports to your src/index.js;

import faceTrackingModel from 'deepar/models/face/models-68-extreme.bin'
import segmentationModel from 'deepar/models/segmentation/segmentation-160x160-opt.bin';

Also add following code to properly load those models into the SDK.

const deepAR = new DeepAR({
// ...
segmentationConfig: {
modelPath: segmentationModel
}
});

deepAR.downloadFaceTrackingModel(faceTrackingModel);
  • segmentationConfig.modelPath - This is the model used for background removal.
  • downloadFaceTrackingModel - The method which loads the face tracking model.

Now add the button to your HTML in public/index.html. We will use it to trigger switching to the next AR effect in the library.

<!DOCTYPE html>

<html>
<head>
<title>AR Web App</title>
</head>
<body>
<canvas id="deepar-canvas" width="640" height="480"></canvas>
<button id="button">Next AR effect</button>
<script type="text/javascript" src="main.js"></script>
</body>
</html>

Add the following code to src/index.js. It implements the logic of switching the AR effects.

const effects = [
'./effects/Neon_Devil_horns.deepar',
'./effects/Hope.deepar',
'./effects/MakeupLook.deepar',
'./effects/Ping_Pong.deepar',
'./effects/galaxy_background.deepar'
];
let currentEffectIdx = -1;
const btn = document.getElementById('button');
btn.addEventListener('click', () => {
currentEffectIdx = (currentEffectIdx + 1) % effects.length;
const effect = effects[currentEffectIdx];
deepAR.switchEffect(0, "slot", effect);
});

NOTE: Use switchEffect to change the preview of an AR effect.

Now just rerun the npm commands to test your fist complete AR Web App!

npm run build
npm run dev