Class DeepAR

Main class for interacting with DeepAR SDK. To get a DeepAR object call initialize.

Hierarchy

  • DeepAR

Properties

callbacks: DeepARCallbacks = {}

Callbacks property is used to add/remove/change callbacks called by the DeepAR SDK. See list of all callbacks at DeepARCallbacks.

Example: To add/change certain callback:

let deepAR = deepar.initialize({...});
deepAR.callbacks.onFaceVisibilityChanged = () => {
// do something
};

To remove certain callback:

deepAR.callbacks.onFaceTracked = undefined;
ScriptingAPI: ScriptingAPI

Scripting API property used to access all the scripting interop methods.

Example

let scriptingVariable = deepAR.ScriptingAPI.getStringVar('variableName');

Methods

  • Switch the AR effect for preview.

    Parameters

    • effect: string | ArrayBufferLike

      A path (URL) to the AR effect file or an ArrayBuffer object of an already fetched AR effect file.

    • Optional effectOptions: {
          slot?: string;
          face?: number;
          envmap?: string | ArrayBufferLike;
      }

      Effect options.

      • Optional slot?: string

        Default value is "DEFAULT_SLOT" if slot is not given. Slot is a container for a given AR effect. When replacing the already loaded AR effect, call switchEffect with that same slot. To remove AR effect entirely, call clearEffect with the same slot.

      • Optional face?: number

        If AR effect is face filter, select to which face to attach it to. The value should be between 0 and 3. Default value is 0.

      • Optional envmap?: string | ArrayBufferLike

    Returns Promise<void>

    Throws

    SwitchEffectCanceled If the switch effect is canceled by something.

    Example

    // Switch filter 1.
    await deepAR.switchEffect('url/path/to/filter1');
    // Later switch fo filter 2.
    await deepAR.switchEffect('url/path/to/filter2');

    // Remove the current filter.
    deepAR.clearEffect();

    // Put two filters at the same time.
    await deepAR.switchEffect('url/path/to/backgroundReplacement', {slot: 'background'});
    await deepAR.switchEffect('url/path/to/glasses', {slot: 'faceMask'});
    // Replace the glasses filter.
    await deepAR.switchEffect('url/path/to/glasses2', {slot: 'faceMask'});
    // Remove those filters.
    deepAR.clearEffect('background');
    deepAR.clearEffect('faceMask');

    // Load filters for two people.
    await deepAR.switchEffect('url/path/to/faceMask1', {face: 0, slot: 'mask1'});
    await deepAR.switchEffect('url/path/to/faceMask2', {face: 1, slot: 'mask2'});
    // Clear effect for the second person.
    deepAR.clearEffect('mask2');
  • Clears the given slot of any loaded effects.

    Parameters

    • Optional slot: string

      The effect slot name. Default is "DEFAULT_SLOT".

    Returns void

  • Captures a screenshot of the current screen.

    Returns Promise<string>

    Data URL promise containing the image in data:image/png format.

  • Start video recording of the DeepAR preview.

    To stop video recording call finishVideoRecording. By default, the audio is not recorded. To record audio set the recordAudio parameter. Note that if the user did not give the microphone permission, the recording will not start until the permission is granted. If permission is denied, the function will throw. The recorded video will be mp4 on all browsers except Firefox where it will be webm. Audio recording is currently not available on Android.

    Parameters

    • Optional options: {
          recordAudio?: boolean;
          audioTrack?: MediaStreamAudioTrack;
          audioBitRate?: number;
          audioSampleRate?: number;
          videoBitRate?: number;
          videoFrameRate?: number;
      }

      Parameters that specify the format of recorded videos

      • Optional recordAudio?: boolean

        If set, microphone sound will be recorded as well. If this parameter is set, options.audioTrack is ignored.

      • Optional audioTrack?: MediaStreamAudioTrack

        If passed, this audio track is going to be recorded.

      • Optional audioBitRate?: number

        Sets audio bit rate. By default 128000.

      • Optional audioSampleRate?: number

        Set audio sample rate. On firefox it is 48k. On other browsers defaults to audioTrack.getCapabilities().sampleRate.max or to the value passed here.

      • Optional videoBitRate?: number

        Sets video bit rate. By default 4000000.

      • Optional videoFrameRate?: number

        Sets video frame rate. By default 30.

    Returns Promise<void>

  • Stops the video recording and returns a video blob.

    Returns Promise<Blob>

    A promise resolving to Blob of a video.

  • Enable background blur.

    Background blur is usually used in video calling use cases.

    Parameters

    • enable: boolean

      Boolean indicating whether to enable or disable the background blur effect.

    • strength: number

      Blur strength. Integer number in range 1-10.

    Returns Promise<void>

  • Enable background replacement (also known as background removal or green screen effect).

    Parameters

    • enable: boolean

      Boolean indicating whether to enable or disable the background replacement effect.

    • image: string

      The URL of the image to be used as the background.

    • mode: RequestMode = ...

      Request mode used while fetching the image.

    Returns Promise<void>

  • Starts the camera preview. By default, the camera will be user facing. The returned promise will resolve after the camera starts or it will reject if camera permission was denied.

    Parameters

    • Optional cameraOptions: {
          mirror?: boolean;
          mediaStreamConstraints?: MediaStreamConstraints;
          cameraPermissionAsked?: (() => void);
          cameraPermissionGranted?: (() => void);
      }

      Camera options.

      • Optional mirror?: boolean

        Mirror the camera horizontally. True by default.

      • Optional mediaStreamConstraints?: MediaStreamConstraints

        Options passed to MediaDevices.getUserMedia(). The default is the user facing camera.

      • Optional cameraPermissionAsked?: (() => void)
          • (): void
          • Callback called when camera permission is asked.

            Returns void

      • Optional cameraPermissionGranted?: (() => void)
          • (): void
          • Callback called when camera permission is granted.

            Returns void

    Returns Promise<void>

  • Stops the camera preview.

    Returns void

  • Stops the camera preview or custom video preview set by setVideoElement.

    Returns void

  • Used to pass the HTMLVideoElement to the DeepAR SDK. The SDK will use this video as camera source. This method should be used instead of startCamera when you want to handle getUserMedia outside the SDK or you need to apply the masks to any video stream. To disable automatic camera preview by DeepAR:

    const deepAR = deepar.initialize({
    // ...
    additionalOptions: {
    cameraConfig: {
    disableDefaultCamera: true
    }
    }
    });

    Parameters

    • videoElement: HTMLVideoElement

      Video element.

    • mirror: boolean

      Mirror the video horizontally.

    Returns void

  • Change preview element.

    If DeepAR is initialized with previewElement then this method can be used to change the preview element.

    Parameters

    • newPreviewElement: HTMLElement

      The previewElement to change to.

    Returns void

  • Shutdown the DeepAR SDK and release all resources associated with it. It is invalid to call any function from this DeepAR object after shutdown. After shutdown call, it is possible to call initialize again.

    Returns void

  • Mutes or un-mutes all the sounds that are currently playing.

    Parameters

    • muteSound: boolean

      true if you want to mute all the sounds.

    Returns void

  • Feed RGBA image to DeepAR as input instead of camera or video. Used for processing single image. Can be used instead of startCamera or setVideoElement. Can be called in a loop.

    Parameters

    • frame: Uint8Array

      Image.

    • width: number

      Width of the image.

    • height: number

      Height of the image.

    • mirror: boolean

      Mirror frame horizontally.

    Returns void

  • If you want to apply DeepAR processing on a single image instead of a camera stream use this method. Can be used instead of startCamera or setVideoElement. See example usage here.

    Parameters

    • image: HTMLImageElement

    Returns void

  • Pauses the DeepAR processing and rendering on canvas.

    Parameters

    • isPause: boolean

      If true, DeepAR will pause. Otherwise, it will resume processing and rendering.

    Returns void

  • Changes a node or component bool parameter of the currently loaded effect. For more details about changeParameter API read our docs here.

    Parameters

    • gameObject: string

      The name of the node from DeepAR Studio. If multiple nodes share the same name, only the first one will be affected.

    • component: string

      The name of the component. If the name of the component is null or an empty string, the node itself will be affected.

    • parameter: string

      The name of the parameter.

    • value: number

      New parameter value.

    Returns void

  • Changes a node or component float parameter of the currently loaded effect. For more details about changeParameter API read our docs here.

    Parameters

    • gameObject: string

      The name of the node from DeepAR Studio. If multiple nodes share the same name, only the first one will be affected.

    • component: string

      The name of the component. If the name of the component is null or an empty string, the node itself will be affected.

    • parameter: string

      The name of the parameter.

    • value: boolean

      New parameter value.

    Returns void

  • Changes a node or component vector parameter of the currently loaded effect. For more details about changeParameter API read our docs here.

    Parameters

    • gameObject: string

      The name of the node from DeepAR Studio. If multiple nodes share the same name, only the first one will be affected.

    • component: string

      The name of the component. If the name of the component is null or an empty string, the node itself will be affected.

    • parameter: string

      The name of the parameter.

    • x: number

      X component of the new parameter vector.

    • y: number

      Y component of the new parameter vector.

    • z: number

      Z component of the new parameter vector.

    • w: number

      W component of the new parameter vector.

    Returns void

  • Changes a node or component texture parameter of the currently loaded effect. For more details about changeParameter API read our docs here.

    Parameters

    • gameObject: string

      The name of the node from DeepAR Studio. If multiple nodes share the same name, only the first one will be affected.

    • component: string

      The name of the component. If the name of the component is null or an empty string, the node itself will be affected.

    • parameter: string

      The name of the parameter.

    • textureUrl: string

      Url of the image that is going to be used as texture.

    • mode: RequestMode = ...

      Request mode used while fetching the texture.

    Returns Promise<void>

  • This method allows the user to fire a custom animation trigger for model animations from code. To fire a custom trigger, the trigger string must match the custom trigger set in the Studio when creating the effect. Read more here.

    Parameters

    • trigger: string

      The name of the trigger.

    Returns void

  • Change face detection sensitivity

    Parameters

    • sensitivity: number

      0 .. 5 (0 is fast, 4,5 is slow but allows to find smaller faces)

    Returns void

  • Enable/disable forced rendering on the canvas. It is useful to enable offscreen rendering in scenarios when the browser does not call requestAnimationFrame() function. DeepAR internally uses requestAnimationFrame() for the rendering loop. For example, when the browser tab is not focused, the browser will not call requestAnimationFrame() and DeepAR will not render. If offscreen rendering is enabled, DeepAR will use its internal timer for the rendering loop. Note that offscreen rendering enabled will not have as good results in terms of FPS compared to offscreen rendering disabled.

    If you need to use offscreen rendering. The best practice is to enable it only when needed - like when the browser tab is not focused. Otherwise, it is best to always disable offscreen rendering.

    Parameters

    • enable: boolean

      True - DeepAR will use its internal timer for the rendering loop. Rendering will work even when tab is not focused. False - DeepAR will use requestAnimationFrame() for the rendering loop.

    Returns void

  • Retrieves the HTML canvas element used for AR preview.

    The returned canvas is used for DeepAR rendering of camera preview and AR filters. Most commonly canvas is needed when you want to do some postprocessing on it or feed it into some video calling library.

    Returns HTMLCanvasElement

    The HTML canvas element.

  • Set the FPS of DeepAR rendering.

    Parameters

    • fps: number

      New FPS.

    Returns void

  • Initialize segmentation.

    Segmentation is usually lazy loaded on demand when filter loaded with switchEffect requires it. But this method will start loading the segmentation immediately. To start initializing segmentation as soon as possible pass "segmentationInit" hint in the initialize function (see DeepARParams).

    If the segmentation is already initialized it will do nothing. To check if segmentation is initialized call isSegmentationInitialized or wait onSegmentationInitialized callback.

    Returns void

  • Check weather the segmentation is initialized.

    Returns boolean

  • Display profiling metrics on preview.

    Parameters

    • enabled: boolean

    Returns void

  • Enable or disable global physics simulation.

    Parameters

    • enabled: boolean

    Returns void

  • Moves the selected game object from its current position in a tree and sets it as a direct child of a target game object. This is equivalent to moving around a node in the node hierarchy in the DeepAR Studio.

    Parameters

    • selectedGameObject: string

      Node to move.

    • targetGameObject: string

      New node parent.

    Returns void

  • Informs DeepAR that the specified touch event occurred.

    Parameters

    Returns void

    Deprecated

    Since version 5.4.0 DeepAR will automatically register touch events from canvas. There is no need to call this function anymore.

  • Sets the preview zoom.

    Parameters

    • zoomLevel: number

      Floating point number determining how much to zoom in. Value <= 1 will disable zoom. Example, value 2.0 puts 2x zoom.

    Returns void

  • Parameters

    • useDefaultFov: boolean
    • Optional fov: number

    Returns void

  • Gets the closest point on a mesh to a given point.

    Parameters

    • meshParentNodeName: string

      The name of the parent node of the mesh.

    • point: Float32Array

      The point in the same space as a mesh.

    Returns Float32Array

    closest point on the mesh to the given point.

  • Gets the transformation between two nodes.

    Parameters

    • srcNodeName: string

      source node.

    • dstNodeName: string

      destination node.

    Returns Float32Array

    Matrix that describes the transformation from srcNode to dstNode.

  • Set read pixels async or sync. Default is async.

    Parameters

    • readAsyncPixels: boolean

      True - async, false - sync.

    Returns void

  • Sets a new environment map

    Sets a new environment map that will be used for rendering materials with new PBR glTF shader. If an effect file using the new PBR glTF shader is currently active, function changes the environment map to the one provided. Otherwise, it is going to override the one passed in with the effect file.

    Parameters

    • envmap: string | ArrayBuffer

      Path to fetch or ArrayBuffer with an image.

    Returns Promise<void>

  • Resets environment map

    Resets the environment map to the default one. If an effect file using the new PBR glTF shader is currently active, function changes the environment map to the default one.

    Returns void

  • Sets tone mapping exposure

    Sets tone mapping exposure. Value passed here will override value specified within an effect file.

    Parameters

    • exposure: number

      exposure value

    Returns void

  • Gets tone mapping exposure

    Returns number

    currently active tone mapping exposure or undefined otherwise.

  • Resets tone mapping exposure

    Resets tone mapping exposure to the one that was specified within an effect file.

    Returns void

  • Sets environment map intensity

    Sets environment map intensity. Value passed here will override value specified within an effect file.

    Parameters

    • environmentMapIntensity: number

      environment map intensity

    Returns void

  • Gets environment map intensity

    Returns number

    current environment map intensity or undefined otherwise.

  • Resets environment map intensity

    Resets environment map intensity to the one that was specified within an effect file.

    Returns void

  • Sets tone mapping

    Sets tone mapping. Value passed here will override value specified within an effect file. See here for more details about tone mapping functions: https://modelviewer.dev/examples/tone-mapping

    Parameters

    Returns void

  • Gets currently active tone mapping function.

    Returns ToneMapping

    currently active tone mapping or undefined otherwise.

  • Resets tone mapping

    Resets tone mapping to the one that was specified within an effect file.

    Returns void

Generated using TypeDoc