Package ai.deepar.ar

Class DeepAR

java.lang.Object
ai.deepar.ar.DeepAR
All Implemented Interfaces:
android.media.ImageReader.OnImageAvailableListener
public class DeepAR extends Object implements android.media.ImageReader.OnImageAvailableListener
Main class for interacting with DeepAR engine.

  • Field Details

    • ERROR_MODEL_FILE_NOT_FOUND

      public static final String ERROR_MODEL_FILE_NOT_FOUND
      Model file not found error name.
      See Also:

    • ERROR_EFFECT_FILE_LOAD_FAILED

      public static final String ERROR_EFFECT_FILE_LOAD_FAILED
      Effect file load failed error name.
      See Also:

  • Constructor Details

    • DeepAR

      public DeepAR(android.content.Context context)
      Creates a new DeepAR object.
      Parameters:
      context - Android app context.

  • Method Details

    • changeLiveMode

      public void changeLiveMode(boolean value)
      An optimization method and it allows the user to indicate the DeepAR in which mode it should operate. If called with true value, DeepAR will expect a continuous flow of new frames and it will optimize its inner processes for such workload. An example of this is the typical use case of processing the frames from the camera stream. If called with false it will optimize for preserving resources and memory by pausing the rendering after each processed frame. A typical use case for this is when the user needs to process just one image. In that case, the user will feed the image to DeepAR by calling receiveFrame, and DeepAR would process it and stop rendering until a new frame is received. If we did so when the DeepAR is in live mode, it would process the same frame over and over again without ever stopping the rendering process, thus wasting processing time.
      Parameters:
      value - Enable or disable live mode.

    • setFixedDeltaTime

      public void setFixedDeltaTime(boolean isFixed, float deltaTimeMs)

    • startProfiling

      public void startProfiling()

    • stopProfiling

      public void stopProfiling()

    • setLicenseKey

      public void setLicenseKey(String key)
      The method used to set the license key for your project. The license key is linked to the applicationId it's created for. Tracking will not work unless a matching license key and applicationId are used.
      Parameters:
      key - The license key.

    • initialize

      public void initialize(android.content.Context context, AREventListener eventListener, String modelPath)
      Initializes the DeepAR engine. DeepAR engine is contained in the class DeepAR. This class should be instantiated and the method initialize should be called before use. This is typically done in onCreate method in Android Activity.
      Parameters:
      context - Android app context.
      eventListener - The event listener (with callback method implementations).
      modelPath - Use if the model file is downloaded over the air. The model file is usually located in the assets folder. The default value for this parameter is file:///android_asset/models.bin and this value is used if modelPath is null.

    • initialize

      public void initialize(android.content.Context context, AREventListener eventListener)
      Initializes the DeepAR engine. DeepAR engine is contained in the class DeepAR. This class should be instantiated and the method initialize should be called before use. This is typically done in onCreate method in Android Activity.
      Parameters:
      context - Android app context.
      eventListener - The event listener (with callback method implementations).

    • startCapture

      public void startCapture()
      Starts capturing the DeepAR processing output in an off-screen buffer. AREventListener method frameAvailable will be called with the contents of that buffer any time a new frame is ready meaning the frame is processed by the DeepAR engine. frameAvailable will not be called unless this method is called explicitly. User should call this method if they need to further process the DeepAR output like sending them over the network to implement video chat functionality. This method allows the DeepAR to be simultaneously in both on-screen and off-screen rendering mode - one could display the frames in UI and collect them in the frameAvailable.

    • startCapture

      public void startCapture(DeepARPixelFormat format)
      Starts capturing the DeepAR processing output in an off-screen buffer. AREventListener method frameAvailable will be called with the contents of that buffer any time a new frame is ready meaning the frame is processed by the DeepAR engine. frameAvailable will not be called unless this method is called explicitly. User should call this method if they need to further process the DeepAR output like sending them over the network to implement video chat functionality. This method allows the DeepAR to be simultaneously in both on-screen and off-screen rendering mode - one could display the frames in UI and collect them in the frameAvailable.
      Parameters:
      format - Specifies the output image frame pixel format of frameAvailable method. Sometimes 3rd party libraries that do additional frame processing after DeepAR expect the input frames in a specific format (like video streaming libs) and it is controlled by this parameter.

    • startCapture

      public void startCapture(int width, int height, float xmin, float xmax, float ymin, float ymax, DeepARPixelFormat format)
      Starts capturing the DeepAR processing output in an off-screen buffer. AREventListener method frameAvailable will be called with the contents of that buffer any time a new frame is ready meaning the frame is processed by the DeepAR engine. frameAvailable will not be called unless this method is called explicitly. User should call this method if they need to further process the DeepAR output like sending them over the network to implement video chat functionality. This method allows the DeepAR to be simultaneously in both on-screen and off-screen rendering mode - one could display the frames in UI and collect them in the frameAvailable.
      Parameters:
      width - The output resolution width.
      height - The output resolution height.
      xmin - Sub-rectangle left point in unit coordinates (0.0 - 1.0).
      xmax - Sub-rectangle right point in unit coordinates (0.0 - 1.0).
      ymin - Sub-rectangle top point in unit coordinates (0.0 - 1.0).
      ymax - Sub-rectangle bottom point in unit coordinates (0.0 - 1.0).
      format - Specifies the output image frame pixel format of frameAvailable method. Sometimes 3rd party libraries that do additional frame processing after DeepAR expect the input frames in a specific format (like video streaming libs) and it is controlled by this parameter.

    • stopCapture

      public void stopCapture()
      Stops capturing the content of the DeepAR rendering process in the off-screen buffer. Should be called if the user does not need post-processing features to preserve memory and computational resources.

    • setVisionOnly

      public void setVisionOnly()
      Set DeepAR in the vision-only mode. In this mode, DeepAR stops all rendering and it only outputs computer vision data which consists of tracking users' FaceData. To exit vision only mode call either setRenderSurface or setOffscreenRendering to switch DeepAR to rendering mode.

    • setOffscreenRendering

      public void setOffscreenRendering(int width, int height)
      Calling this method sets the engine in off-screen rendering mode. The engine will render the results of engine processing in the off-screen buffer which will be available to the user in the frameAvailable method of the AREventListener. Calling this method will automatically stop any on-screen rendering.
      Parameters:
      width - The output width of the off-screen rendered image.
      height - The output height of the off-screen rendered image.

    • setOffscreenRendering

      public void setOffscreenRendering(int width, int height, DeepARPixelFormat format)
      Calling this method sets the engine in off-screen rendering mode. The engine will render the results of engine processing in the off-screen buffer which will be available to the user in the frameAvailable method of the AREventListener. Calling this method will automatically stop any on-screen rendering.
      Parameters:
      width - The output width of the off-screen rendered image.
      height - The output height of the off-screen rendered image.
      format - Specifies the output image frame pixel format of frameAvailable method. Sometimes 3rd party libraries that do additional frame processing after DeepAR expect the input frames in a specific format (like video streaming libs) and it is controlled by this parameter.

    • setRenderSurface

      public void setRenderSurface(android.view.Surface surface, int width, int height)
      Calling this method sets the engine in on-screen rendering mode (cancels off-screen rendering if set before this call).
      Parameters:
      surface - The render surface, which is typically obtained in surfaceChanged method in SurfaceHolder.Callback. DeepAR engine uses SurfaceView to render 3D augmented reality elements. The SDK user is required to place the SurfaceView in the view hierarchy (through XML layout or manually from code). The user also needs to add a surface holder callback.

      Here's an example of how to use SurfaceHolder.Callback with DeepAR. 
       private DeepAR deepAR;
 ...

 @Override
 public void surfaceCreated(SurfaceHolder surfaceHolder) {
 }

 @Override
 public void surfaceChanged(SurfaceHolder surfaceHolder, int format, int width, int height) {
 // Pass the surface object along with width and height to DeepAR instance. DeepAR object must be instantiated and initialized at this point.
 deepAR.setRenderSurface(surfaceHolder.getSurface(), width, height);
 }

 @Override
 public void surfaceDestroyed(SurfaceHolder surfaceHolder) {
 // If the surface is destroyed, DeepAR engine must be notified by passing null as the Surface object. The rendering will be paused after this call.
 deepAR.setRenderSurface(null,0,0);
 }

 ...

 }
      width - The surface width.
      height - The surface height.

    • release

      public void release()
      Releases all resources required by DeepAR. Must be called after the rendering surface is destroyed. It's usually called in onDestroy method in Android Activity.

    • receiveFrame

      public void receiveFrame(ByteBuffer buffer, int width, int height, int orientation, boolean mirror, DeepARImageFormat imageFormat, int pixelStride)
      DeepAR can be used with Camera API, Camera2 API or CameraX API. The parameter data should be image byte data in YUV NV21 format (Camera API) or YUV 420 888 format (Camera2 API and CameraX API). ByteBuffer must be allocated using allocateDirect. When using CameraX API and you get 3 planes of buffers, just put all the planes in one buffer in the Y, V, U order. The orientation parameter is the camera sensor orientation, which can be read from CameraInfo object. ImageFormat parameter tells whether you are passing a YUV NV21 frame or YUV 422 888. PixelStride is always 1 for Camera API, for Camera2 API pass the pixel stride of U or V plane.
      Parameters:
      buffer - The frame buffer.
      width - The frame width.
      height - The frame height.
      orientation - The camera orientation.
      mirror - Indicates that the camera is mirrored.
      imageFormat - DeepAR image format.
      pixelStride - Pixel stride.

    • receiveFrameExternalTexture

      public void receiveFrameExternalTexture(int width, int height, int orientation, boolean mirror, int externalGlTexture)
      Notify DeepAR that new camera frame is available on the given external GL texture. This is the fastest method to pass camera frames to DeepAR. Using this method you can significantly increase overall DeepAR performance. This method should be called from SurfaceTexture.OnFrameAvailableListener.onFrameAvailable(SurfaceTexture). The externalGlTexture must be acquired by getExternalGlTexture(). Before calling this method, you must call SurfaceTexture.updateTexImage() on the SurfaceTexture created with externalGlTexture. This method must be called on main thread!
      Parameters:
      width - width of the camera resolution
      height - height of the camera resolution
      orientation - phone orientation
      mirror - flag denoting should the image be mirrored
      externalGlTexture - Acquired by getExternalGlTexture() and used to create SurfaceTexture which was used as camera preview Surface.

    • switchEffect

      public void switchEffect(String slot, InputStream inputStream) throws IOException
      Same as switchEffect, but uses an InputStream instead of a file path.
      Parameters:
      slot - An arbitrary slot name.
      inputStream - The effect data input stream.
      Throws:
      IOException - When an IO error occurs.

    • switchEffect

      public void switchEffect(String slot, InputStream inputStream, int face) throws IOException
      Same as switchEffect, but uses an InputStream instead of a file path.
      Parameters:
      slot - An arbitrary slot name.
      inputStream - The effect data input stream.
      face - The face index. Valid values are 0, 1, 2 and 3.
      Throws:
      IOException - When an IO error occurs.

    • switchEffect

      public void switchEffect(String slot, InputStream inputStream, int face, String targetGameObject) throws IOException
      Same as switchEffect, but uses an InputStream instead of a file path.
      Parameters:
      slot - An arbitrary slot name.
      inputStream - The effect data input stream.
      face - The face index. Valid values are 0, 1, 2 and 3.
      targetGameObject - The name of the node where the effect will be attached.
      Throws:
      IOException - When an IO error occurs.

    • switchEffect

      public void switchEffect(String slot, String path)
      The method used to switch any effect in the scene. Effects are places in slots. Every slot is identified by its unique name and can hold one effect at any given moment. Every subsequent call to this method removes the effect that was previously displayed in this slot. To clear effect from a slot, call this method with null for path parameter. Using this method user can combine multiple effects in the scene, like using a face mask and a filter effect:

       deepAR.switchEffect("mask","file:///android_asset/flowers");
 deepAR.switchEffect("filters","file:///android_asset/tv80");
      When tracking multiple faces is enabled this method only affects the first tracked face.
      Parameters:
      slot - An arbitrary slot name.
      path - The path to an effect file in the project which has been exported from DeepAR Studio.

    • switchEffect

      public void switchEffect(String slot, String path, int face)
      The method used to switch any effect in the scene. Effects are places in slots. Every slot is identified by its unique name and can hold one effect at any given moment. Every subsequent call to this method removes the effect that was previously displayed in this slot. To clear effect from a slot, call this method with null for path parameter. Using this method user can combine multiple effects in the scene, like using a face mask and a filter effect. Different faces should use different slots such as "mask_f0" and "mask_f1" instead of "mask" in the previous example. DeepAR SDK supports tracking 4 faces in parallel.

       // apply flowers effect to the first face
 deepAR.switchEffect("mask_f0","file:///android_asset/flowers",0);
 // apply beard effect to the second face
 deepAR.switchEffect("mask_f1","file:///android_asset/beard",1);
 // replace the effect on the first face with the lion
 deepAR.switchEffect("mask_f0","file:///android_asset/lion",0);
 // remove the beard effect from the second face
 deepAR.switchEffect("mask_f1",null,1);
      Parameters:
      slot - An arbitrary slot name.
      path - The path to an effect file in the project which has been exported from DeepAR Studio.
      face - The face index. Valid values are 0, 1, 2 and 3.

    • switchEffect

      public void switchEffect(String slot, String path, int face, String targetGameObject)
      The method used to switch any effect in the scene. Effects are places in slots. Every slot is identified by its unique name and can hold one effect at any given moment. Every subsequent call to this method removes the effect that was previously displayed in this slot. To clear effect from a slot, call this method with null for path parameter. Using this method user can combine multiple effects in the scene, like using a face mask and a filter effect. Different faces should use different slots such as "mask_f0" and "mask_f1" instead of "mask" in the previous example. DeepAR SDK supports tracking 4 faces in parallel.
      Parameters:
      slot - An arbitrary slot name.
      path - The path to an effect file in the project which has been exported from DeepAR Studio.
      face - The face index. Valid values are 0, 1, 2 and 3.
      targetGameObject - The name of the node where the effect will be attached.

    • takeScreenshot

      public void takeScreenshot()
      Captures a screenshot of the current screen. When a screenshot is done AREventListener method screenshotTaken will be called with a Bitmap parameter which will be the resulting screenshot.

    • hasVar

      public boolean hasVar(String name, String slot)
      Check if variable with the given name is already created.
      Parameters:
      name - The variable name.
      slot - The slot of the effect in which to search the variable.
      Returns:
      True if the variable is already created, false otherwise.

    • hasVar

      public boolean hasVar(String name)
      Check if the variable with the given name is already created in at least one effect.
      Parameters:
      name - The variable name.
      Returns:
      True if the variable is already created, false otherwise.

    • getVarType

      public ai.deepar.ar.VarType getVarType(String name, String slot)
      Get the type of the variable with the given name in the specified effect.
      Parameters:
      name - The variable name.
      slot - The slot of the effect in which to search the variable.
      Returns:
      VarType Variable type.
      Throws:
      IllegalArgumentException - if the variable with the specified name does not exist.

    • getVarType

      public ai.deepar.ar.VarType getVarType(String name)
      Get the type of the variable with the given name. The variable is searched in all effects.
      Parameters:
      name - The variable name.
      Returns:
      VarType Variable type.
      Throws:
      IllegalArgumentException - if the variable with the specified name does not exist.

    • getBoolVar

      public boolean getBoolVar(String name, String slot)
      Get boolean variable with the given name.
      Parameters:
      name - The variable name.
      slot - The slot of the effect in which to search the variable.
      Returns:
      Value of the variable with the specified name.
      Throws:
      IllegalArgumentException - if the variable with the specified name does not exist or the variable type is not a boolean.

    • getBoolVar

      public boolean getBoolVar(String name)
      Get boolean variable with the given name. The variable is searched in all effects.
      Parameters:
      name - The variable name.
      Returns:
      Value of the variable with the specified name.
      Throws:
      IllegalArgumentException - if the variable with the specified name does not exist or the variable type is not a boolean.

    • getIntVar

      public int getIntVar(String name, String slot)
      Get int variable with the given name.
      Parameters:
      name - The variable name.
      slot - The slot of the effect in which to search the variable.
      Returns:
      Value of the variable with the specified name.
      Throws:
      IllegalArgumentException - if the variable with the specified name does not exist or the variable type is not an int.

    • getIntVar

      public int getIntVar(String name)
      Get int variable with the given name. The variable is searched in all effects.
      Parameters:
      name - The variable name.
      Returns:
      Value of the variable with the specified name.
      Throws:
      IllegalArgumentException - if the variable with the specified name does not exist or the variable type is not an int.

    • getDoubleVar

      public double getDoubleVar(String name, String slot)
      Get double variable with the given name.
      Parameters:
      name - The variable name.
      slot - The slot of the effect in which to search the variable.
      Returns:
      Value of the variable with the specified name.
      Throws:
      IllegalArgumentException - if the variable with the specified name does not exist or the variable type is not a double.

    • getDoubleVar

      public double getDoubleVar(String name)
      Get double variable with the given name. The variable is searched in all effects.
      Parameters:
      name - The variable name.
      Returns:
      Value of the variable with the specified name.
      Throws:
      IllegalArgumentException - if the variable with the specified name does not exist or the variable type is not a double.

    • getStringVar

      public String getStringVar(String name, String slot)
      Get string variable with the given name.
      Parameters:
      name - The variable name.
      slot - The slot of the effect in which to search the variable.
      Returns:
      Value of the variable with the specified name.
      Throws:
      IllegalArgumentException - if the variable with the specified name does not exist or the variable type is not a string.

    • getStringVar

      public String getStringVar(String name)
      Get string variable with the given name. The variable is searched in all effects.
      Parameters:
      name - The variable name.
      Returns:
      Value of the variable with the specified name.
      Throws:
      IllegalArgumentException - if the variable with the specified name does not exist or the variable type is not a string.

    • setBoolVar

      public boolean setBoolVar(String name, boolean value, String slot)
      Set the boolean variable with the given name.
      Parameters:
      name - The variable name.
      value - Value to be set.
      slot - The slot of the effect in which to set the variable.
      Returns:
      True if the variable is created, false if the variable already exists and the new value is set.

    • setBoolVar

      public boolean setBoolVar(String name, boolean value)
      Set the boolean variable with the given name. The variable is set globally, for all effects.
      Parameters:
      name - The variable name.
      value - Value to be set.
      Returns:
      True if the variable is created, false if the variable already exists and the new value is set.

    • setIntVar

      public boolean setIntVar(String name, int value, String slot)
      Set the int variable with the given name.
      Parameters:
      name - The variable name.
      value - Value to be set.
      slot - The slot of the effect in which to set the variable.
      Returns:
      True if the variable is created, false if the variable already exists and the new value is set.

    • setIntVar

      public boolean setIntVar(String name, int value)
      Set the int variable with the given name. The variable is set globally, for all effects.
      Parameters:
      name - The variable name.
      value - Value to be set.
      Returns:
      True if the variable is created, false if the variable already exists and the new value is set.

    • setDoubleVar

      public boolean setDoubleVar(String name, double value, String slot)
      Set the double variable with the given name.
      Parameters:
      name - The variable name.
      value - Value to be set.
      slot - The slot of the effect in which to set the variable.
      Returns:
      True if the variable is created, false if the variable already exists and the new value is set.

    • setDoubleVar

      public boolean setDoubleVar(String name, double value)
      Set the double variable with the given name. The variable is set globally, for all effects.
      Parameters:
      name - The variable name.
      value - Value to be set.
      Returns:
      True if the variable is created, false if the variable already exists and the new value is set.

    • setStringVar

      public boolean setStringVar(String name, String value, String slot)
      Set the string variable with the given name.
      Parameters:
      name - The variable name.
      value - Value to be set.
      slot - The slot of the effect in which to set the variable.
      Returns:
      True if the variable is created, false if the variable already exists and the new value is set.

    • setStringVar

      public boolean setStringVar(String name, String value)
      Set the string variable with the given name. The variable is set globally, for all effects.
      Parameters:
      name - The variable name.
      value - Value to be set.
      Returns:
      True if the variable is created, false if the variable already exists and the new value is set.

    • deleteVar

      public boolean deleteVar(String name, String slot)
      Delete the variable with the given name.
      Parameters:
      name - The variable name.
      slot - The slot of the effect in which to search the variable.
      Returns:
      True if the variable is deleted, false otherwise.

    • deleteVar

      public boolean deleteVar(String name)
      Delete the variable with the given name. The variable is searched in all effects.
      Parameters:
      name - The variable name.
      Returns:
      True if the variable is deleted, false otherwise.

    • clearVars

      public boolean clearVars(String slot)
      Clear all variables or variables from the specified effect.
      Parameters:
      slot - ID of the effect from which to clear all the variables.
      Returns:
      True if at least one variable is deleted, false otherwise.

    • clearVars

      public boolean clearVars()
      Clear all variables.
      Returns:
      True if at least one variable is deleted, false otherwise.

    • setAREventListener

      public void setAREventListener(AREventListener listener)
      Sets the AREventListener for DeepAR object.
      Parameters:
      listener - DeepAR event listener object (with implemented callbacks).

    • onImageAvailable

      public void onImageAvailable(android.media.ImageReader imageReader)
      Callback that is called when a new image is available from ImageReader.
      Specified by:
      onImageAvailable in interface android.media.ImageReader.OnImageAvailableListener
      Parameters:
      imageReader - The ImageReader the callback is associated with.
      See Also:

    • getRenderWidth

      public int getRenderWidth()
      Gets the render width.
      Returns:
      The render width.

    • getRenderHeight

      public int getRenderHeight()
      Gets the render height.
      Returns:
      The render height.

    • startPreparedVideoRecording

      public void startPreparedVideoRecording()
      Starts the prepared video recording.

    • startVideoRecording

      public void startVideoRecording(String filePath)
      Starts video recording of the camera preview. The video will be the same size as SurfaceView. Video recording events can be observed through AREventListener implementation.
      Parameters:
      filePath - The video file path.

    • startVideoRecording

      public void startVideoRecording(FileDescriptor fd)
      Starts video recording of the camera preview. The video will be the same size as SurfaceView. Video recording events can be observed through AREventListener implementation.
      Parameters:
      fd - The video file Descriptor.

    • startVideoRecording

      public void startVideoRecording(String filePath, int outputWidth, int outputHeight)
      Starts video recording of the camera preview. The SurfaceView will be skewed to the desired resolution. To preserve the aspect ratio, enter the desired output width or height and set the other parameter to zero. Video recording events can be observed through AREventListener implementation.
      Parameters:
      filePath - The video file path.
      outputWidth - The output video width.
      outputHeight - The output video height.

    • startVideoRecording

      public void startVideoRecording(FileDescriptor fd, int outputWidth, int outputHeight)
      Starts video recording of the camera preview. The SurfaceView will be skewed to the desired resolution. To preserve the aspect ratio, enter the desired output width or height and set the other parameter to zero. Video recording events can be observed through AREventListener implementation.
      Parameters:
      fd - The video file descriptor.
      outputWidth - The output video width.
      outputHeight - The output video height.

    • startVideoRecording

      public void startVideoRecording(String filePath, android.graphics.Rect subframe, int outputWidth, int outputHeight)
      Starts video recording of the camera preview. The SurfaceView will be skewed to the desired resolution. To preserve the aspect ratio, enter the desired output width or height and set the other parameter to zero. Video recording events can be observed through AREventListener implementation.
      Parameters:
      filePath - The video file path.
      subframe - Use to record only part of SurfaceView.
      outputWidth - The output video width.
      outputHeight - The output video height.

    • startVideoRecording

      public void startVideoRecording(FileDescriptor fd, android.graphics.Rect subframe, int outputWidth, int outputHeight)
      Starts video recording of the camera preview. The SurfaceView will be skewed to the desired resolution. To preserve the aspect ratio, enter the desired output width or height and set the other parameter to zero. Video recording events can be observed through AREventListener implementation.
      Parameters:
      fd - The video file descriptor.
      subframe - Use to record only part of SurfaceView.
      outputWidth - The output video width.
      outputHeight - The output video height.

    • startVideoRecording

      public void startVideoRecording(FileDescriptor fd, android.graphics.Rect subframe, int outputWidth, int outputHeight, boolean recordAudio)
      Starts video recording of the camera preview. The SurfaceView will be skewed to the desired resolution. To preserve the aspect ratio, enter the desired output width or height and set the other parameter to zero. Video recording events can be observed through AREventListener implementation.
      Parameters:
      fd - The video file descriptor.
      subframe - Use to record only part of SurfaceView.
      outputWidth - The output video width.
      outputHeight - The output video height.
      recordAudio - Indicates whether audio will be recorded or not.

    • startVideoRecording

      public void startVideoRecording(FileDescriptor fd, Map<String,Object> parameters)
      Starts video recording of the camera preview. The SurfaceView will be skewed to the desired resolution. To preserve the aspect ratio, enter the desired output width or height and set the other parameter to zero. Video recording events can be observed through AREventListener implementation.
      Parameters:
      fd - The video file descriptor.
      parameters - A dictionary of parameters for configuring the video recording. The following parameters are supported:
      • "outputWidth" (optional): The output video width. Defaults to the render width.
      • "outputHeight" (optional): The output video height. Defaults to the render height.
      • "subframe" (optional): Use to record only part of SurfaceView. Defaults to the entire render area.
      • "recordAudio" (optional): Indicates whether audio will be recorded. Defaults to true.
      • "bitRate" (optional): The output video bitRate. Defaults to 0.15 * frameRate * outputWidth * outputHeight.
      • "audioSampleRate" (optional): The audio recording sample rate in Hz. Defaults to 44100Hz which will work on all devices.

    • startVideoRecording

      public void startVideoRecording(String filePath, android.graphics.Rect subframe, int outputWidth, int outputHeight, boolean recordAudio)
      Starts video recording of the camera preview. The SurfaceView will be skewed to the desired resolution. To preserve the aspect ratio, enter the desired output width or height and set the other parameter to zero. Video recording events can be observed through AREventListener implementation.
      Parameters:
      filePath - The video file path.
      subframe - Use to record only part of SurfaceView.
      outputWidth - The output video width.
      outputHeight - The output video height.
      recordAudio - Indicates whether audio will be recorded or not.

    • startVideoRecording

      public void startVideoRecording(String filePath, Map<String,Object> parameters)
      Starts video recording of the camera preview. The SurfaceView will be skewed to the desired resolution. To preserve the aspect ratio, enter the desired output width or height and set the other parameter to zero. Video recording events can be observed through AREventListener implementation.
      Parameters:
      filePath - The video file path.
      parameters - A dictionary of parameters for configuring the video recording. The following parameters are supported:
      • "outputWidth" (optional): The output video width. Defaults to the render width.
      • "outputHeight" (optional): The output video height. Defaults to the render height.
      • "subframe" (optional): Use to record only part of SurfaceView. Defaults to the entire render area.
      • "recordAudio" (optional): Indicates whether audio will be recorded. Defaults to true.
      • "bitRate" (optional): The output video bitRate. Defaults to 0.15 * frameRate * outputWidth * outputHeight.
      • "audioSampleRate" (optional): The audio recording sample rate in Hz. Defaults to 44100Hz which will work on all devices.

    • getMinSupportedVideoHeight

      public int getMinSupportedVideoHeight()
      Gets the minimum supported video height.
      Returns:
      The minimum supported video height.

    • getMaxSupportedVideoHeight

      public int getMaxSupportedVideoHeight()
      Gets the maximum supported video height.
      Returns:
      The maximum supported video height.

    • getMinSupportedVideoWidth

      public int getMinSupportedVideoWidth()
      Gets the minimum supported video width.
      Returns:
      The minimum supported video width.

    • getMaxSupportedVideoWidth

      public int getMaxSupportedVideoWidth()
      Gets the maximum supported video width.
      Returns:
      The maximum supported video width.

    • stopVideoRecording

      public void stopVideoRecording()
      Stops video recording and starts the process of saving the recorded video to the file system. When the file is saved, the method videoRecordingFinished will be called.

    • setAudioMute

      public void setAudioMute(boolean mute)
      Mutes or unmutes the audio while video recording.
      Parameters:
      mute - Indicates whether you want to mute or unmute the audio.

    • getBitRate

      public int getBitRate()
      Gets the video bitrate.
      Returns:
      The video bitrate.

    • setBitRate

      public void setBitRate(int bitRate)
      Sets the video bitrate. The default value is 0.15 * frame rate * width * height.
      Parameters:
      bitRate - The video bitrate.

    • getIFrameInterval

      public int getIFrameInterval()
      Gets the I frame interval (KEY_I_FRAME_INTERVAL).
      Returns:
      The I frame interval.

    • setIFrameInterval

      public void setIFrameInterval(int iFrameInterval)
      Sets the I frame interval (KEY_I_FRAME_INTERVAL). The default value is 5.
      Parameters:
      iFrameInterval - The I frame interval.

    • getKeyFrameRate

      public int getKeyFrameRate()
      Gets the key frame rate (KEY_FRAME_RATE).
      Returns:
      The key frame rate.

    • setKeyFrameRate

      public void setKeyFrameRate(int keyFrameRate)
      Sets the key frame rate (KEY_FRAME_RATE). The default value is 25.
      Parameters:
      keyFrameRate - The key frame rate.

    • getFrameRenderedCallback

      public DeepAR.FrameRenderedCallback getFrameRenderedCallback()
      Gets the frame rendered callback.
      Returns:
      The frame rendered callback.

    • setFrameRenderedCallback

      public void setFrameRenderedCallback(DeepAR.FrameRenderedCallback frameRenderedCallback)
      Sets the frame rendered callback.
      Parameters:
      frameRenderedCallback - The frame rendered callback.

    • getFaceTrackedCallback

      public DeepAR.FaceTrackedCallback getFaceTrackedCallback()
      Gets the face tracked callback.
      Returns:
      The face tracked callback.

    • setFaceTrackedCallback

      public void setFaceTrackedCallback(DeepAR.FaceTrackedCallback faceTrackedCallback)
      Sets the face tracked callback.
      Parameters:
      faceTrackedCallback - The face tracked callback.

    • setAnimationTransitionedCallback

      public void setAnimationTransitionedCallback(DeepAR.AnimationTransitionedCallback animationTransitionedCallback)
      Sets the animation transitioned callback.
      Parameters:
      animationTransitionedCallback - Animation transitioned callback.

    • setAntialiasingLevel

      public void setAntialiasingLevel(int level)
      Sets the MSAA level. The allowed levels are 0 (MSAA off), 2, 4, 8, 16. This should be called before initialize to have any effect.
      Parameters:
      level - The MSAA level.

    • touchOccurred

      public void touchOccurred(ARTouchInfo touchInfo)
      Informs DeepAR that the specified touch event occurred.
      Parameters:
      touchInfo - Touch event information.

    • pauseVideoRecording

      public void pauseVideoRecording()
      Pauses the video recording.

    • resumeVideoRecording

      public void resumeVideoRecording()
      Resumes the video recording.

    • changeParameterFloat

      public void changeParameterFloat(String gameObject, String component, String parameter, float val)
      Changes a node or component float parameter. For more details about changeParameter API read our article here.
      Parameters:
      gameObject - The name of the node. If multiple nodes share the same name, only the first one will be affected.
      component - The name of the component. If the name of the component is null or an empty string, the node itself will be affected.
      parameter - The name of the parameter.
      val - New parameter value.

    • changeParameterBool

      public void changeParameterBool(String gameObject, String component, String parameter, boolean val)
      Changes a node or component boolean parameter. For more details about changeParameter API read our article here.
      Parameters:
      gameObject - The name of the node. If multiple nodes share the same name, only the first one will be affected.
      component - The name of the component. If the name of the component is null or an empty string, the node itself will be affected.
      parameter - The name of the parameter.
      val - New parameter value.

    • changeParameterVec4

      public void changeParameterVec4(String gameObject, String component, String parameter, float valX, float valY, float valZ, float valW)
      Changes a node or component 4D vector parameter. For more details about changeParameter API read our article here.
      Parameters:
      gameObject - The name of the node. If multiple nodes share the same name, only the first one will be affected.
      component - The name of the component. If the name of the component is null or an empty string, the node itself will be affected.
      parameter - The name of the parameter.
      valX - New parameter X value.
      valY - New parameter Y value.
      valZ - New parameter Z value.
      valW - New parameter W value.

    • changeParameterVec3

      public void changeParameterVec3(String gameObject, String component, String parameter, float valX, float valY, float valZ)
      Changes a node or component 3D vector parameter. For more details about changeParameter API read our article here.
      Parameters:
      gameObject - The name of the node. If multiple nodes share the same name, only the first one will be affected.
      component - The name of the component. If the name of the component is null or an empty string, the node itself will be affected.
      parameter - The name of the parameter.
      valX - New parameter X value.
      valY - New parameter Y value.
      valZ - New parameter Z value.

    • changeParameterTexture

      public void changeParameterTexture(String gameObject, String component, String parameter, android.graphics.Bitmap image)
      Changes a node or component texture parameter. For more details about changeParameter API read our article here.
      Parameters:
      gameObject - The name of the node. If multiple nodes share the same name, only the first one will be affected.
      component - The name of the component. If the name of the component is null or an empty string, the node itself will be affected.
      parameter - The name of the parameter.
      image - New parameter Bitmap texture.

    • fireTrigger

      public void fireTrigger(String trigger)
      Fire named trigger of an fbx animation set on the currently loaded effect. To learn more about fbx and image sequence animations on DeepAR please read our article here.
      Parameters:
      trigger - The trigger name.

    • setFaceDetectionSensitivity

      public void setFaceDetectionSensitivity(int sensitivity)
      This method allows the user to change face detection sensitivity.
      Parameters:
      sensitivity - The sensitivity parameter can range from 0 to 3, where 0 is the fastest but might not recognize smaller (further away) faces, and 3 is the slowest but will find smaller faces. By default, this parameter is set to 1.

    • setPaused

      public void setPaused(boolean paused)
      Pauses or unpauses the rendering. This method will not release any resources and should be used only for temporary pause (e.g. user goes to the next screen). Use the shutdown method to stop the engine and release the resources.
      Parameters:
      paused - Indicates whether to pause or unpause the rendering.

    • showStats

      public void showStats(boolean enabled)
      Display debugging stats on screen.
      Parameters:
      enabled - Enable debugging stats.

    • simulatePhysics

      public void simulatePhysics(boolean enabled)
      Enable or disable global physics simulation.
      Parameters:
      enabled - Enable global physics simulation.

    • showColliders

      public void showColliders(boolean enabled)
      Display physics colliders preview on screen.
      Parameters:
      enabled - Enable physics colliders preview.

    • moveGameObject

      public void moveGameObject(String selectedGameObject, String targetGameObject)
      Moves the selected game object from its current position in a tree and sets it as a direct child of a target game object. This is equivalent to moving around a node in the node hierarchy in the DeepAR Studio.
      Parameters:
      selectedGameObject - Node to move.
      targetGameObject - New node parent.

    • useSingleThreadedMode

      public void useSingleThreadedMode(boolean useClientsGLContext)
      This methods puts DeepAR in single-threaded mode. By default this will put DeepAR in non live-mode. This means that DeepAR will do internal updates (ticks) and rendering only when DeepAR receives new frames by calling receiveFrame(ByteBuffer, int, int, int, boolean, DeepARImageFormat, int) or receiveFrameExternalTexture(int, int, int, boolean, int). In single-threaded mode, calls to receiveFrame(ByteBuffer, int, int, int, boolean, DeepARImageFormat, int) and receiveFrameExternalTexture(int, int, int, boolean, int) will be blocking. When those methods return, the frame will be rendered in the surface set by setRenderSurface(Surface, int, int). If you wish to call receiveFrameExternalTexture(int, int, int, boolean, int) with parameter externalGlTexture set to texture that was not created by getExternalGlTexture() but rather to the texture created by your openGL context - then you must set useClientsGLContext flag. When useClientsGLContext flag is set, then your OpenGL context must be made current before calling setRenderSurface(Surface, int, int). This way DeepAR can initialize its internal OpenGL context by sharing your OpenGL context. Your OpenGL context will be preserved (current) after calls to DeepAR methods that do rendering. WARNING: This method must be called before first call to setRenderSurface(Surface, int, int).
      Parameters:
      useClientsGLContext -

    • getExternalGlTexture

      public int getExternalGlTexture()
      Get Android External GL texture. Used to feed frames to DeepAR via receiveFrameExternalTexture(int, int, int, boolean, int). This is a GL texture bound to GL_TEXTURE_EXTERNAL_OES. Use this GL texture to create SurfaceTexture. Use created SurfaceTexture as preview Surface for camera. When SurfaceTexture.OnFrameAvailableListener.onFrameAvailable(SurfaceTexture) is called, call receiveFrameExternalTexture(int, int, int, boolean, int) to notify DeepAR of the new camera frame. This method must be called on main thread!
      Returns:
      GL texture handle. 0 if external texture cannot be produced.

    • releaseExternalGlTexture

      public void releaseExternalGlTexture(int textureHandle)
      Release the external texture aqured from getExternalGlTexture().
      Parameters:
      textureHandle -

    • setFaceTrackingInitParameters

      public void setFaceTrackingInitParameters(DeepAR.FaceTrackingInitParameters initParameters)
      Sets parameters that will determine how the face tracking is initialized. Note: Must be called before initializing DeepAR.
      Parameters:
      initParameters - DeepAR.FaceTrackingInitParameters

    • muteSound

      public void muteSound(boolean muteSound)
      Mutes or un-mutes all the sounds that are currently playing.
      Parameters:
      muteSound -

    • backgroundReplacement

      public void backgroundReplacement(boolean enable, android.graphics.Bitmap image)
      Enable background replacement (also known as background removal or green screen effect).
      Parameters:
      enable - - Boolean value indicating whether to enable or disable the background replacement effect.
      image - - Image to be used as the background.

    • backgroundBlur

      public void backgroundBlur(boolean enable, int strength)
      Enable background blur. Background blur is usually used in video calling use cases.
      Parameters:
      enable - - Boolean value indicating whether to enable or disable the background blur effect.
      strength - - Blur strength. Integer value in range 1-10.

    • setEnvironmentMap

      public void setEnvironmentMap(android.graphics.Bitmap bitmap)
      Sets a new environment map that will be used for rendering materials with new PBR glTF shader. If an effect file using the new PBR glTF shader is currently active, function changes the environment map to the one provided. Otherwise, it is going to override the one passed in with the effect file.
      Parameters:
      bitmap - image to be used as an environment map.

    • setEnvironmentMapIntensity

      public void setEnvironmentMapIntensity(float environmentMapIntensity)
      Sets environment map intensity. Value passed here will override value specified within an effect file.
      Parameters:
      environmentMapIntensity - environment map intensity

    • setExposure

      public void setExposure(float exposure)
      Sets tone mapping exposure. Value passed here will override value specified within an effect file.
      Parameters:
      exposure - exposure value

    • setToneMapping

      public void setToneMapping(DeepAR.ToneMapping toneMapping)
      Sets tone mapping. Value passed here will override value specified within an effect file. See here for more details about tone mapping functions
      Parameters:
      toneMapping - tone mapping function

    • setBloomEnabled

      public void setBloomEnabled(boolean enable)
      Toggles bloom postprocessing.

    • setBloomThreshold

      public void setBloomThreshold(float threshold)
      Sets bloom threshold parameter. Value passed here overrides the one specified within an effect file.
      Parameters:
      threshold - bloom threshold

    • setBloomStrength

      public void setBloomStrength(float strength)
      Sets bloom strength parameter. Value passed here overrides the one specified within an effect file.
      Parameters:
      strength - bloom strength

    • setBloomRadius

      public void setBloomRadius(float radius)
      Sets bloom radius parameter. Value passed here overrides the one specified within an effect file.
      Parameters:
      radius - bloom radius

    • setDiamondEnvironmentMap

      public void setDiamondEnvironmentMap(android.graphics.Bitmap bitmap)
      Sets new environment map that will be used for rendering materials with diamond shader. Overrides environment map specified within an effect file.
      Parameters:
      bitmap - Path to fetch or ArrayBuffer with an image.

    • getSdkVersion

      public static String getSdkVersion()