Skip to main content

Changelog

Web SDK version changelog​

v5.6.12 - 2024/11/11​

Fixed:

  • Effects with diamond bloom flickering on Windows with Nvidia and AMD GPUs.
  • Using Animated textures (image sequence) sometimes crashing the SDK.
  • Effects with diamond bloom when auto framing is used not displaying properly.
  • Effects with diamond bloom when setZoom API is used not displaying properly.
  • Sometimes crashing when changing video source.

v5.6.11 - 2024/10/08​

Added:

  • CNN face tracking model. Use hint enableFaceTrackingCnn to enable CNN face tracking. It is more accurate and works well at long distances, but it is slower. Current use-cases cover long distance face tracking and better accuracy with beauty. Only works with single person face tracking, ie. cannot track multiple faces. Currently not compatible with EyeComponent.
  • Support for loading effects with Draco compression with blendshapes and bones.
  • switchEffect API can now accept optional diamond bloom parameters.

Changed:

  • Loading 16-bit PNGs will now always internally convert to 8-bit (RGB16 texture format is not well supported across devices).

Fixed:

  • Offscreen rendering not working in Safari iOS 18.
  • Changing video source could crash SDK.
  • Effects with multiple diamonds not displaying properly. Cube camera render pipeline now renders only one frame per mesh.
  • Diamond effect with bloom sometimes flickering.
  • Diamond effects with transmissive materials not working properly.
  • Diamond effects not applying tone mapping properly.
  • Loading HDR envmaps from effect would always interpret it in sRGB. Not it can interpret it in linear color space also.

v5.6.10 - 2024/09/12​

Added:

  • Added API for customizing diamond materials: setBloomEnabled, setBloomThreshold, setBloomStrength, setBloomRadius, setDiamondEnvironmentMap.

Changed:

  • Switched from using wrist-track-181-fast-q.bin (2.4 MB) to wrist-track-181-q.bin (3.6 MB) wrist tracking model. The new model is more accurate but also bigger.
  • Turned on temporal filter for wrist tracking which smoothes it out but also introduces 3 frames of camera delay in preview.
  • Optimize camera/video pipeline.
  • Improve GPU memory usage in camera/video pipeline. This should improve Safari crashes on iPhones.

Fixed:

  • Passing in the environment map in switchEffect API would only work for the first time.
  • Bloom not displaying properly after resolution change.

v5.6.9 - 2024/08/26​

Added:

  • diamond rendering workflow.
  • pixelRatio API.
  • correct escaping of document.title when packing into JSON for authentication.

Fixed:

  • memory leaks related to the tone mapping API.

v5.6.8 - 2024/07/30​

Fixed:

  • Tone-mapping not working properly on non-transmissive materials.

v5.6.7 - 2024/07/26​

Added:

  • Check for meta viewport tag. If <meta name="viewport" content="width=device-width, initial-scale=1 "/> is not defined in HTML, DPR will always be 1.
  • Input camera/video rotation API. With this API wi can have better camera quality for magic mirror use cases.
  • The SDK platform and version are now logged in console every time the SDK is initialized for easier debugging purposes in production.

Changed:

  • Don't use HDR pipeline when HDR is not needed - if environment map is not HDR or tone mapping is linear. This improves performance in some cases and improves backwards compatibility with older versions of PBR effects.

Fixed:

  • Tone-mapping not working properly on transmissive materials.
  • Fix setting unassigned textures when loading effects.
  • Fix simple pendulum physics component not working properly with rigid face tracking.
  • Environment map not loading when provided in switchEffect API.

v5.6.5 - 2024/06/12​

Fixed:

  • setExposure, setEnvironmentMapIntensity and setToneMapping API not working properly when loading an effect which has multiple nodes with the same name.
  • Incorrect loading of PNGs of type grayscale with alpha.

v5.6.4 - 2024/05/16​

Changed:

  • Foot and wrist tracking are removed from DeepAR SDK. They are available in ShopAR

v5.6.3 - 2024/05/13​

Added:

  • Loading support for grayscale and palette PNG images.
  • Loading support up to 16 bit PNG images.
  • Generating mipmaps for textures in runtime when effect is loaded.

Changed:

  • Bump internal SDK version for loading DeepAR effect to 4.
  • switchEffect API now accepts also ArrayBufferLike, not just ArrayBuffer and string. This allows to directly load Uint8Array as an effect.

Fixed:

  • Environment HDR maps loaded through API (setEnvironmentMap) are now correctly interpreted as in linear space.
  • Tone mapping and HDR not working in full dynamic range on iOS. On iOS, SDK wrongly detected that 16 bit textures were not supported and fell back to 8 bit.
  • Fixed crash when additionalOptions parameter missing in initialization parameters.

v5.6.2 - 2024/04/29​

Added:

  • HDR support for glTF PBR materials
  • Added API setEnvironmentMap, setExposure, setToneMapping, setEnvironmentMapIntensity

v5.6.0 - 2024/03/15​

Added:

  • Full support for glTF 2.0 PBR materials. List of supported extensions:
    • KHR_DRACO_MESH_COMPRESSION,
    • KHR_LIGHTS_PUNCTUAL,
    • KHR_MATERIALS_ANISOTROPY,
    • KHR_MATERIALS_CLEARCOAT,
    • KHR_MATERIALS_EMISSIVE_STRENGTH,
    • KHR_MATERIALS_IOR,
    • KHR_MATERIALS_IRIDESCENCE,
    • KHR_MATERIALS_SHEEN,
    • KHR_MATERIALS_SPECULAR,
    • KHR_MATERIALS_TRANSMISSION,
    • KHR_MATERIALS_UNLIT,
    • KHR_MATERIALS_VARIANTS,
    • KHR_MATERIALS_VOLUME,
    • KHR_TEXTURE_TRANSFORM,
    • EXT_TEXTURE_WEBP.
  • Draco decoding for DeepAR files.
  • Perform camera/video image reading asynchronously. This improves performance on Android devices.
  • New initialization hints "faceInit", "footInit" and "wristInit" that initialize the tracking as soon as possible.
  • Added API getClosestPointOnMesh, getTransformationBetween, setReadPixelsAsync.

v5.5.2 - 2024/01/19​

Changed:

  • The glasses face tracking stability is improved. Updated the rigid face tracking model.
  • By default, DeepAR rendering is now limited to 60 FPS. Before it was limited to 30 FPS. This can be changed with setFps API.

Fixed:

  • Using backgroundBlur with front camera doesn't work as expected.

v5.5.1 - 2023/12/22​

Changed:

  • Revert deepar.wasm is now compiled with simd support for better performance since it caused the SDK to crash on iOS Safari 16.3 and below.

v5.5.0 - 2023/12/21​

Breaking changes:

  • startCamera() API takes in an optional parameter mediaStreamConstraints which before expected only the video entry of the MediaStreamConstraints object. Now it takes the actual MediaStreamConstraints object.

Added:

  • Added support for WebP textures.
  • Lazy-loading of Javascript and Wasm modules.
    • DeepAR has a lot of functionality that may not be needed in each application and use-case. Some of that functionality is in Javascript and some is in Wasm modules.
    • DeepAR has been split into main and side Javascript and Wasm modules. Main modules are always loaded and side modules are lazy-loaded only when they are needed.
    • Main modules:
      • Javascript - 484 KB.
      • Wasm - 1.5 MB.
    • Side modules:
      • (JS) Background segmentation - 120 KB.
      • (JS) Glasses face, foot and wrist tracking - 919 KB.
      • (WASM) (Legacy) Face tracking - 818 KB.
      • (WASM) Scripting - 449 KB.
      • (WASM) Physics - 479 KB.
    • Note: All the sizes listed above are uncompressed files and those are not including the ML models and backends.
  • Added setFov() API for configuring the 3D rendering horizontal FOV. The default values are listed in the section below.
  • Added changePreviewElement() API for changing the previewElement which holds the DeepAR viewport.
  • Added WebGL2 Offscreen canvas to image pipeline. It is used whenever possible. Falls back to 2D Offscreen canvas or DOM canvas depending on what is available.

Changed:

  • Updated wrist tracking to V2.
  • Removed EngineDebug code and assets to improve WASM size and initialization.
    • Improved SDK initialization time by 80%.
    • Reduced WASM size by 0.3 MB.
  • deepar.wasm is now compiled with simd support for better performance.
  • 3D rendering horizontal FOV changes:
    • Mobile devices will use 85° FOV.
    • MacBooks will use 54° FOV.
    • Other will assume 90° FOV.
  • deepar.initialize() could be called several times in a row which would have undefined behaviour. So we check if initialize is called and throw if someone tries to call it before the initialization ends.
  • Removed most of the warnings that were outputted to console on DeepAR initialization.
  • Transition to native wasm exceptions rather then Emscripten emulated exceptions. Reduces wasm size b 0.2 MB.

Fixed:

  • Performance issues introduced in iOS 17 Safari private browsing due to privacy and fingerprinting updates.
  • Glasses face, foot and wrist tracking not working properly after the input camera/video resolution changes.
  • Glasses face tracking not able to initialize in some environments due to invalid OBJ file fetching.
  • Glasses face, foot and wrist tracking fails when the background in the input camera/video is static.

v5.4.4 - 2023/10/13​

Added:

  • optional RequestMode arguments for changeParameterTexture and backgroundReplacement.

v5.4.3 - 2023/09/21​

Fixed:

  • background segmentation regression.

v5.4.2 - 2023/09/15​

Fixed:

  • Shutdown crash.
  • touchEvent handling on mobile platforms.

v5.4.1 - 2023/08/17​

Reverted:

  • Reduced the initial size of deepar.js file from 1MB to 200kb. DeepAR Web will now lazy-load additional modules when it needs them.

v5.4.0 - 2023/08/11​

Added

  • DeepAR now takes in previewElement parameter at initialization. That is an HTML element like <div> which will be used for AR preview placement and sizing. It is intended to be used instead of canvas parameter which is now deprecated. This approach is now used so users do not have to manually crete and manage the canvas.
  • New method: getCanvas() which returnes the underlying canvas used for rendering.

Changed

  • Improved background segmentation performance.
  • [BREAKING CHANGE] Improved video recording. Videos are now produced in mp4 format that is compatible with the most devices. Audio capture from microphone is now available as well. The startVideoRecording() function is now async and returns Promise<void>. Before it was a plain function that returned void.
  • Reduced the initial size of deepar.js file from 1MB to 200kb. DeepAR Web will now lazy-load additional modules when it needs them.
  • The touchOccurred() function is now deprecated. DeepAR will now track the touches on the preview out-of-the-box.

Fixed

  • Freezing that occurred when the background segmentation effects were loaded for the first time.
  • Frozen camera on iOS Safari when using startCamera().

v5.3.0 - 2023/07/17​

Added

  • glTF transmission support.
  • Background blur and background replacement API.
  • Mute/unmute all the sounds coming from DeepAR.
  • HDR textures support.

Changed

  • Improved foot and wrist tracking.
  • Audio can now be played from multiple animation controllers at the same time. Previously, only one sound could be active.

Fixed

  • DeepAR would sometimes crash on shutdown.
  • Unloading effect from the unkwnown slot could break the currently loaded effect.
  • Crash caused by effects with scripting.
  • Audio not playing if part of initial node in animation controller.

v5.2.0 - 2023/05/23​

Added

  • Wrist tracking.
  • Chrome extension support. Get in touch with us for more info.
  • CameraPermissionDenied error is now thrown when DeepAR requires camera from the browser, but the permission is denied.

Changed

  • Removed the polygon count limit in the scene which was ~100k before.
  • Improved rendering performance with regard to mesh instances.

Fixed

  • Simple pendulum direction not consistent for front and back camera.
  • Sound not playing if part of the first state.
  • Crashing when passing null to changeParameter API for the component parameter.

v5.1.1 - 2023/03/31​

  • Fix front camera problems with beauty filters.
  • Fix camera sync issues.
  • Fix meshes using bones (but not face bones) disappearing when face not visible.

v5.1.0 - 2023/03/24​

  • Added scripting interop API.
  • Changed startCamera() can now be called again without stopVideo().
  • Fix 2D face landmarks not being mirrored properly.
  • Fix multiple calles to changeParameterTexture crashes the app.
  • Fix blend-shapes are inverted on effects using 2D face landmarks like eyes and lips.
  • Fix changing video sources scrambles up the effect textures.

v5.0.0 - 2023/03/15​

  • Completely new modern and user-friendly API. See migrating to v5
  • Improved error handling.
  • Reduced foot tracking initialization time and improved performance.
  • switchEffect() now resolves after the effect is loaded and all ML models are loaded. This mainly concerns switching effects with background segmentation or foot tracking which have somewhat longer first initialization time.
  • Now using WebGL2 canvas context for rendering instead of WebGL.
  • Fix non power of 2 textures not working.
  • Fix reduced depth precision when using postprocessing or render textures.
  • Fix eye position not working.

v4.1.0 - 2023/02/14​

  • Updated the foot tracking API:
    • Added the initializeFootTracking method. Accepts a callback that is triggered when foot tracking is initialized.
    • Added the onFeetTracked callback which gets called when feet are tracked.
  • Greatly improved foot tracking's accuracy.

v4.0.3 - 2022/11/11​

  • Fixed foot tracking when changing the video feed.
  • Render error message in case of too many polygons in effect.
  • Clean up logging.

v4.0.2 - 2022/10/26​

  • Fixed foot tracking not working.
  • Added texture size uniform that is updated every frame for every texture.
    • In shaders it is accessible with TEXTURE2D_SIZE(sampler_name) macro.

v4.0.1 - 2022/10/20​

  • Updated the Scripting API:
    • Added the Face Position component class.
    • Added trigger events and functions.
    • Fixed Debug.renderText() default color parameters.
  • Added MIME type parameter for video recording.

v4.0.0 - 2022/09/16​

  • SDK is now written in TypeScript and bundles appropriate .d.ts files with classes, types, methods and documentation.
  • SDK is now available on NPM npm install deepar.
  • Performance improvements.
  • 40% smaller JavaScript file and 20% smaller WebAssembly file.
  • New API:
    • setOffscreenRenderingEnabled - Force rendering on canvas even when tab is not focused.
  • Fix memory leak in processFrameRawPixels.
  • Fix caching u_modelNormal uniform when using the same material on multiple mesh renderers. More details: https://forum.deepar.ai/t/pbr-shader-web-sdk/71/16
  • Breaking changes:
    • DeepAR is now a class and is called with new keyword let deepAR = new DeepAR({ ... }).
    • Constructor parameters:
      • canvasWidth and canvasHeight - Passing these parameters has no effect anymore. In previous versions of SDK, these parameters would resize the canvas. You could change the canvas size with setCanvasSize() method. In new SDK, DeepAR will adapt to the canvas size. So you just need to set the size of the canvas. If canvas is resized, DeepAR will pick it up automatically.
      • numberOfFaces - This parameter has no effect anymore. DeepAR will track minimum number of faces according to the current configuration of loaded effects.
      • segmentationInfoZip - Path to the segmentation (background removal) model. This parameter is now changed to segmentationConfig.modelPath.
      • deeparWasmPath - Path to the deepar.wasm file.
    • Callbacks
      • You can provide callbacks in the constructor of the DeepAR class in the callbacks parameter.
      • Add, change or remove callbacks via DeepAR.callbacks property.

v3.4.5 - 2022/09/08​

  • Add setFaceDetectionSensitivity function.
  • Add scripting console log API.

v3.4.4 - 2022/08/12​

  • Introducing foot tracking as a beta* feature!
    • Allows you to easily create a shoe try-on app using the DeepAR Web SDK.
    • *Foot position component available only in DeepAR Studio 4.0-beta.
  • Breaking API changes:
    • Removed the libPath parameter.
    • Consequently, segmentationInfoZip should contain the full relative path to segmentation.zip.
  • Added new Dynamic UVs options in SDK and Studio.
    • For dynamic UVs you can choose between face and real options.
    • face option's behaviour is the same as it was before - UVs are adjusted for sampling the person’s face from camera texture.
    • real option adjusts the UVs so that the camera texture is sampled based on the real positions of the mesh vertices.
  • Fixed dynamic UVs not working for nodes that are not in face space.
    • Dynamic UVs now work for everything:
      • Nodes in orthographic layers.
      • Nodes in perspective layers.
      • Nodes in face space.
      • Nodes in face space with parent or child node transformations.
      • Nodes not in face space.
  • Fixed undefined symbols in the FaceData struct in onFaceTracked callback function.
  • Improved segmentation performance.

v3.4.3 - 2022/07/28​

  • Fix landmark2D camera mirror bug
  • Fix faceRect empty bug

v3.4.2 - 2022/02/24​

  • Greatly improved face tracking performance (up to 100%).
  • The numberOfFaces parameters is now removed when initializing DeepAR as we can determine the number of faces required in the runtime.

v3.4.1 - 2021/12/07​

  • Fix for switch effect with path and face parameter not working.

v3.3.0 - 2021/11/15​

  • Fix crash on shutdown.
  • New particle system feature

v3.2.0 - 2021/10/04​

  • Improved touch handling.

v3.1.3 - 2021/09/20​

  • No changes on Web SDK

v3.1.2 - 2021/09/17​

  • Web recording

v3.1.1 - 2021/09/15​

  • Bugfix in processImage(image) where the image didn't display properly when loading some image formats and changing resolutions.
  • Fixed scripting subsystem resource leaks
  • General refactoring and bugfixes

v3.1.0 - 2021/08/20​

  • Want to have more control over physics components in runtime? You can now enable/disable entire physics with enablePhysics method or show/hide physics colliders showColliders method. Also, you can change any physics component parameter through our ChangeParameter API
  • General refactoring and bugfixes

v3.0.0 - 2021/07/23​

  • New API reference pages
  • News features: physics and scripting
  • Core Engine optimisations and refactor

v2.4.0 - 2021/03/26​

  • Improved background segmentation

v2.2.1 - 2020/10/02​

  • Web background segmentation added
  • moveGameObject API method added (check API docs for more detail)
  • exposed fireTrigger API method to trigger custom animation state triggers
  • Fixed setVideoElement call where the main processing loop was not started after the call
  • Stability and bugfixes

v2.2.0 - 2020/09/07​

Major upgrade.

  • Added ability to process a single image in addition to the camera stream
  • Added ability to change bool and texture parameters to changeParameter API
  • Bugfixes and stability issues

v2.0.0 - 2020/07/22​

  • Major upgrade
  • New segmentation model
  • Tracking improvements
  • Bugfixes and optimizations

v1.4.6 - 2020/02/28​

  • Fixed issue with numberOfFaces less than 4

v1.4.5 - 2019/10/09​

  • Get face data for each face
  • Enable multi face tracking during the initialization

v1.4.4 - 2019/10/04​

  • Change transform API

v1.3.2 - 2019/01/31​

  • External camera

v1.3.1 - 2019/01/08​

  • Multi face tracking
  • setCanvasSize function to reset the viewport size

v1.3.0 - 2019/01/02​

  • Added onFaceTracked

v1.2.5 - 2018/11/05​

  • Added audio playback

v1.2.4 - 2018/10/18​

  • Improved API and documentation
  • Transition to Web Assembly