Effect Creation Guidelines
Effect creation in DeepAR
Whether you’re just getting started in augmented reality or you're a seasoned 3D pro, you’ll be able to create awesome things in DeepAR!
In this document, we’ll share tips and best practices for creating the best quality augmented reality assets for DeepAR using DeepAR Studio.
As with all creative processes, a great first step is to gather references and plan out how your filter idea is going to look in the end, using images, sketches and simple mockups. Starting without a clear idea makes it much harder to achieve great results. Experimenting and changing your mind is of course inevitable, as soon as you go back to this stage once you have a plan. The 3D creation part can be frustrating without a clear vision. Try things out and explore ideas now, before moving forward.
You can join the DeepAR Community Forum to get and share ideas with other people who are making cool things!
3D Model Preparation
In this step, you can use your 3D tool of choice to prepare your model. Any program that can export files as .fbx is fine. If you’re unsure where to start, we recommend Blender as a great first step (and it’s free!).
Use the reference Dense Mesh (or any other of the Useful Assets, depending on your project) and use it as a spatial reference for your filter. You can start building your filter around this model. You can always move and scale things later in the DeepAR Studio if needed, it will save you time later if you place everything correctly, to begin with.
To create assets that avoid complex and invalid topologies, use quads and the edge loops should follow the flow of the mesh. If you are buying or 3D scanning objects you might want to retopologize first, or at least optimize them as much as you can.
Good steps for optimization include:
- Convert the mesh to quads
- Fix topology
- Reduce polycount
- Fix normals
- Remove overlapping edges or faces
Low poly count
Keep the poly count as low as you can. This means avoiding excessive smooth passes or unnecessary geometry that you are not able to see on the screen. This will keep the file size low which will run more smoothly, especially on older or less powerful devices. Maximum file size of 10MB for a final filter is a good target.
You should unwrap the UVs of all 3D meshes that are going to need textures. It’s best to put them all in the same UV space with 0 to 1 window and make sure they don’t overlap.
Try to use as much of that 2D square as possible (the larger the UV isles are, the better quality you’ll get from your textures). The pieces of geometry that are physically larger or are more frontal/important in the filter should have larger UVs (tiny pieces won’t need a lot of UV space since you won’t be able to see them close up anyway).
Before you export your FBX, make sure all of the meshes have no transformations (reset and freeze transformations), so your pivot points are all centered in (0,0,0) and your translation and rotation values are all set to 0 and scale values are 1.
Experiment with shaders and maps
It’s a good idea to experiment with the shaders provided within the DeepAR Studio, understand how they work and choose which shaders work best for you. Then think about the maps that you need for that shader. Albedo (color), AO and Normal maps are the most common.
Optimize your textures
The smaller the textures you have, and the smaller the final file size, the better your effect will perform. Keep in mind also the size of your elements in your final filter composition and if your creation is for a smaller screen (like a mobile phone) larger screen like a TV.
Import your FBX and texture files, then set up the shaders and animations, if you have any.
Preview & Test
You can use the live preview and test on device features to see how your effect will look, then export it ready for use.
Test your effect on a mobile device with our effect tester apps for iOS and Android. Make sure to test on your target platform and device.
Export & Publish
You can export your effect ready for use in your project using the DeepAR SDK. For best results and to ensure your effect will run smoothly on a wide range of devices, check out the technical requirements below.
To integrate the DeepAR SDK you’ll need to know the basics of how to programme on that particular platform. Don’t worry if you don’t yet know how to code - you can certainly pick it up with the help of YouTube, find an official partner of ours that can help, or enlist the help of a friendly robot.
Let’s make sure your effects are not only super fun and top-notch but also run smoothly on all kinds of devices. This section gives you a bunch of tips on how to boost the performance of your effects and also lays out some technical guidelines for imported objects and textures.
While the hard limit on texture size is 2048px x 2048px, it’s best not to exceed 512 x 512. Larger textures often create slow loading times and affect tracking quality, especially on older devices. You might be surprised how smaller texture files can still produce high quality results.
We tend to use a rule of thumb used in the gaming industry: Both quality and performance matter, but performance matters more. People may enjoy a lower-poly experience that performs well (think Lego, Minecraft) but nobody tolerates a high graphic game that is jittery and jumpy.
For this reason, we recommend keeping the geometry as minimal as possible:
- It’s best to use quads and avoid funky geometry - it should be the default in most 3D tools anyway.
- For best performance, try to keep a poly count below 25k per mesh, with a total poly count below 100k.
- We recommended adding no more than 50 objects to a scene.
You can check the triangle count in your 3D modelling software. Here’s a great guide for finding it in Blender.
It’s a good idea to avoid complex blend shapes where possible, as well as using bones or animations only where necessary, as all of these things will impact the performance of your final effect.
You can also consider removing polygons that won’t be visible, for example the inside of shoes. Just like a stage set, we only need to see the facade for the magic to be believable. 🪄
Importing Models and Meshes
DeepAR Studio has handy built-in meshes to help with common tasks, such as Eyes2D, Foot Occluder, Sphere, etc. They can be found and imported into your project from Assets -> Add Mesh and we have already made sure they conform to the guidelines.
Static meshes: Has the smallest performance impact, so favor these if possible.
Bone-driven meshes: Use when tracking specific facial features, such as lips or chin.
Vertex-driven meshes: Use when working with the Dense Mesh (for face) and the eyes or lips 2D meshes (usually for makeup)
We recommend using face (Clockwise/Counterclockwise) culling if possible. Face culling improves performance by not drawing invisible triangles. You can improve the performance of your effects by selecting a culling mode other than ‘Off’ wherever it suits your project.
Errors and Warnings
Studio will try to be helpful by reporting issues with your project as they happen, so you can avoid any broken or unexpected effects.
You can find a list of all of your errors and warnings by opening the ‘Troubleshooting’ window (Windows -> Troubleshooting or Shift+CMD+T/Shift+CTRL+T)
❗️Errors (Red): Something is most likely broken, and will not work in your effect. It’s quite important to get these fixed as soon as you can since you can’t export effects with errors.
⚠️ Warnings (Yellow): `Something might work okay in for now, but might not on a different platform or cause an issue at some point in the future. Examples include: Shaders that will work, but not display in the Editor, textures that will work on Native SDKs, but not the Web, Duplicate names that might cause an issue when trying to change some parameters at runtime/ using Scripts, etc.
If you’re stuck with issues you can’t solve, your answer may already be on the DeepAR community forum, or if not, please do post and a member of staff or the community will be able to help. If you’d like to share your effect with us privately, you can report an issue.
Nodes and components
When adding the same component to multiple nodes, it’s usually best to add all nodes you’d like to receive the component under one parent node (in a ‘group’), then apply the component to the parent node.
This is because each instance of a component is yet another thing your user’s device will need to process, which can impact their experience.
For example, in this example, we have the Face Position component which will cause the Mask and Hat nodes to be tracked by the face:
A better organization would be to make a parent node called ‘Face’ and apply the Face Position component to it, then add both Mask and Hat as children:
It’s a good idea to check your project now and again for ‘dead nodes or components that no longer serve a purpose. As you try different things out, it’s quite common to end up with nodes or components that aren't doing anything anymore but are still impacting performance. Trimming these back will keep your effect as efficient as possible.
It’s best to avoid using the same node name twice. This can cause issues if you ever need to use scripting or change parameters at runtime because the node name is used to target parameters on this node.
The smaller the textures you have, the better your AR effect will perform. If your creation is for a smaller screen (like a mobile phone), or people aren’t likely to see that aspect of your effect up-close, you can consider using a smaller texture.
When creating textures you should make sure they are as small as possible. Textures above 512x512 start to increase the size of your effect export size significantly, so it’s best to only use them when needed.
Alpha channels (like transparent backgrounds) also increase the size of the texture, so it’s best to use them only if your texture has an irregular shape. Consider using compressed textures and small resolutions in order to get a faster loading time and better performance on your effect.
💡A good way to improve performance is to use ‘blend modes’ instead of transparent backgrounds in your texture. For example, like using ‘Multiply’ with all unnecessary areas of your texture in pure white or ‘Add/Lighten’ for a bright texture on a black background.
Other good tips are:
Ensure LUTs, normal maps, metallic maps, etc have been imported as non-color data
Ensure mipmaps have been turned off when importing LUTs
Ensure non-color data (LUTs, normal maps, metallic maps, etc) are in PNG format
Ensure color data has been imported as lossy
We attempt to do this automatically on import, but if something doesn’t look right, it’s best to check your texture settings. You can find them by selecting your texture and clicking ‘Change settings’ in the preview panel. From here, you can adjust the settings and re-import the texture.
For more information on texture importing, check out the Texture Import Settings tutorial.
For best performance on web
Due to restrictions out of our control, textures work best on the web when they’re square and have a width/height that is a power of 2, such as:
- 2 x 2
- 4 x 4
- 16 x 16
- 32 x 32
- 64 x 64
- 128 x 128
- 256 x 256
- 512 x 512
- 1024 x 1024
- 2048 x 2048
Video textures are not supported on the WebSDK, For other platforms, any resolution is supported, however, we recommend using square, power of 2 resolutions. Non-square videos can have "green line" artifacts on some Androids devices. For best results:
- Use .mp4 files with the h264 (AVC) codec
- Reduce the bitrate of your videos when encoding
- Audio from the videos can not be played, removing it from the video will save resources
- Use square, power of 2 video resolution
- Reduce the video resolution as much as possible. Try to not exceed 512x512
Try to optimize model animations. Ensure that the animation affects the common parent object, not every node individually. Try to make simple looping animations instead of overly long repeating ones.
Should follow the guidelines for regular textures regarding their sizing, alpha channels etc., but they must also be:
- The same size
- Contained in the same directory
- Named in alphanumerical order they need to be displayed in. Eg frame000000.jpeg, frame000001.jpeg, frame000002.jpg, etc.
It’s recommended to use the version of the shader with the least options. Eg. if you aren’t using the Texture slot of Unlit Texture Color, use “Unlit Color” instead.
Testing and final touches
We don't enforce a hard limit on effect size, but for the best results, aim to keep the size as small as possible. 5 MB is a great benchmark for mobile web effects, although it might not always be realistic to achieve if you’re making effects for larger screens like Smart TVs.
If your project effect will be bigger than 10 MB, Studio will display a warning. You will still be able to export the effect, although it might be worth double-checking if there are any optimisation steps you can still achieve before exporting your effect.
Project File Formats
This file is created when you export your effect from the DeepAR Studio. It can be used directly in the DeepAR SDK to display your effect.
This file is created when you creare a new project. It will contain all of the source files used to create your effect.