tl;dr - Making experiences in Augmented Reality is fun; Crema decided to try our hand at a holiday-themed filter; we created a neat scene-switching mechanic; our project file got corrupted and our friend helped us solve it; we learned valuable lessons along the way.
---
Great Augmented Reality experiences are like magic: interactive, always within reach, and integrate seamlessly with the world around you. They let you see into another world where anything is possible. What makes them successful is a solid connection between you - the user - and an experience that is both natural and engaging.
But like all technologies, AR has its tradeoffs. Interactions with the technology occur within a small portion of your vision, power and battery life is often limited, and sometimes cellular network speeds can even affect the final output.
This is why building a convincing experience is so important when transporting users through that tiny portal in their hands.
Setting the scene
Back at the end of 2020, everyone at Crema was settling in for the holidays and we wanted to send off the strange year with a bang. One idea was to play with a themed AR effect for Instagram that would add a little fun and some positive seasonal vibes.
This also coincided with a rise in bobblehead-adjacent discussion around the office, thanks to an internal video featuring toy-sized likenesses of our founders, George and Dan. (You never know what you’ll find being announced in Crema’s Slack channels!)
It was decided: let’s make an AR effect where the user could place themselves into a mini festive scene. What would be more fun than putting your face on a wobbly toy surrounded by holiday decorations? After doing a bit of research and finding a severe lack of quality bobblehead content on the platform, it was time to dive in and play.
Like many creative ventures, it wasn’t enough to just make one scene, we could do better! To continue learning and growing in our craft, we decided to take on an additional technical challenge: switching between multiple environments.
Planning and shopping
Now that we had a theme (bobbleheads and several holiday settings) plus a simple gamification mechanic (switching scenes for user customization), it was time to do the fun part and start building!
Spark AR was our tool of choice, thanks to the ease of use when creating AR effects for Instagram or Facebook. It’s an incredibly powerful tool for AR content creation and has a low barrier to entry for creators of all skill levels. Anyone who has used the tools available in our rich ecosystem of 3D toolkits these days can attest to the power and relative ease of producing high-end work with these free applications.
We found that sketching out the scene was a great place to start, as it helps define the user’s perspective into our miniature digital world. This practice also provides a helpful constraint before traveling down the rabbit hole of fun features, since not all ideas serve the greater purpose of our scene’s story.
It's vital that users remain at the center of the action. If they cannot engage with the world you have created, everything you build will be both confusing to use and ultimately produce lower engagement.
We used free and paid 3D model sources - CGTrader, Free3D, and Sketchfab, to name a few - for almost everything in these scenes. This allowed us to move quickly through rapid prototyping and spend our time on the interactions instead of creating custom assets.
If this were an experience for a client, brand, or social cause, using proprietary assets would be another important way to convince users that they were deep within the fabricated scene. Users know right away when something feels fake or out of place.
Shrinking and growing
Creating mini-worlds within these AR effects is a highly experimental process. Because this effect centers around placing the user's face on a model in 3D space, the angle has to be just right to sell the illusion. And since switching between different scenes is a really important part of this effect, it was worth taking time to establish this at the very start.
3D assets, lighting, and environmental effects are slowly layered in as the scenes take shape. The more that the effect has to do, however, the more "weight" we have to deal with when processing the final output. We’re trying to cram three scenes into one, which presents fun challenges for resource management.
Then, after demos had been passed around and we started to feel really good about the interactions, we hit a huge blocker. It ultimately halted the publishing of this effect entirely. We had an asset that couldn't be properly removed from the Asset Manager due to an error within Spark AR’s file structure.
We weren’t able to adhere to publishing guidelines after that point, as the large object was always present and the scene expanded beyond the allowed filesize. Honestly, this was something that we hadn't seen before, so it was doubly frustrating at the time.
Learning requires failing
After a lot of trial and error, our team decided to pivot away from making this effect externally available over the holidays. Instead, we’d use it as an internal learning exercise and fuel for this eventual blog post. We create a lot of content at Crema and ultimately value Constant Improvement above finished products when exploring new things during our “Lab” times.
We did eventually figure out how to remove these locked/hidden/corrupted assets, thanks to friend-of-the-agency, Luke Hurd. He's an instructor and creator in the AR space and had dealt with this bug in his work as well.
If you find yourself up against a similar wall, know that you can unpack the contents of the scene file as an archive and simply delete the troublemakers from the package directly. Hopefully, we’ll see this patched in future versions of Spark AR.
Building new playgrounds
We have played with augmented reality in some of our Lab Friday contexts over the past several years. It’s a great place to learn more about image recognition, camera overlays, virtual characters, and more! All these amazing technologies get wrapped in a fun little world that you can carry in your pocket and access anywhere.
This effect may have ended in “failure” since we weren't able to launch it on Instagram. If we didn’t have a culture of safety around experimentation, that would be a really hard place to land. But we always look at learning opportunities as successful ventures, so long as we share our findings with others and leverage those lessons for next time.
