At the Augmented World Expo, Snap showcased an early version of its real-time, on-device image diffusion model, promising to revolutionize augmented reality (AR) experiences. The company also introduced new generative AI tools for AR creators.
Snap co-founder and CTO Bobby Murphy highlighted that this new model is compact enough to operate on smartphones and swift enough to re-render frames in real time based on text prompts. He emphasized the need for faster generative AI image diffusion models to make a significant impact on AR, which has driven Snap’s efforts to enhance the speed of its machine learning models.
Snapchat users can expect to see Lenses incorporating this generative model in the coming months, with plans to make it available to creators by the end of the year.
“This and future real-time on-device generative ML models represent an exciting new direction for augmented reality,” Murphy stated, suggesting a new approach to rendering and creating AR experiences.
Launch of Lens Studio 5.0
Murphy also announced the launch of Lens Studio 5.0, offering developers new generative AI tools that significantly expedite the creation of AR effects, potentially saving them weeks or even months of work.
AR creators can now produce selfie Lenses with highly realistic machine learning (ML) face effects and generate custom stylization effects that transform the user’s face, body, and surroundings in real time. Additionally, creators can quickly generate 3D assets to include in their Lenses.
Advanced AI Capabilities
Snap’s advancements also allow AR creators to generate characters such as aliens or wizards using text or image prompts through Face Mesh technology. This technology enables the quick generation of face masks, textures, and materials within minutes.
The latest version of Lens Studio includes an AI assistant to support AR creators by answering questions they may have, further enhancing the creative process.
Snap’s innovations in generative AI and real-time AR rendering mark a significant step forward in the development of immersive and interactive AR experiences.
