Quick answer: SnapML is Snap’s machine learning framework built into Lens Studio, letting creators drop a trained model into an AR Lens so it can recognize a specific object, gesture, or scene instead of just a generic face or floor. It is model agnostic: a creator imports a model trained elsewhere, tells Lens Studio what to feed it and what to do with the output, and SnapML handles running it efficiently on the phone. It powers the Lenses that do more than overlay a sticker, the ones that know what they are looking at.
What Is SnapML?
SnapML extends what an AR filter can do beyond Lens Studio’s built-in face and world tracking. Instead of only reacting to a face or a flat surface, a Lens built with SnapML can react to whatever the model underneath it was trained to recognize, a product, a pet, a hand gesture, a type of food.
The framework does not need to understand what is happening inside the model. A creator wires up its inputs and outputs, and Lens Studio treats the model as a component like any other piece of the Lens.
For anyone who has trained or deployed a small model for another purpose, the shape of this will feel familiar. It is the same pattern behind most on-device AI today: a compact model tuned for one narrow job, running entirely on the hardware in someone’s pocket instead of calling out to a server.
How Does SnapML Work?
A creator brings in a model that is already trained, then connects it to the Lens through an ML Component, which works much like the Image Component used for a texture. The component takes camera input, feeds it to the model, and exposes the model’s output for other parts of the Lens to use, triggering an effect, drawing a mask, or placing an object.
Everything runs on the device itself, in real time, with Lens Studio applying hardware acceleration where the phone supports it so the model does not stall the camera feed. That on-device constraint shapes the whole workflow: a model has to be light enough to run inside a live camera Lens, not just accurate in a lab.
The training itself happens elsewhere. A creator typically trains a model outside Lens Studio entirely, in PyTorch or TensorFlow, then exports it in a format SnapML can read. That split is deliberate: SnapML is built to be a bridge between a model and a Lens, not a place to train one from scratch.
What Models and Templates Does SnapML Support?
Lens Studio reads two model formats: ONNX, the open exchange format most training frameworks can export to, and TensorFlow Lite. A model built in PyTorch or full TensorFlow therefore has to be exported to one of those two before it reaches a Lens, and that conversion step is where most creators hit their first wall.
Lens Studio also ships seven starter templates, each one wiring a common type of model output to a working effect. Classification covers a model that returns a simple probability, such as whether the person in frame is wearing glasses. Object detection locates and labels something in the camera feed, and multi-object detection does the same across several classes at once.
The remaining templates handle other shapes of output. Style transfer takes a texture and returns a modified one, repainting the camera feed in a trained visual style. Custom segmentation and ground segmentation both mask at the pixel level, one for an arbitrary subject and one for the ground underfoot, while keyword detection covers audio models rather than visual ones.
Where Is SnapML Used?
SnapML sits underneath any Lens that needs to recognize something specific rather than just track a generic face or surface. A branded Lens that only reacts when a particular product is in frame, a pet Lens that tells a dog from a cat, a game that fires when a hand makes the right shape: each of those needs a model doing the recognizing before any effect can run.
The practical effect is that Lens creators are not limited to Snap’s own tracking systems. If a creator can train or find a compatible model for a task, whether that is spotting a specific product or classifying an object in the scene, SnapML gives them a path to wire it into a working Lens without writing a custom inference engine from scratch.
Ground segmentation is a good example of what that unlocks. Instead of only anchoring an object to a flat surface the way a basic world-tracking Lens does, a ground segmentation model can identify the walkable ground itself, letting an effect visually replace or transform that surface as the camera moves across it. That is a meaningfully different result from placing one 3D object at one anchor point, and it comes from swapping in a purpose-built model rather than writing new tracking code by hand.
SnapML vs General On-Device ML Frameworks: What’s the Difference?
SnapML is not a general-purpose machine learning runtime in the way TensorFlow Lite or Core ML are. Those frameworks let a developer run a model inside any mobile app for any purpose. SnapML is narrower by design: it is a plugin layer built specifically for Lens Studio’s rendering and tracking pipeline, meant to get a model talking to camera input and Lens effects, not to serve as a standalone inference engine outside that environment.
In practice, SnapML often sits on top of one of those broader formats. A model trained and exported as ONNX or TensorFlow Lite is the input; SnapML is the layer that connects it to a Lens’s camera feed, screen, and effects rather than replacing the training or export step itself.
What Are the Limits of SnapML?
A model still has to be small and fast enough to run every frame on a phone without draining the battery or dropping the frame rate, which usually means a creator compresses or optimizes a model before it fits comfortably in a Lens. Converting a model trained in a full desktop framework so it behaves correctly on-device is also not always a clean, automatic step.
SnapML is bound to Snap’s own platform too. A model built and tuned for a Lens does not transfer to TikTok’s Effect House or another platform’s AR tool without redoing that integration work elsewhere.
It also does not fix a badly trained model. A classifier that performs poorly in testing performs just as poorly once it is wrapped in a Lens, only now it is doing so live, on a stranger’s phone, in whatever lighting they happen to be standing in. That tradeoff, real recognition inside a Lens in exchange for staying inside one platform’s pipeline and one model’s limits, is what SnapML actually offers creators building past a basic tracked filter.