Quick answer: Virtual try-on is augmented reality technology that overlays a digital version of a product, glasses, makeup, shoes, or clothing, onto a live camera feed of your face or body so you can see roughly how it would look before buying. It works by tracking your face or body in real time and mapping the product onto that tracked surface. Retailers use it to help shoppers browse and compare items they cannot physically handle online.
The technology has moved fast. Early versions overlaid a flat rendered product onto tracked feature points; newer ones use generative models to synthesize how real fabric would actually drape on a specific body. This entry covers how virtual try-on works mechanically, where retailers use it today, how generative AI changed it, and where it still falls short.
What Is Virtual Try-On?
Virtual try-on is a category of augmented reality software that places a computer-generated product onto a live video feed of a real person. Point a phone camera at your face and a pair of glasses appears anchored to your nose and ears; point it at your feet and a sneaker appears wrapped around them.
Magic-mirror kiosks tried a version of this in physical stores over a decade ago, but it became a mainstream shopping feature only once phone cameras got good enough to track a face or body reliably in real time.
Two things separate virtual try-on from ordinary product photography: it responds to your own specific face or body shape rather than a generic model, and it usually runs live on your own device. That immediacy is also why it works as a shopping tool rather than a novelty filter: you can compare several frames or shades back to back in seconds instead of ordering each one to check.
How Does Virtual Try-On Work?
Most virtual try-on tools start with face tracking or body pose tracking. A machine learning model finds landmark points, the corners of your eyes, the bridge of your nose, the outline of your shoulders, and follows them frame by frame as you move.
Those landmarks get stitched into a rough 3D mesh of your face or body. For glasses or makeup, the product renders onto that mesh and locks to it, so a virtual frame stays put on your nose even as you turn your head. For shoes or clothing, the system estimates body pose and scale so the item wraps convincingly around a limb or torso.
Depth-sensing cameras make this more accurate by capturing actual distance data instead of inferring shape from a flat image, though most consumer try-on tools work with a standard camera and a trained model that estimates depth from ordinary 2D video instead.
Occlusion is one of the harder problems to get right. If a hand passes in front of the face, or a sleeve should fall behind an arm rather than in front of it, the system has to understand depth ordering well enough to hide the parts of the virtual product a real object is blocking. Poor occlusion handling is usually the fastest way to spot a rough try-on demo, the product appears to float in front of everything rather than sit believably within the scene.
Where Is Virtual Try-On Used?
Eyewear was one of the first categories to adopt it at scale. Warby Parker’s app lets customers use a photo or a live camera feed to see how different glasses frames sit on their own face before ordering a pair.
Beauty brands followed a similar path. L’Oreal acquired the beauty technology company ModiFace in 2018 specifically to build augmented reality engines for virtual makeup try-on, mapping lipstick shades or eye makeup onto a shopper’s own face on camera.
The camera apps that popularized real-time face filters built much of the underlying tracking technology this relies on. Snapchat has offered real-time face-tracking lenses since 2015, the same core mechanism, tracking facial landmarks and rendering something onto them live, that beauty and eyewear brands later adapted for shopping.
Clothing and footwear took longer to catch up because they depend on tracking a full body in motion rather than a mostly still face. Early attempts often locked a shoe or jacket to a rigid 3D model that barely reacted to how a person actually stood or moved, which is a large part of why this category lagged behind eyewear and beauty for years.
Some clothing retailers take the idea further by building a reusable digital avatar from a shopper’s own photos, so new garments can be draped onto a stored body model instead of re-tracking a live camera feed every time.
How Did Generative AI Change Virtual Try-On?
Older virtual try-on rendered a product as a separate layer wrapped around your tracked body, which looked convincing for rigid items like glasses but often looked pasted-on for soft goods like a t-shirt or dress.
Diffusion models changed that by generating the clothed image directly instead of overlaying a rendered garment on top. IDM-VTON, a 2024 research project, showed a diffusion model taking a photo of a person and a photo of a garment and generating a new image of that exact person wearing that exact garment, keeping fine details like fabric texture and print that earlier systems tended to lose.
The result looks closer to a photograph than a rendered overlay, with folds and shadows that respond to the actual pose in the photo rather than a generic body template. It is also a heavier computation than live face tracking, so most generative try-on still runs on a server rather than instantly on your phone the way a glasses filter does.
What Are the Limits of Virtual Try-On?
The biggest gap is fit versus look. Virtual try-on, generative or not, shows an approximation of appearance; it does not measure whether a shoe is a half size too small or whether a fabric will stretch the way you expect.
Lighting, camera quality, and how well the tracking model handles a given face shape or skin tone all affect how convincing the result looks, and tools trained on a narrow range of faces or bodies tend to work less reliably outside that range.
Generative try-on carries its own version of this problem. Because a diffusion model is synthesizing pixels rather than overlaying a known 3D object, it can invent plausible-looking texture or fit detail that is not actually accurate to the real garment, a subtler failure than an obviously misplaced overlay, and one that is harder for a shopper to catch at a glance.
None of that erases the value virtual try-on offers, narrowing down options and catching an obviously wrong color or style before checkout, but it is worth treating the preview as a rough guide rather than a guarantee.