Blog

The promise of being able to drop your own custom neural networks into Lens Studio as Lenses, which can then be deployed to millions of devices, is potentially game-changing.

But working with a tool this powerful and versatile inevitably involves some nuances you’ll need to consider while building.

While working through a demo project (stay tuned for a tutorial soon), I was able to identify some key areas where working with custom ML models required some tweaking, fine-tuning, and adaptation.

Continue reading “Exploring SnapML: Working with Custom Neural Networks in Lens Studio”

Imagine you have to read a document that’s very dense and has numerous words you don’t know the meanings of. What would you do?

The answer seems obvious—get out your phone, open a search engine or online dictionary, and search for the word’s meaning.

Instead of typing, what if you could instead find out all you needed to know about a word, displayed on your smartphone, just by pointing at the word on the document?

Continue reading ““Just Point It”: Machine Learning on iOS with Pose Estimation + OCR Using Core ML and ML Kit”

Sentiment analysis refers to the use of natural language processing, text analysis, computational linguistics, and other techniques to identify and quantify the sentiment (i.e. positive, neutral, or negative) of text or audio data.

Because it’s really hard for a model to learn language when only provided with a single value — the sentiment — FastAI lets you first train a language model — a model that predicts the next word — and then use that encoder for the model that actually classifies the sentiment.

Continue reading “FastAI Sentiment Analysis”

Smartphones are ideal devices for machine learning because of the sheer number of sensors they have. Combining data from multiple sensors at the same time can allow developers to make more accurate and quicker predictions inside their apps.

Today almost every smartphone comes with location sensors that provides user’s geolocation with high accuracy.

Continue reading “Leveraging AI with Location Data in Mobile Apps”

Machine learning models are often black boxes to end users. Without access to the underlying model architecture and parameters, they are nearly impossible to reconstruct with inputs and outputs alone.

Hosting a model in the cloud effectively prevents access to these underlying structures. Without breaching the hosting servers, an attacker has no access to the model: they can’t look at the layers, get the trained weights, or even see the framework it’s running on.

Continue reading “Reverse Engineering Core ML”

Alie Jackson has pretty much done everything. An extremely talented (and awarded) multimedia artist, she’s worked with everyone from Nike to Disney to Sony Pictures.

The Most Artistic Lens Creator of Lens Fest 2019, her work, a blend of AR and traditional media, is downright cool. Insightful, funny, bright, engaging, beautiful — I don’t have enough adjectives to describe Jackson’s style and work. You’ll just have to see for yourself.

Continue reading “Snapchat Lens Creator Spotlight: Alie Jackson”

Though it didn’t get a ton of stage time during the keynote at this year’s WWDC, there’s a lot to be excited about in the latest iteration of Apple’s machine learning framework, Core ML 3.

In this post, I’ll highlight the biggest changes to the software and discuss their implications for developers and machine learning engineers.

Continue reading “What’s new in Core ML 3”

Companies across all industries are exploring the opportunities AI holds for their business models. While identifying AI use cases is challenging, actually implementing them adds a whole new layer of complexity.

In this process, one of the most fundamental questions you have to answer is whether to build or buy. As we’ll see in this post, the answer to this question is rarely a simple “either / or” decision.

Continue reading “The build or buy decision for AI”

A couple of months ago, I wrote an article about training an object detection Core ML model for iOS devices. What made this tutorial unique, though, was that I used a tool I’d built called MakeML, which allow you to start training neural networks literally in minutes. The only model type available to train in that version was a tinyYOLO based Turi Create model.

After publishing that post, many people wrote to our team letting us know that the Turi Create framework doesn’t include enough flexibility to train their models.

Continue reading “Train a MobileNetV2 + SSDLite Core ML model for object detection—without a line of code”