Machine Learning on iOS: Computer Vision

A catalogue of Heartbeat posts covering the use of computer vision-based machine learning tasks on iOS

Heartbeat Collections

Computer Vision on iOS

With recent advances in both iPhone camera and AI-accelerated chip technology (i.e. the A13 Bionic in the iPhone 11 series), it’s no wonder that some of the most innovative, transformative use cases for machine learning on iOS come in the form of computer vision.

From understanding scenes and creating artistic masterpieces to tracking human movement and even changing hair color, the promise of real-time computer vision on iOS is real.

But it can be tough to know where to get started. That’s why we’ve worked to curate and assemble an authoritative collection of articles and tutorials that explore what’s possible with computer vision-based machine learning on iOS.

Automatically Pixelate Faces on iOS using Face Detection with Native Swift Code

Leveraging the native Swift library to perform face detection in an iOS app

— by Omar M’Haimdat

Safer and Smarter: Contactless shopping with on-device object detection

Scaling out to a cloud platform for fast model training, evaluation, inferencing, logging, and monitoring.

— by Jamshed Khan

Build a SwiftUI + Core ML Emoji Hunt Game for iOS

Create a fun machine learning iOS camera app that lets you search for things in your house that are similar to emojis.

— by Anupam Chugh

Semantic and Instance Segmentation on iOS Using a Flask API — DeepLabV3+ and Mask R-CNN

Build an API that performs image segmentation and consume it with an iOS application

— by Omar M’Haimdat

Implement Depth Estimation on iOS Using a FCRN Model

Predict the depth of a scene and estimate how close an object is to the camera.

— by Omar M’Haimdat

Using TensorFlow.js in a Native iOS App to Perform Object Detection

Browser-based machine learning — in an iOS app.

— by Konrad Mokiejewski

Face Recognition and Detection on iOS Using Native Swift Code, Core ML, and ARKit

Leveraging the native Swift library to perform face recognition and detection in an iOS app.

— by Omar M’Haimdat

Scanning Credit Cards with Computer Vision on iOS

Leverage Vision’s Rectangle Detection and Text Recognizer to detect credit and other business cards in a live camera feed.

— by Anupam Chugh

Build a Touchless Swipe iOS App Using ML Kit’s Face Detection API

Leverage ML Kit’s Face Detection API to perform swipe gestures using blinks, winks, and head turns

— by Anupam Chugh

Compute Image Similarity Using Computer Vision in iOS

Determine the Euclidean distance between images using their feature prints to determine image similarity on iOS.

— by Anupam Chugh

Training a TensorFlow Lite model for mobile using AutoML Vision Edge

Learn how to leverage Google’s AutoML Vision Edge training framework to train a custom image classification TensorFlow Lite model that’s ready for mobile deployment.

— By Harshit Dwivedi

Computer Vision in iOS: Determine the Best Facial Expression in Live Photos

Work through an iOS implementation of Vision framework’s new face capture quality request to programmatically select the best video frame from a Live Photo.

— by Anupam Chugh

License Plate Recognition, Detection, and Plate Number Extraction on iOS

Creating an iOS application to recognize, detect, and extract license plate numbers.

— by Omar M’Haimdat

Incorporating machine learning into iOS apps

Learn how to get started with machine learning on iOS: Classification using Core ML and Vision.

— by Pradnya Nikam

Building a Barcode Scanner in Swift on iOS

Learn how to use Apple’s Vision framework to build an iOS app that can scan barcodes and return information about what’s being scanned.

— by Rick Wierenga

[Text Recognition] Building an iOS camera calculator with Core ML’s Vision and Tesseract OCR

Using Core ML’s Vision in iOS and Tesseract, learn how to build iOS apps powered by computer vision and optical character recognition.

— by Khoa Pham

[Object Detection] Building a real-time object detection iOS app that detects sushi

Learn how to build an iOS app that can see and detect objects in real time.

— by Junji Watanabe

[Image Segmentation, Object Detection, AR] Hand Detection with Core ML and ARKit

ARKit allows mixing virtual objects and real world environment. In this post we explore how we can make our real hands interact with virtual objects using machine learning and Core ML in particular.

—by Gil Nakache

[Image Classification] Detecting Pneumonia in an iOS App with Create ML

Learn how to use Create ML to train and implement your own image classification model in an iOS app.

— by Özgür Şahin

[Gesture Recognition + AR] Using Core ML and ARKit to Build a Gesture-Based Interface iOS App

Harnessing mobile machine learning and augmented reality on iOS to browse the Internet with hand gestures.

— by Bruno Muniz

[Image Classification] Using Core ML and Vision in iOS for Age Detection

Learn how to build an iOS app that detects age using Core ML and Apple’s Vision framework.

— by Sayalee Pote

[Image Classification] Using Core ML and Custom Vision to Build a Real-Time Hand Sign Detector in iOS

Learn how to train your own Core ML model and integrate it in an iOS app with Custom Vision.

— by Sayalee Pote

[Image Classification] How to fine-tune ResNet in Keras and use it in an iOS App via Core ML

An end-to-end tutorial that shows you how to fine-tune a Keras model, convert it to Core ML, and integrate it into an iOS app.

— by Özgür Şahin

[Text Recognition] Intro to machine learning on iOS: Using Core ML to recognize handwritten digits

A crash course on using machine learning in iOS app development — handwritten text recognition.

— by manu rink

[Image Classification] Training a Core ML Model with Turi Create to Classify Dog Breeds

Learn how to train an image classification Core ML model using Turi Create — to classify dog breeds!

— by Vardhan Agrawal

[Image Classification] Building Not Hotdog with Turi Create and Core ML — in an afternoon

Recreating the (in)famous Not Hot Dog app from HBO’s Silicon Valley using Turi Create and Core ML.

— by Jameson Toole

[Image Classification] Making a “Pokédex” for iOS Using Create ML and Core ML with Vision

Learn how to use Apple’s machine learning frameworks to build a custom image classification model that can identify different Pokémon in real-time.

— by Kyle Aquino

[Pose Estimation + Text Recognition] “Just Point It”: Machine Learning on iOS with Pose Estimation + OCR Using Core ML and ML Kit

A look a “Just Point It”, an iOS app that leverages machine learning to allow users to point at text on a paper document and look up the word that’s being pointed at.

— by tucan9389

[Image Classification] Moving AI from the Cloud to the Edge with Crowd Count and Apple’s Core ML

Learn how to build and implement a crowd classifier model that predicts crowd size, density, and more.

— by Dimitri Roche

[Object Detection] Evaluate Construction Site Safety on iOS using Machine Learning

Building an iOS application for safety on site with Swift, Turi Create, and Core ML.

— by Omar M’Haimdat

Style Transfer iOS Application Using Convolutional Neural Networks

Training a style transfer neural network using Turi Create to create artistic images.

— by Omar M’Haimdat

PyTorch Mobile: Image Classification on iOS

Learn how to implement an image classification machine learning model on iOS with PyTorch Mobile.

— by Anupam Chugh

Building a multi-class image classifier on iOS

Learn how to build an iOS app that uses deep learning to predict an image’s content based on twenty possible classes of food.

— by Navdeep Singh

Simple Semantic Image Segmentation in an iOS Application — DeepLabV3 Implementation

Quick overview of image segmentation and leveraging Core ML to use it in iOS applications.

— by Omar M’Haimdat

Avatar photo


Our team has been at the forefront of Artificial Intelligence and Machine Learning research for more than 15 years and we're using our collective intelligence to help others learn, understand and grow using these new technologies in ethical and sustainable ways.

Comments 0 Responses

Leave a Reply

Your email address will not be published. Required fields are marked *