Quick answer: Core ML is Apple’s framework for running machine learning models directly on an iPhone, iPad, Mac, Apple Watch or Apple TV, instead of sending data to a server for a prediction. It takes a trained model, converts it into a format the operating system can load, and picks the fastest local way to run it across the CPU, GPU or the dedicated Neural Engine. Apple builds higher level frameworks like Vision and Natural Language directly on top of it, and developers use it on its own for anything from photo tagging to offline translation.
So what does that look like day to day? Core ML is a big part of why an iPhone can recognize faces in Photos, transcribe speech, and suggest the next word in a message while the device is offline and running on battery. This entry covers how Core ML actually runs a model, where it shows up across Apple’s own apps and third party ones, how it compares with Google’s TensorFlow Lite, and where it falls short.
What Is Core ML?
Core ML is not primarily a training framework. It is a way to package a model that was trained elsewhere, PyTorch and TensorFlow are common starting points, and run it efficiently on an Apple device. Apple introduced it alongside iOS 11 in 2017 as the standard on-device path for shipping a model inside an app.
Models reach Core ML in one of two ways. A developer can build one directly with the Create ML app bundled in Xcode, using drag and drop training for common tasks like image classification, or convert an existing model trained in another framework using the Core ML Tools Python library. Either path produces a .mlmodel file, or the newer .mlpackage format, that Xcode compiles into the app.
Because the model ships inside the app bundle, a prediction costs nothing per request and does not meter against an API budget. That is a different economic model from calling a hosted endpoint, and it is often the reason a small developer picks Core ML in the first place.
How Does Core ML Work?
Once a model is bundled into an app, Core ML compiles it into a representation optimized for the hardware it will run on. At prediction time, it chooses among three compute options, the CPU, the GPU, or Apple’s dedicated Neural Engine, and can split a single model’s layers across more than one of them.
The Neural Engine is Apple’s own NPU, and it is far more power efficient at neural network math than the CPU or GPU sitting on the same chip. Apple first added it to the A11 in 2017, but that version was reserved for Apple’s own features like Face ID, and third party apps could not use it at all.
That changed with the A12 in 2018, which opened the Neural Engine to third party apps. It is the reason a lot of Core ML advice from the framework’s first year no longer describes how a model actually runs today.
Developers can let Core ML pick automatically or set a compute policy that favors a particular unit, which is useful when a model needs to stay responsive while other GPU intensive work, like a game or a camera preview, is also running.
Core ML also supports updating a model after it ships, within limits. An app can retrain part of a model on data the device itself collects, so a handwriting recognizer or a recommendation model can adapt to one person’s usage without that data ever leaving the phone. Most apps still ship a model that was fully trained beforehand and never change it, since on-device personalization adds real complexity for a fairly narrow benefit.
Where Is Core ML Used?
Several of Apple’s own frameworks sit on top of Core ML rather than being separate systems. Vision handles image analysis tasks like face detection, text recognition and object tracking, while Natural Language covers language identification and named entity recognition.
Speech and Sound Analysis handle audio. All four hand their actual model execution to Core ML underneath, so a developer using them is using Core ML whether or not they ever touch it directly.
Third party developers use Core ML the same way, usually for the parts of an app that benefit most from running instantly and offline: classifying a photo as it is taken, applying a style filter, or scoring input as a user types, all without waiting on a network request.
On-device AI and edge AI are the general names for this pattern. Core ML is one vendor’s implementation of it, welded to Apple’s own chips and operating systems, which is both its main advantage and its main constraint.
Core ML vs TensorFlow Lite: What’s the Difference?
TensorFlow Lite, now called LiteRT, is Google’s equivalent idea: a compact runtime for running a trained model locally on a phone or embedded device. The two frameworks solve the same problem, but for different platforms.
Core ML only targets Apple’s own operating systems and only takes real advantage of the Neural Engine on Apple silicon. TensorFlow Lite and its successor run across Android, iOS, embedded Linux and microcontrollers, and lean on whatever neural accelerator or GPU a given device happens to have through a system of hardware specific delegates.
A team building only for iPhone usually reaches for Core ML because of how tightly it integrates with Xcode and Apple’s own vision and language frameworks. A team that needs the same app to run on Android too usually standardizes on TensorFlow Lite or a cross-platform toolkit instead, since a Core ML model will not run on anything but Apple hardware.
What Are the Limits of Core ML?
The most obvious limit is that Core ML is Apple only. A model packaged for Core ML cannot run on Android, on a server, or in a browser without being converted again for a different runtime, which usually means maintaining two separate model pipelines for a cross platform product.
Conversion itself is not always clean. Core ML Tools supports a wide range of layers and operations from PyTorch and TensorFlow, but a model built with a newer or unusual architecture can hit an operation Core ML does not support, forcing a developer to rewrite part of the model or fall back to a custom layer.
On-device compute is also still modest next to a cloud GPU. Core ML is well suited to compact, purpose built models, an image classifier, a small language model, a filter, but a full scale large language model generally will not fit comfortably on a phone, which is why apps that need that kind of model still call out to a server even when everything else runs locally.