Quick answer: an NPU (neural processing unit) is a specialized processor built to run the math behind neural networks directly on a device, instead of sending that work to a remote server. It handles tasks like voice transcription, photo search and on-device chat models using far less power than a CPU or GPU would need for the same job. Most phones sold today carry one, and so does most new laptop hardware.
So why does a device need a third chip alongside the CPU and GPU? Because AI features that run constantly, like face unlock or live transcription, would drain a battery fast on hardware built for something else. This entry covers what an NPU is, how it works, where you already have one, how it differs from a GPU, and what it cannot do.
What Is an NPU?
An NPU is a chip, or a block inside a larger chip, designed for one thing: the repeated matrix and vector math that neural networks run millions of times per second. It sits next to the CPU and GPU on the same system on a chip rather than replacing either of them.
You will also see NPUs called AI accelerators or AI chips, and the terms are interchangeable in practice. The CPU still handles general logic, the GPU still handles graphics and heavy parallel jobs, and the NPU takes over specifically when the task is a trained model doing inference.
That division of labor is what makes an NPU useful. A phone or laptop can run a small AI model continuously, for something like a camera adjusting exposure in real time, without draining the battery the way a GPU would if it stayed active for the same stretch. The NPU is the hardware that makes edge AI practical rather than merely possible.
Keeping that work on the device carries a privacy benefit too. A voice command or a photo processed by an NPU generally never has to leave the phone, since the model runs locally instead of uploading the raw audio or image to a server for the cloud to handle.
How Does an NPU Work?
Neural networks reduce to one repeated operation: multiply two numbers, add the result to a running total, over and over across huge grids of values called matrices. An NPU is built almost entirely out of small arithmetic units wired to do exactly that operation in parallel, thousands of times per clock cycle.
The other trick is precision. Training a model typically needs high-precision math, but a trained model answering a query usually does not. NPUs lean on lower-precision formats such as INT8 or FP16 instead of the 32-bit math a CPU defaults to, which cuts both the power draw and the chip area each calculation needs.
Manufacturers measure that combined throughput in TOPS, short for trillion operations per second. It is a useful rough gauge of an NPU’s ceiling, though real workloads rarely reach the peak number, since actual performance depends heavily on the model and the software feeding it.
Memory placement matters as much as raw math. Shuttling data back and forth between a chip and separate memory costs more time and power than the calculation itself, so NPU designs keep small pools of fast memory close to the arithmetic units and try to minimize how often data has to travel.
Where Are NPUs Used?
Apple’s Neural Engine, built into every iPhone and Mac chip since the A11, handles Face ID matching, photo search and on-device Siri processing without a network call. Qualcomm’s Hexagon NPU anchors the Snapdragon X laptop chips, and the design has scaled quickly: the first Snapdragon X Elite launched in 2024 at 45 TOPS, and Qualcomm’s 2026 generation reaches roughly 80 TOPS.
Microsoft set a line in the sand for Windows laptops with its Copilot+ PC category, which requires an NPU rated at 40 TOPS or higher so the machine can run features like live captioning and background blur locally. Intel’s AI Boost NPU and AMD’s XDNA block, found in current Core Ultra and Ryzen AI chips, both target that same bar.
The pattern across all of them is the same: local, always-on AI tasks that would be wasteful or too slow to route through a data center every time. Google’s Tensor chip follows the same logic in Pixel phones, handling on-device features like call screening and live transcription.
NPUs have also spread well beyond phones and laptops. Cars, security cameras and smart-home devices increasingly carry small NPUs of their own, running object detection or wake-word listening without a network connection to depend on.
NPU vs GPU: What’s the Difference?
The confusion is understandable, since both chips run heavily parallel math and both show up in the same AI marketing paragraph. The difference is what each one is built for.
A GPU is a general parallel processor, originally built for rendering graphics, that turned out to be excellent at the matrix math neural networks need. It is still the workhorse for training large models and for heavy large language model inference in a data center, where power draw matters less than raw throughput.
An NPU is narrower on purpose. It is tuned for lower-precision inference at low power, which makes it the right chip for a phone or laptop running a small model continuously, but not the chip you would use to train a frontier model from scratch. Training still happens on GPU or TPU clusters; the NPU picks up the job once a model is trained and needs to run locally.
By the time a model reaches your device, the heavy lifting on training data already happened somewhere else. The NPU never touches that original dataset; it just runs the finished model as efficiently as it can.
Google’s TPU adds a third label to the mix. It is conceptually similar to an NPU, purpose-built matrix hardware, but the term usually refers to Google’s own data-center accelerators rather than the on-device chips inside consumer hardware.
What Are the Limits of NPUs?
TOPS numbers are not directly comparable across manufacturers, since there is no single agreed benchmark and vendors count differently. A 45 TOPS chip from one company is not automatically slower in practice than a 50 TOPS chip from another.
Software support is the bigger gap. An NPU only helps if the app in front of you is written to target it, and a large share of AI features on phones and laptops still fall back to the CPU or GPU by default while developers catch up.
An NPU also is not a substitute for cloud infrastructure. It is built for inference on a model that already exists, not for training one, so it changes what you can run locally rather than what a company can build.
Device memory is a related ceiling. Phones and laptops carry far less memory than a data-center server, so the models an NPU runs are usually smaller, distilled versions built to fit that budget rather than the largest models a company offers.
For a reader shopping for an AI phone or an AI PC in 2026, the practical takeaway is simple: the NPU is why on-device features like transcription, photo cleanup and small local chat models work without a network connection or a constant hit to your battery. As more apps target these chips directly, that local capability is where most of the visible change is likely to show up next, ahead of the artificial intelligence you interact with over the network.