An NPU is a low-power chip purpose-built to run AI models on your device. After years of AI living mostly in the cloud, the neural processing unit (NPU) is now inside laptops, phones and desktop chips, which is why vendors are suddenly selling “AI PCs” and quoting TOPS numbers on the box. Here is what an NPU actually is, how it differs from the GPU and the CPU it sits next to, and what those headline numbers really tell you.
A neural processing unit is a class of specialized accelerator designed to run machine-learning workloads, especially the already-trained neural networks used for inference. It can be a standalone chip, a block inside a CPU, or a block inside a GPU. On consumer devices the intent is consistent: small, power-efficient, and reasonably fast at running small models while drawing very little power, typically using low-bitwidth math such as INT4, INT8, FP8 and FP16.
The NPU moment
The reason you are hearing about NPUs now is that AI moved onto the device. Background blur in video calls, live captions, photo enhancement and local large language models all run better on a dedicated engine than on the main processor. An NPU is that engine. Its job is matrix math, the multiplication-and-accumulate operations at the core of neural networks, executed with enormous parallelism and tight power efficiency.
NPU vs GPU: both parallel, different jobs
The fastest way to confuse an NPU with a GPU is to note that both crunch matrices in parallel, and the confusion is fair. Graphics processing units from Nvidia and AMD include dedicated units for low-precision matrix multiplication, and they are widely used as AI accelerators for both training and inference. The difference is intent. A GPU is a general-purpose parallel accelerator that also draws and games, and it is built to be fast even if that costs power. An NPU is purpose-built for sustained, low-power inference with its own instruction set optimized for INT8 and FP16 matrix work. As Qualcomm describes its Snapdragon X Elite, the CPU, GPU and hexagon NPU are three separate engines in the same package, each handling what it does best.
NPU vs GPU vs CPU: three engines in one chip
Look inside a modern system-on-chip and you will find the division the vendors keep repeating. The CPU is the general-purpose engine that runs the operating system and ordinary serial logic. The GPU is the parallel engine for graphics and general-purpose compute. The NPU is the AI engine, added so that neural workloads do not steal cycles from the other two. This is the layout on Intel Core Ultra processors and Qualcomm’s Snapdragon X Elite alike, and it is why an “AI PC” spec sheet now lists three separate compute engines.
The current NPU landscape, in numbers
The numbers below are the vendors’ own advertised peak figures, and they are worth knowing because they now appear in marketing everywhere.
Qualcomm rates the Snapdragon X Elite NPU at up to 45 TOPS. AMD’s Ryzen AI 300 “Strix Point” parts with the XDNA 2 NPU are rated at up to 50 TOPS, up from the roughly 10 TOPS of the first-generation 7040 parts. Intel’s first Core Ultra series carried an 11 TOPS NPU within a 34 TOPS platform total, while the second generation, Lunar Lake, jumped the NPU alone to 48 INT8 TOPS. Apple’s M4 Neural Engine is rated at 38 TOPS, part of a lineage that began with the A11 Bionic’s Neural Engine in 2017.
What TOPS means, and why to be skeptical
TOPS stands for trillions of operations per second, and it is almost always quoted as INT8 additions and multiplications. The problem is that a raw throughput ceiling says nothing about memory bandwidth, power draw, software efficiency, or whether the model you actually want to run can use the unit at all. Advertised TOPS are theoretical peak numbers, not sustained real-world performance, and real results vary with the model, the precision and the software stack.
There is also a marketing twist worth knowing. Microsoft set a de facto 40 TOPS NPU bar for Copilot+ PCs, which is precisely why so many PC vendors advertise peak figures that clear 40. The number on the box reflects a certification target as much as a real workload, so treat a 45 versus 50 TOPS difference as a marketing signal, not a performance verdict.
Why vendors are adding an NPU now
The timing is not an accident. For years, on-device AI was a niche for phones, with Apple leading on the neural engine and everyone else following. What changed is that the capabilities became useful on laptops and desktops too, and the software caught up. Microsoft’s Copilot+ pushed PC makers toward a common AI baseline, and the chip vendors responded by putting a dedicated NPU in mainstream silicon rather than leaving AI work to the CPU or GPU. The result is that an NPU is no longer a premium feature; it is increasingly the default, even on midrange chips where vendors sometimes still leave it off. That matters because it changes the value calculation for anyone weighing an upgrade this year.
Do you need an NPU?
Right now the NPU matters most for people who want AI that runs locally: private LLM inference, on-device image and video features, and the battery savings that come from offloading neural work to a low-power engine instead of burning the CPU or GPU. If your workload lives in the cloud, or you have no local AI use case, the NPU is largely a future-proofing checkbox rather than a day-one reason to upgrade. The honest takeaway is that the NPU is real and increasingly useful, but the number on the box is not the whole story.
For more context, AMD has been shipping some Ryzen desktop chips without an NPU at all, and supports local models on its Ryzen AI Max hardware. That is the same on-device-AI debate we explored in our look at what local AI models mean for owning your hardware.
