With AMD's 80-TOPS Ryzen AI and Hailo's 6.9 tok/s at 1.87W, the NPU-GPU battle is reshaping edge hardware design. We compare architectures at the PCB level.
The edge AI hardware landscape in 2026 is defined by a single metric that has become the industry's obsession: TOPS per watt. As AI inference moves from cloud data centers to embedded devices — factory floors, autonomous vehicles, medical instruments, and consumer electronics — the ability to deliver maximum AI compute within a strict power budget determines which products succeed and which fail. Two competing architectures are battling for dominance: dedicated Neural Processing Units (NPUs) and general-purpose Graphics Processing Units (GPUs). For hardware engineers designing the boards that host these silicon, the choice between NPU and GPU has profound implications for power delivery, thermal management, and system architecture.
The 2026 Landscape: Key Silicon Contenders
AMD's Ryzen AI Embedded P100 series, announced at Embedded World 2026, delivers up to 80 TOPS of AI compute by combining Zen 5 CPU cores with an XDNA 2 NPU and integrated RDNA 3.5 GPU. The total platform power ranges from 15W to 54W depending on configuration, yielding an NPU-specific efficiency of approximately 4.7 TOPS per watt when the NPU operates at its rated 37 TOPS within a 7.8W power allocation.
At the other end of the spectrum, the Hailo-10H dedicated NPU demonstrated 6.914 tokens per second for large language model inference at just 1.87W, with a throughput coefficient of variation of 0.04 percent — meaning virtually zero performance fluctuation. This level of consistency is critical for real-time applications where inference latency must remain predictable.
The Ceva NeuPro Nano, which won the Embedded Award 2026, targets ultra-constrained devices with an NPU IP block that delivers meaningful AI capability in microcontroller-class power envelopes below 500 milliwatts. Meanwhile, Axelera AI secured $250 million in funding for its Europa processor, which aims to bring generative AI capability to edge devices using a novel dataflow architecture.
On the GPU side, NVIDIA's Jetson Orin NX remains the benchmark for edge GPU compute, delivering 100 TOPS at 25W (4.0 TOPS per watt) with CUDA compatibility that simplifies software development. The upcoming Jetson Thor promises 800 TOPS, but at a significantly higher power envelope that pushes thermal design requirements beyond what passive cooling can handle.
Power Delivery: The Hidden Complexity
The choice between NPU and GPU fundamentally shapes the power delivery network (PDN) on the host PCB. GPUs demand high peak currents with rapid transients. The NVIDIA Orin NX, for example, requires a VDD_CORE rail at 0.75V with transient current slew rates exceeding 100A per microsecond. This demands a multi-phase voltage regulator module (VRM) with low-ESR ceramic capacitors (MLCC) placed within 2 mm of the BGA pads — typically 40 to 60 capacitors in the 100 nF to 10 uF range, plus bulk capacitors of 100 to 470 uF for energy storage.
NPUs, by contrast, typically operate at lower voltages (0.6V to 0.85V) with more predictable current profiles. The Hailo-10H's 1.87W power consumption translates to approximately 2.5A at its core voltage — manageable with a single-phase buck converter and significantly fewer decoupling capacitors. This simplification reduces PCB layer count requirements from 10 to 12 layers (GPU) to 6 to 8 layers (NPU), with direct cost implications of $2 to $5 per board in volume production.
The AMD Ryzen AI P100 presents an interesting middle ground. Its heterogeneous architecture — CPU, GPU, and NPU on a single die — requires a complex PDN with separate voltage rails for each compute domain. The CPU cores need 1.1V at up to 30A, the GPU needs 0.75V at up to 15A, and the NPU operates at 0.65V at approximately 12A. Managing the interaction between these domains during workload transitions — when AI inference shifts from NPU to GPU for unsupported operators — requires careful PDN impedance analysis across a frequency range of 1 kHz to 1 GHz.
Thermal Design: Watts Per Square Centimeter
The thermal challenge scales directly with power density. The Jetson Orin NX dissipates 25W across a module footprint of approximately 70 mm x 45 mm, yielding a power density of 0.79 W per square centimeter. Passive cooling with a finned heatsink is feasible in environments with adequate airflow (1 to 2 meters per second), but fanless designs require heat spreaders with thermal conductivities above 200 W per meter-Kelvin and careful attention to thermal interface material (TIM) selection — typically achieving 3 to 8 W per meter-Kelvin for silicone-based compounds or 50 to 80 W per meter-Kelvin for indium-based TIMs.
The Hailo-10H, at 1.87W, can be cooled with a simple copper pad or small heatsink, enabling truly fanless, sealed enclosures rated to IP67 for industrial deployment. The thermal resistance from junction to ambient (theta-JA) needs to be below approximately 30 degrees Celsius per watt to maintain a junction temperature under 85 degrees Celsius in a 40-degree ambient — achievable with a 20 mm x 20 mm copper spreader and natural convection.
For the AMD P100 at 54W TDP, active cooling is mandatory. The thermal solution typically involves a copper heat pipe assembly with an aluminum fin stack and a 40 mm to 60 mm fan, maintaining acoustic noise below 35 dBA at 1 meter for commercial applications. The PCB must include thermal vias under the processor — typically a 6x6 or 8x8 array of 0.3 mm diameter vias on 1.0 mm pitch — to conduct heat from the top-side BGA to a bottom-side ground plane that serves as a heat spreader.
INT8 Quantization: The Software-Hardware Interface
The efficiency gains of NPUs are amplified by INT8 quantization, which reduces model weights and activations from 32-bit floating point to 8-bit integers. This 4x reduction in data width translates to proportional improvements in memory bandwidth utilization and compute throughput. The Hailo-10H's architecture is optimized specifically for INT8 operations, which is why it achieves such remarkable TOPS-per-watt figures.
GPUs, designed for floating-point compute, are less efficient at INT8 operations but offer greater flexibility. The NVIDIA Orin NX supports INT8, FP16, and FP32 operations, allowing developers to mix precision levels within a single inference pipeline. This flexibility comes at a power cost — the GPU's wider datapaths and larger register files consume silicon area and power even when executing narrow INT8 operations.
For PCB designers, the quantization choice affects memory subsystem design. INT8 models require approximately 4x less memory bandwidth than FP32 equivalents, which means NPU-based designs can use lower-cost LPDDR4X (bandwidth up to 34.1 GB/s) instead of the LPDDR5 (bandwidth up to 51.2 GB/s) typically required for GPU-based designs. The memory interface routing — 32-bit or 64-bit wide, with matched trace lengths within 0.5 mm — is simpler for narrower buses.
The Emerging Middle Ground: Heterogeneous SoCs
The industry is converging on heterogeneous SoCs that combine CPU, GPU, and NPU on a single die, with the PCB designer responsible for optimizing the board-level implementation. The AMD Ryzen AI P100, Qualcomm Snapdragon X Elite (45 TOPS NPU), and Intel Lunar Lake (up to 48 TOPS NPU) all follow this pattern.
For hardware engineers, heterogeneous SoCs simplify board design by reducing chip count but increase the complexity of each individual design. A single SoC with three compute domains requires three separate power domains, three thermal zones (since each domain has different activity patterns), and careful clock distribution to avoid electromagnetic interference between high-speed digital clocks (up to 5 GHz for CPU cores) and sensitive analog circuits (ADCs, PLLs).
Guoman & Partners' Perspective
Guoman & Partners has designed edge AI platforms across both NPU and GPU architectures for clients in industrial automation, autonomous vehicles, and medical imaging. Our recommendation depends on the application: dedicated NPUs for high-volume, cost-sensitive deployments where the AI workload is well-defined; GPUs for research platforms and applications requiring model flexibility; and heterogeneous SoCs for products that need to balance AI performance with general-purpose computing.
Conclusion
The NPU-GPU efficiency war is not a winner-take-all contest. Both architectures will coexist, serving different segments of the edge AI market. What matters for hardware engineers is understanding the board-level implications of each choice — from power delivery complexity and thermal management to memory subsystem design and manufacturing cost. As edge AI deployments scale from millions to billions of devices, the engineers who can optimize these trade-offs at the PCB level will determine which products succeed in the market.
