Qualcomm AI chips ditch HBM for efficient inferencing
Qualcomm introduced artificial intelligence processors that use mobile memory technology instead of high-bandwidth memory chips common in competing products from Nvidia and AMD. The AI200 and AI250 accelerators support up to 768 gigabytes of low-power double data rate memory, which provides greater capacity than standard solutions while reducing energy consumption and manufacturing costs. The San Diego manufacturer positioned the chips for inference workloads rather than training operations.
The memory choice offers advantages such as lower power per bit and reduced heat generation compared with high-bandwidth memory modules. However, the approach sacrifices memory bandwidth and increases latency because of narrower data connections. The rack systems consume 160 kilowatts of power and feature direct liquid cooling, with connectivity via PCI Express and Ethernet interfaces.
The processors feature Hexagon neural processing units that handle inference tasks and support advanced data formats. Intel and Nvidia have released similar products focused on inference applications as companies recognize the growing demand for specialized hardware. Market observers responded positively to the announcement amid expanding competition in the artificial intelligence chip sector.
