Phononic eyes new cooling solutions for AI chip demands
Phononic Chief Product Officer Larry Yang discussed the artificial intelligence chip-cooling requirements after Nvidia introduced its Rubin processors last month. Yang has worked in the technology industry for more than 30 years at companies such as Google, IBM, Microsoft, and Cisco. He explained that high-bandwidth memory chips generate significant heat alongside graphics processing units and require specialized cooling solutions.
Data centers traditionally use air cooling with heat sinks and fans to manage temperature. Liquid cooling systems have become more common as heat output from artificial intelligence processors has increased. Yang noted that mechanical cooling systems are slow to respond, leading data centers to overcool entire facilities rather than targeting specific hot spots.
Phononic develops thermoelectric coolers that use solid-state technology without moving parts. The devices respond within milliseconds and can be placed directly on high-bandwidth memory chips to eliminate performance throttling. Yang said the company’s cooling solutions allow operators to extract more performance from existing graphics processing units and avoid purchasing additional hardware.
The Nvidia Blackwell B200 NVL72 rack consumes 100-120 kilowatts, while the upcoming Rubin Ultra will require 600 kilowatts. Every watt of electricity consumed by a server rack converts to heat that must be removed. High-bandwidth memory chips sit in stacks next to processors, with electrical insulators between layers, making it difficult for heat to escape from the chips at the bottom of each stack.
Phononic ships its thermoelectric coolers to optical transceiver manufacturers and serves as the exclusive provider for Nvidia’s 1.6 terabit transceivers. The company has delivered more than 30 million thermoelectric chips over the past decade. Its thermal management system uses firmware to monitor chip temperatures and adjust cooling intensity in response to workload demands.
The company’s orchestration software connects to individual cooling controllers through a programming interface built on the Redfish data center standard. System operators can set cooling modes for different computing clusters based on whether servers are performing quick inference tasks or longer training operations. Yang said emerging technologies may eventually integrate thermoelectric materials directly into silicon chips to further reduce thermal interface layers.
