AI workloads are pushing processor compute densities into previously uncharted territory. Today's ultra-dense AI server racks produce 120kW of heat, with GPUs exceeding 1200-watt TDPs. Compare that with historically dense, general-purpose compute systems at 12kW per rack and CPUs dissipating 125 watts of heat. The engineering challenge of keeping hardware cool has become a critical design constraint. Effective cooling is essential to system performance.
Unchecked heat causes performance degradation, system instability and permanent hardware failure. With AI server rack costs in excess of $4M, this also places a premium on reliability. That's why coldplate design has become one of the most consequential disciplines in modern computer system engineering.
Why liquid cooling became non-negotiable
Air cooling successfully managed compute thermal loads for years. This changed once processor TDPs climbed above 800W. Liquid cooling manages heat densities that overwhelm air-based systems, with a heat capacity 3,500 times higher and a heat transfer coefficient hundreds of times higher than those of air cooling.
"At the beginning of liquid cooling, any coldplate was enough because you would take advantage of the heat capacity of water," says Milad Samie, Senior Thermal Engineer at CoolIT, an Ecolab company. "But now, with TDPs going up, we have to optimise coldplates as well."
Coldplate optimisation has two dimensions: thermal performance and pressure management.
Coldplates must deliver maximum cooling capacity while maintaining a low enough pressure drop that a single coolant distribution unit (CDU) can serve as many server compute nodes as possible. As deployments scale to tens or hundreds of thousands of coldplates, those engineering margins have a direct impact on performance, efficiency and total system design.
Where thermal bottlenecks emerge
At the processor level, the most significant bottleneck today is the thermal interface material (TIM). A TIM is a paste-like material that sits between chip and coldplate. While TIMs are necessary to avoid problematic metal-to-metal contact, they introduce a thermal resistance that can account for up to a 10°C temperature rise across the interface alone. CoolIT works with TIM vendors to identify and validate the best-performing materials for high-TDP applications.
Achieving 100% heat capture is increasingly important. With server racks now expected to dissipate more than 500kW, hybrid liquid-air cooling becomes unsupportable. Assuming peripherals and power supplies account for 25% of the heat generated, this amounts to 125kW. These components alone represent the totality of the heat generated from today's liquid-cooled AI systems. Eliminating air cooling entirely with fully fanless designs is becoming a critical design goal as thermal density increases.
Evolving coldplate architectures
CoolIT has developed a series of coldplate architectures to address progressively more demanding thermal environments.
CoolIT's unique Split-Flow™ design addresses a fundamental limitation of conventional side-in, side-out coldplates by directing coolant to the centre of the coldplate. By splitting coolant flow, pressure drop is reduced by upward of 30%, improving system performance. Additionally, coolant can be targeted directly to the hottest region of most symmetric processor dies. Targeted cooling ensures uniform overall and optimal chip operation. CoolIT's Split-Flow coldplates are well suited to high-TDP, high-heat-flux applications.
The OMNI™ all-metal coldplate answers a different problem: leakage risk. Gasket-based coldplates carry an inherent risk that threatens the million-dollar servers they cool. Brazed coldplates introduce possible contamination and deformation from the annealing process. OMNI coldplates are made by aerospace-grade friction stir welding to form a fully metallic, unibody coldplate. OMNI all-metal coldplates dramatically reduce failure points.
AI processors will place even greater demands on coldplate design. As package power, heat flux and rack density continue to rise, customers will need coldplate architectures that capture more heat, manage higher flow requirements and support reliable performance across increasingly complex server designs.
CoolIT is already engineering for that future. With proven 100% heat capture expertise, advanced coldplate R&D and a track record of developing architectures for the most demanding environments, CoolIT delivers the liquid cooling technologies AI system engineers require to support higher-density compute performance while ensuring reliability.
Demonstrating the single-phase ceiling is higher than assumed
A persistent assumption in the industry has held that single-phase liquid cooling tops out at around 2kW per device. CoolIT has disproved that.
"Until a few years ago, many in the industry believed single-phase liquid cooling would hit a practical limit around 2kW per package, with higher-power chips requiring two-phase cooling beyond that," says Samie. "Recent high-performance single-phase coldplates have shown that assumption was not based on a fundamental physical limit. A year ago, CoolIT designed and tested a 4kW coldplate. Then a few months ago, recognizing the 10-year performance roadmap of the leading GPU vendors, we said, let's push coldplate design even further."
The result was a validated 15kW coldplate design that extends the single-phase direct liquid cooling roadmap beyond 2030. This was a huge milestone for liquid cooling and AI. For data centre operators, OEMs and ODMs planning infrastructure investments, the implication was significant: proven single-phase DLC has a clear path to support many generations of AI infrastructure well into the future.
How single-phase compares with alternative approaches
Immersion cooling has attracted considerable attention as a route to 100% heat capture. By submerging entire servers in dielectric fluid, there is nowhere for heat to go except through the liquid. However, the thermal properties of immersion coolants limit their ability to handle the high heat fluxes of today’s processors. Immersion has valid use cases, but high-performance AI compute is not among them.
Two-phase direct liquid cooling, sometimes called pumped two-phase, is more thermodynamically interesting. Latent heat offers substantially higher cooling capacity than sensible heat alone. The physics are favourable. The engineering challenges, however, are significant. Flow regulation across multiple nodes is complex. Achieving the 50–60°C saturation temperature required for effective chip cooling demands operating pressures above 100psi for most high-pressure synthetic refrigerants. The cooling loop for the entire facility must be designed and rated accordingly, which requires more complex system engineering
Regulatory constraints add another layer of difficulty. Many refrigerants used in two-phase systems face restrictions that vary significantly between jurisdictions. European regulations are more stringent than those in North America. Variations at state and provincial levels add further complexity. Standardised frameworks for large-scale two-phase deployments do not yet exist.
"Two-phase cooling has some specific use cases, but there is still a long way to go before pumped two-phase is ready for large-scale deployment," says Samie. "The engineering, infrastructure and regulatory hurdles are substantial. Single-phase direct liquid cooling is already proven at hyperscale. It will remain the de facto technology for AI infrastructure well into the future while the industry continues to mature two-phase solutions."
The liquid cooling R&D priorities shaping what comes next
For coldplate technology, Samie identifies several areas of active development:
- TIM optimisation remains a priority for as long as coldplates are thermally separated from chips.
- Manufacturing process innovation is equally important. Computational fluid dynamics can produce highly optimised fin geometries that are difficult or impossible to fabricate at scale using conventional methods, making additive manufacturing and advanced bonding techniques such as diffusion bonding increasingly relevant.
- Material science is also under active investigation. Copper remains the standard material for coldplates, but higher thermal conductivity alternatives could meaningfully improve coldplate performance, and CoolIT is exploring options that improve both coldplate and cooling loop performance and reliability.
The direction is clear. AI workloads will continue to demand more from the thermal infrastructure beneath them. The engineering challenge is ensuring the cooling systems keep pace, and recent results show single-phase liquid cooling has considerably more headroom. The heat capture with single-phase shows it’s enough to lead the industry for the next decade.
