As high-performance computing (HPC) and artificial intelligence converge, the definition of a "bottleneck" has fundamentally shifted. No longer confined to simple memory bandwidth or raw FLOP counts, today's performance walls emerge at the complex intersection of massive multi-node communication fabrics, hardware-specific loop optimisations, and the sheer power demands of real-time model serving.
These challenges took centre stage at the Benchmarking in the Data Center (BID 2026) workshop, held on September 28, 2026, at NTU@One North in Singapore, in conjunction with ICPP. Bringing together researchers, national laboratory experts, and industry leaders, the half-day program offered a timely snapshot of how the community is tackling the physical and architectural limits of modern infrastructure.
SCW75 Alumni Samar Aseeri gives her thoughts on the recent workshop and outlines how researchers can approach optimisation and sustainability in HPC and AI.
The scale of modern workloads
A recurring theme across the sessions was the sheer magnitude of data required by state-of-the-art computational models. For instance, creating high-resolution multi-year simulations—such as continental-scale weather models at 1-kilometre resolution—demands immense orchestration, spanning hundreds of specialised accelerators like NVIDIA H100 GPUs. Yet, achieving this scale exposes a paradox: as clusters grow larger, network interconnect overhead can quickly erode the theoretical gains of horizontal scaling.
This scaling paradox was highlighted sharply in discussions of distributed inference workloads. Recent profiling campaigns exploring massive language model deployments (such as SGLang with DeepSeek-R1 across multi-node H200 clusters) suggest that "less is more" can often trump brute-force distribution. Keeping data tightly coupled within single-node memory fabrics via high-speed interconnects like NVLink frequently outperforms complex multi-node scaling over InfiniBand fabrics, forcing a re-evaluation of cluster design strategies for inference.
Optimisation from the ground up
While system-level architecture dictates macro-efficiency, microscopic code-level tuning and strict algorithmic validation remain vital. Achieving performance gains must never compromise mathematical fidelity; a guiding principle for holistic optimisation is ensuring we deliver the same math faster by rigorously aligning software pipelines with hardware capabilities.
Building on this philosophy, modern GPU optimisation increasingly moves beyond standard grid-stride loops to rely on bespoke paradigms—such as warp-stride and block-stride loop patterns—to maximise L1/L2 cache utilisation and thread reusability. These granular techniques prove that substantial performance and energy efficiency are often left on the table if software design fails to align with hardware topology.
The sustainability horizon
Underpinning all technical optimisations is an inescapable physical constraint: power and cooling. As data centres grapple with the electrical demands of dense GPU deployments and continuous AI training loops, energy efficiency is no longer just a secondary metric—it is the primary hard limit governing future infrastructure investments.
Moving forward
The discussions at BID 2026 reinforced a central truth for our field: benchmarking and performance analysis cannot exist in a vacuum. Whether evaluating weather simulations, dense tensor cores, or hybrid execution pipelines, bridging the gap between hardware capability and software execution requires continuous, open community exchange.
As we look toward the future of exascale computing and dense AI integration, the path forward relies not just on building larger systems, but on understanding them more deeply.
Samar Aseeri is a Senior Computational Scientist at King Abdullah University of Science and Technology.
