A Memory-centric Approach to System Strategy: 6 Takeaways from Supercomputing 2025

Artificial intelligence workloads are reshaping how memory is produced, priced, and prioritized. Not because the supply chain has fundamentally broken, but because manufacturers are making deliberate decisions about where to place capacity and capital. Wafer lines are being steered toward high-margin, long-term AI demand, not toward broad, undifferentiated expansion.

HBM, advanced DRAM, and other AI-optimized memory now command the majority of investment and forward planning. Legacy NAND and commodity DRAM fabs, by contrast, are being expanded cautiously, if at all. What the market is seeing in pricing volatility and allocation pressure is not a classic production shortage, but a structural shift in how memory economics now work.

In that environment, AI systems are increasingly defined by memory bandwidth and dataflow efficiency. As more work is pushed through GPUs and accelerators, the pressure does not stop at HBM capacity. It extends across the entire data path, including how data is staged, how metadata is managed, how it’s retrieved, and how it is governed at scale. This, in turn, increases demand for object storage, metadata rich file systems, versioning, and governance.

By keeping GPUs fully utilized and the data path efficient, modern AI systems can extract significantly more productive work from the same underlying infrastructure, but that efficiency comes with trade-offs. Considering the level of investment made in GPUs, return on insight becomes the only measure that really matters. More productive GPUs generate more output, more intermediate artifacts, more frequent checkpoints, and more derived data, all of which must be stored, tracked, versioned, and governed over time. In that sense, a memory-centric design does not diminish the role of storage at all. It reinforces it by increasing both the volume and the importance of the data being produced. ‑to‑end toward accelerators. Memory, interconnects, and dataflow have become the defining factors in system performance — a major change from the priorities of past industry events.

The industry is rethinking how HPC and AI infrastructure differ in practice and how each should be built, focusing less on peak throughput and more on how data moves end-to-end toward accelerators: Supercomputing 2025 clearly highlighted this shift. Memory, interconnects, and dataflow have become the defining factors in system performance, representing a meaningful change from the priorities that dominated past industry events.

Here are the major breakthroughs from Supercomputing 2025:

1. The Convergence of HPC and AI Is Becoming an Operational Reality

High performance computing centers are no longer treating scientific simulation and AI training or inference as separate worlds. What stood out at Supercomputing 2025 was how many organizations are now planning for environments where both must operate side by side on shared infrastructure. That shift brings new requirements with it, including predictable memory bandwidth, stronger interconnect architectures, container readiness, and unified access to data across workloads.

As a result, peak FLOPS and classical filesystem benchmarks are losing their role as primary indicators of system value. They’re still important, but they no longer explain whether a system will perform well for AI. What matters more is whether the infrastructure can keep GPUs consistently productive without interruption or starvation. And that is one of the biggest differences between those two workloads.

Takeaway: Vendors are increasingly able to participate in AI infrastructure conversations without being measured solely against traditional HPC throughput or IO benchmarks, as long as they can demonstrate sustained GPU utilization and predictable system behavior.

2. Memory-centric Architecture Emerged as the Defining Industry Theme

Across the exhibit floor, technical sessions, and informal conversations, memory consistently came up as both the primary bottleneck and the primary accelerator for AI workloads. The focus was not just on capacity, but on how memory bandwidth, latency, placement, and memory management affect the entire system.

HBM4, next generation DRAM, and early demonstrations of CXL connected memory pools were positioned as foundational elements of future system design. Vendors spent far more time talking about tail latency, memory bandwidth, and end-to-end dataflow than about raw component speeds. That emphasis reinforces the need to treat memory, interconnects, and data movement as first class architectural considerations rather than secondary tuning exercises.

Takeaway: AI infrastructure needs to be evaluated based on the workload characteristics of AI training and inference, not on average case metrics that were designed for more predictable, batch-oriented environments.

3. Storage Vendors are Reframing Their Role as Infrastructure

Another noticeable shift at Supercomputing 2025 was the language storage vendors used to describe their role in AI environments. Rather than positioning themselves as performance engines for AI training, many described themselves as the plumbing that moves data into, through, and around the AI pipeline.

This reflects a growing acknowledgment that filesystems alone do not determine GPU performance during training. Their impact is indirect, shaped by how efficiently data is staged, discovered, and prepared before it ever reaches accelerators.

Takeaway:  Storage platforms that excel at metadata handling, retrieval workloads, and upstream data preparation remain highly relevant, even if they are no longer framed as the source of raw training performance.

4. Rack Scale, High Density Designs are Becoming the Default

System vendors have largely moved away from talking about individual servers and toward full rack level solutions. These designs are intended to support both HPC and AI workloads, with high GPU densities, large HBM footprints, advanced cooling approaches, and carefully optimized interconnect topologies.

Many announcements centered on eight GPU building blocks delivered as part of a larger modular rack architecture. That approach reflects a shift toward integrated system design, where power, cooling, networking, and management are planned together rather than optimized independently.

Takeaway: System-level solutions (like Hitachi iQ) are increasingly favored over component level assembly when addressing the complexity of modern AI infrastructure.

5. Data Loading, Retrieval Efficiency, and Metadata are Priority Constraints

As AI clusters scale, performance bottlenecks are moving away from raw compute and toward the supporting data pipeline. Discussions with both vendors and end users emphasized the importance of efficient data loading, avoiding GPU starvation, managing large metadata catalogs, and supporting frequent checkpointing without disrupting training progress.

When high frequency checkpointing becomes a design requirement, it naturally points back to memory centric approaches. Checkpoint performance is governed by how quickly data can be staged, buffered, and transferred without stalling the training loop. RAG workflows, agent-based systems, and multimodal applications add further pressure by increasing metadata intensity and sensitivity to tail latency.

Takeaway: For retrieval heavy and inference driven AI workloads, tail latency is becoming more important than peak throughput.

6. A Shift from Component Thinking to Pipeline Thinking

Perhaps the most important signal from Supercomputing 2025 was a broader change in how the industry is thinking about AI infrastructure as a whole. Rather than treating  compute, storage, networking, and data management as separate domains, organizations are planning for complete pipelines where data must move predictably from ingest through training and into inference. That shift favors integrated architectures that can support both training and retrieval paths, like Hitachi iQ paired with VSP One, rather than loosely assembled stacks of individual components.

The value of these systems lies in end-to-end orchestration, unified data access, and consistent behavior as environments scale. That reflects a growing understanding that optimizing the data path is just as important as optimizing the compute path, and that predictable data flow is a prerequisite for maintaining GPU utilization at scale.

Takeaway: Adopting a memory-centric design philosophy helps ensure that data can be staged, transformed, delivered, and retrieved in ways that avoid stall conditions and keep the pipeline moving.

A Memory-centric Approach with Hitachi iQ

The industry is starting to converge on the real reasons GPUs stall. It is not simply a lack of raw compute, but slow data loading, unpredictable latency, and practical limits on how memory is provisioned and accessed. That is why new approaches are emerging, including architectures like WEKA’s Augmented Memory Grid, and why a memory-centric strategy is becoming increasingly relevant.

Scaling memory capacity by continuing to add GPUs is not sustainable over the long term. HBM is expensive, capacity is constrained, and the economics do not favor tying memory growth directly to compute growth. This is where technologies like CXL, and other methods for expanding or pooling memory independently of compute, begin to matter. The prominence of CXL at Supercomputing 2025 reinforced that the industry is actively exploring these paths.

Hitachi Vantara’s architecture reflects this broader shift. Hitachi iQ is designed to support high-performance training paths and data acceleration, while VSP One and Hammerspace address the retrieval, preparation, and orchestration paths required for RAG pipelines, agent-based systems, and multimodal AI. Together, they form a unified system approach that aligns with how modern AI pipelines are actually built and operated.

Ready to learn more? Find out why Hitachi iQ was named a leader in storage optimized for AI workloads.