Overview
Challenge
- Fast, reliable data storage is vital to support the high-performance computing (HPC) systems that power major science and engineering projects.
Solution
- Quadrics recommends Hitachi Vantara storage systems to help its clients build robust data delivery platforms for HPC.
Outcome
- By eliminating storage-related downtime, clients can rely on their HPC jobs running to completion, avoiding weeks of delays and rework.
100%
Data availability guarantee underpins a highly reliable HPC platform
Weeks
Saved by delivering stable performance for long-running simulations
50-year
Data retention facilitates compliance with government mandates
Challenge
Over the past two decades, Quadrics has established itself as one of Europe’s leading specialists in high-performance computing (HPC) solutions for a wide range of organizations, from digital animation studios and large-scale AI factories to major European science and engineering projects. The company’s consultants help clients design, develop, implement and maintain HPC clusters, offering comprehensive expertise across the full technology stack.
“HPC projects pose a unique set of technical challenges when it comes to infrastructure, especially in terms of storage,” explains José Luis Tallón, CEO and CTO of Quadrics. “With most workloads, such as enterprise databases and applications, you have a fairly constant low-intensity access pattern. But when you are running complex large-scale simulations, like those used to model computational fluid dynamics, the workloads have extreme spikes which put the storage arrays under stress. Most vendors’ storage systems just aren’t designed to cope with that kind of pressure.”
This is a serious problem, because these simulations are effectively giant batch jobs that can take hours or even days to complete. If a storage failure causes a job to crash, it typically needs to be restarted from the beginning. If these failures happen too frequently, it can stall progress on a project for weeks.
The large-scale international research programs that Quadrics supports often have additional requirements that make designing storage architectures even more challenging. In some cases, these requirements are government mandates, and noncompliance could result in financial penalties or even loss of contracts.
“In one of our projects, our client is streaming data from thousands of sensors for real-time analysis, and if that data stream is interrupted for even a short period, they will miss their SLA for reporting,” says Tallón. “We have a project where data needs to be replicated across more than 500 kilometers to comply with disaster recovery policies, and another one where all the data needs to be retained for more than 50 years. Also, a common theme is that when our clients’ storage systems need to be replaced with new hardware, the migrations must be completed without any downtime.”
“Hitachi Vantara has helped us prove that Quadrics is the right choice for complex HPC infrastructure projects. We are one of the few specialists in Europe with the HPC expertise to provide an end-to-end managed solution.”
José Luis Tallón, CEO and CTO, Quadrics
Solution
For all these projects, Quadrics recommends storage systems from Hitachi Vantara. “I first encountered Hitachi Vantara storage systems 20 years ago, in the very first HPC project that I worked on,” recalls Tallón. “Even back then, their reputation for reliability was legendary. Today, Hitachi Vantara’s guarantee of 100% data availability is absolutely what is needed for HPC use cases; and in my experience, their systems always live up to that promise. We even did a test where we went into the data center and started pulling out cables, and afterwards the system came back online again perfectly. So, when clients ask us for advice on robust storage, the choice is obvious.”
For a recent project supporting a major European nuclear energy research organization, Quadrics built a solution using Hitachi VSP One Block for Mid-sized Enterprises. “They had been suffering frequent outages and performance problems with their previous storage, but they were also anxious about moving to a different vendor,” explains Tallón. “We worked with Hitachi Vantara to provide a ‘seed unit’ that they could test in their own environment, with their own data. As soon as we installed the VSP One Block system, all the reliability issues vanished, and they were happy to commit to the new platform.”
Quadrics consultants then helped the client migrate their data seamlessly to the new VSP One Block platform. The seed unit was used to virtualize the storage volumes on the old array, and then Quadrics used Hitachi Universal Volume Manager to copy the data onto the new production array.
“We had about a five-minute slot that we could use as a migration window,” explains Tallón. “Thanks to Hitachi Vantara’s non-disruptive migration capabilities, we were able to complete the migration while keeping downtime within that window.”
Today, the client’s simulations can run continuously for up to three weeks on a cluster of compute nodes. Throughout that run time, the simulation manipulates a “working set” of approximately 2 terabytes of data, with interleaved sequential writes and random reads—an extremely demanding workload pattern at this scale. “The client’s old Dell storage arrays used to time out approximately once per week,” says José Luis Tallón. “But the VSP One Block array has never timed out, not even once, since the beginning of the trial a year ago.”
In another project for a client that works with government organizations, Quadrics develops and operates a near-real-time monitoring platform for civic infrastructure such as bridges. The core of the solution is a custom integrated online analytical processing (OLAP) solution and data warehouse that Quadrics has built on PostgreSQL. With the VSP One Block solution, which is pre-validated for running PostgreSQL workloads, Quadrics’ client can retrieve data as soon as it is ingested, perform the analysis in bounded time to comply with government-mandated parameters, and submit the results to the relevant government department.
Outcome
Quadrics sees its partnership with Hitachi Vantara as a key advantage in delivering HPC solutions that meet its clients’ complex and demanding requirements.
“It’s not just the core performance and reliability of the VSP One Block arrays,” says Tallón. “It’s the breadth of Hitachi Vantara’s storage hardware and software portfolio. If our clients need to replicate data across long distances, Hitachi Universal Replicator is the best solution available. If they are concerned about ransomware and cyberattacks, VSP One Block has built-in immutable snapshots. Whatever the technical challenge, we can always find a solution.”
He concludes: “The recent projects we have worked on with Hitachi Vantara have helped us prove that Quadrics is the right choice for complex HPC infrastructure projects. We are one of the few specialists in Europe with the HPC expertise to provide an end-to-end managed solution, from architecture design through to server and storage implementation, data migration, security, monitoring and ongoing support.”
Industry
- Technology
Location
- Spain, EMEA
Solutions
Hardware