CSCS Needed a Centralized Data Warehouse

CSCS was managing a large and complex environment of various leading-edge systems, with up to 27 petaflops of computing power, on their most powerful supercomputer (Pitz Daint). This translates to 43,451,090 node hour per year consumed and around 10PBs of data available for more than 1200 end users.

The company found that gathering data for reports was a cumbersome manual task that took about two to three weeks. The process included many interactions with various staff members because CSCS had no centralized data warehouse or reporting tool.

It was difficult to manage the growing data volumes – to the tune of 70 million rows of data. Since their customers' data volumes continue to grow, CSCS needed to seamlessly scale up or down their operations.

Because their investment in equipment is extremely high, they needed to optimize CPU utilization for customer projects that include CERN, MeteoSwiss, universities, and other EU projects (Tier 0 Prace). The universities include ETH Zurich, the University of Zurich, the University of Geneva. CSCS needed a single view of their data to improve business operations, enhance machine optimization and to become more efficient. As CSCS is a government agency, they are required to report metrics around their operations daily.

The Solution: Pentaho Data Integration, Pentaho Business Analytics

Hitachi Vantara's Pentaho platform provides a 360-degree view of CSCS's environment by blending CPU data from their supercomputers with financial data from SAP, allowing them to manage projects more efficiently. Recently, also data from cloud services on premises are integrated in the Pentaho Data Warehouse.

Additionally, the company delivers a series of daily on-demand reports about the utilization and performance of their equipment to their management team, government agencies and customers.

CSCS took advantage of Pentaho services from Hitachi Vantara for deployment assurance that includes installing and configuring the server, architecture review and best practices.

The Outcome: Real-time Data Analysis for Increase in Profit

By increasing the utilization of their equipment, CSCS has been able to increase productivity and profit margins. Hitachi Vantara's platform allows CSCS to deliver their metrics to the public through several dynamic reports, which make the data more accessible to employees.

Hitachi Vantara's Pentaho Data Integration eliminated the need for manual CPU data gathering, which used to take from two to three weeks. The simplicity and speed of gathering machine data in near real time enables the company to provide a deeper level of reporting that was not previously available.

This customer chose the Pentaho platform for a variety of reasons. They valued Pentaho's end-to-end data integration and business analytics platform that provides them with a single view of their environment. This solution enabled them to deliver their metrics to the public through a number of dynamic reports. Lastly, Pentaho is a flexible, lightweight solution with robust automated feature sets.