Lakehouse Logic: Why Object Storage is the New Analytics Engine

Data analytics boils down to a simple objective. To provide people – and increasingly AI agents – with the right information, in the right place, at the right time.

That's it. It's easy to state. But historically difficult to achieve.

The good news is the landscape is changing. With the rise of data lakehouse architectures, open table formats, and the growing importance of metadata – especially in the age of GenAI – object storage is evolving into a high-performance platform for structured analytics.

Let’s explore how we got here, and why this shift matters.

From Data Warehouses to Data Lakes to Lakehouses

To understand how far analytics have come, consider that just a few decades ago the process of extracting and transforming data and storing it in a data warehouse was time-consuming at best. Business users provided the requirements and domain expertise, but the development of the entire solution, including the design of reports and dashboards, was up to IT.

Then came self-service BI, enabling users to explore data independently – provided it was available in the warehouse. But slow delivery cycles and rigid schemas often limited access.

Combined with advances in massively parallel processing and storage (MapReduce, Hadoop), the focus eventually shifted to data lakes – scalable, flexible environments for storing vast amounts of raw data. Data had become the new oil and those that had it were supposed to have a competitive advantage. But data quality and governance often lagged behind.

But as with oil, which needs to be refined, people realized that to provide value, data needed to be transformed, cleansed and cataloged. Data lakes delivered on that need by  combining the flexibility of data lakes with the transactional integrity and performance of data warehouses.

Life seemed good down by the lake. For a while.

Enter Object Storage

While data lakes were great for storing both structured and unstructured data, they weren’t always ideal. No wonder they were often also called "data swamps," since the focus was on volume and variety, but not always on data quality.

Object stores simplified data management. Their ability to store immutable objects made them ideal for both structured and unstructured data. But they weren’t perfect.

Challenges included:

• ROT (redundant, obsolete, trivial) data accumulation.
• Dark data – stored but unused, taking up valuable space, but offering no business value.
• Limited visibility into what data existed and how it could be used.

To unlock the full potential of object storage, we needed better metadata, governance and query capabilities.

And now it’s arrived.

Introducing Native S3 Table Support in VSP One Object

Hitachi Vantara’s VSP One Object takes object storage to the next level by natively supporting S3 Tables. This is an industry first for on-premises object stores. And the implications are significant.

Native support for S3 Tables means:

• You can create and manage S3 table buckets, namespaces and tables using familiar S3 Tables APIs.
• SQL-based analytics can be run directly on open-format data – no complex ETL or data movement required.

Combined with built-in features like an Apache Iceberg REST catalog, a zero-config SQL engine and an advanced metadata service, VSP One Object becomes an even more powerful foundation for data lakehouse and AI workloads.

The transformative capabilities of VSP One Object's native Amazon S3 Table support enable simplified data lakehouse architecture, faster analytics and seamless integration with modern tools like Apache Iceberg. It showcases how organizations can bridge the gap between data lakes and data warehouses to accelerate insights and support AI/ML workloads.
 

Unlocking S3 Metadata for Smarter Analytics

Beyond table support, VSP One Object introduces S3 metadata logging. When enabled, every object event – creation, update, deletion – is recorded in an immutable log stored as an S3 Table.

This provides critical functionality including:

• A complete, queryable history of object activity.
• Rich metadata about each object.
• Seamless integration with Iceberg clients and SQL engines.

Now you can manage and analyze both your data and its metadata using the same tools – boosting governance, observability and performance.

The Right Object Storage Solution at the Right Time

Organizations are under pressure to do more with their data than ever. And to do it faster, smarter and at greater scale. Whether you're building AI models, enabling real-time analytics, or simply trying to manage explosive growth, the infrastructure you choose matters more than ever.

VSP One Object is purpose-built for this moment. It transforms traditional object storage into a high-performance, intelligent platform that supports modern analytics and AI workloads with ease.

It’s the right solution at the right time, providing:

1. Native S3 Table support for structured analytics.
2. Built-in intelligence and metadata services.
3. Optimized for AI and lakehouse workloads.
4. Simplicity, scalability and speed.

VSP One Object is more than just storage. It’s a strategic enabler for organizations looking to unlock the full potential of their data – structured or unstructured, historical or real-time, human or machine-generated.

Find out more about why GigaOm Radar for Object Storage recognized Hitachi Vantara for innovation in object storage or connect with your Hitachi Vantara representative to get your object storage modernization journey started.

Read More:

• News Release: Hitachi Vantara Recognized by GigaOm for Innovation in Object Storage, Adds Industry-First S3 Table Functionality to VSP One Object
• Solution Profile: Modernize Data Lakehouse Environments With VSP One Object
• Top 10 Reasons Why: Hitachi Virtual Storage Platform One Object
• Webpage: Accelerate Innovation with Scalable Object Storage