Predictions 2024: A Data-Driven Journey Beyond the Hype

Simon Ninan
Senior Vice President of Business Strategy

February 09, 2024

“Data is the new gold”: this saying was coined in the mid-2000s and caught on like wildfire as businesses were scrambling to find ways in which data could unearth new value for them. In the twenty years since, the data gold rush has seen several spikes as new innovations generate hype and inflated expectations, that subsequently get tempered as reality sets in. The latest provocateur is Generative AI, thrust into the public consciousness by the launch of ChatGPT and related advancements between 2022 and 2023. The waves of investment and breathless prognostication that were unleashed will no doubt abate to some degree, but there is little question that we are entering a new era of data possibilities.

In that context, 2024 will be an interesting year: we will see data reality checks materialize, but rather than stymie innovation, these checks will actually open up further opportunity enabled by a more robust data foundation. Just like gold, raw data sitting unearthed, unrefined, or unused will hold no value. The key to thriving in this new landscape lies in harnessing the power of data and AI through a robust data foundation in the right environment to drive competitive advantage. Businesses that embrace this will better set themselves up to survive and thrive; those that don’t won't just be missing out – they will be left behind.

1. AI Empowers, Gets Real: Infused with Trust and Explainability

We can still hear echoes of the initial thunder over ChatGPT. Bloomberg Intelligence estimates that Generative AI could be a $1.3 Trillion market by 2032, growing at a CAGR of 42%. In 2023 alone, nearly $50 Billion was raised by Generative AI and AI-related startups, per Crunchbase data. But the honeymoon phase may be ending. The flood of investment dollars that poured in at the outset has settled into a steady but rationalized flow. Now comes the real test: getting past business uncertainties, fears, and skepticism, and making these big, ambitious concepts work in real-world situations.

AI itself isn't new, but ChatGPT's explosion brought Generative AI into the spotlight by democratizing these capabilities, putting them into the hands of everyone, with a variety of use cases primarily geared towards increasing productivity. But can we trust it for critical decisions and processes in a business context? Not just yet. The problem: ChatGPT and similar public LLMs (Large Language Models) lack context and expertise. They can spin impressive yarns but lack the depth for real-world applications and the ability to be adequately meaningful in a specific domain or customer context. And, with widespread examples of data hallucinations as well as risks of data poisoning and fraud, they also lack the trustworthiness needed to drive real adoption by business users and leaders.

To truly harness GenAI, applications need to be “grounded” with specific context and data. This might involve pointing the AI to relevant and unique sources, validating, and providing visibility to data lineage, providing additional clarity around desired outcomes, establishing protections to safeguard data, or even building custom LLMs trained on specialized data. The goal: accurate, explainable results you can trust. The trustworthiness of outcomes driven by data depends on the ensuring the quality of the data itself (garbage-in, garbage-out), the quality of the processes that manipulate and interpret this data, and of course the quality of the infrastructure that stores, secures, and supervises this data.

Tech leaders in 2024 will embrace an iterative approach, continuously learning and adapting as the field evolves. They will adopt the dual strategy of proving out the value of AI via incremental use cases, while putting into place the foundational building blocks for future AI innovation. In addition, the emergence of new regulation and standards will advance the adoption and prioritization of ethical AI principles, transparency, and responsible AI development. This can help not only mitigate risks but also begin to unlock the full potential of AI for social good and business success. The honeymoon may be over, but the future of GenAI is just beginning.

2. Edge and IoT: Taking the Bite Out of the Data Explosion

The lines continue to blur between the physical and digital worlds. Forbes estimates that IoT adoption will continue to grow, with data being shared in real-time among more than 200 billion IoT connected devices by the end of 2024. A convergence that massively expands our understanding of our surroundings and paves the way for truly immersive experiences. That’s the good news.

The challenge: Businesses are already struggling to keep up with the exponential growth of this data. The noise created by this data avalanche makes it harder for true insights and value to come through. Much of that data is generated at the edge, where challenges with connectivity, security and scalability become obstacles to value.

One solution lies in edge AI: the coupling of edge computing with the recent innovations in AI effectiveness. The deployment of AI algorithms and applications in edge devices allows for filtering, processing, and refining of data closer to the source, rather than in a private data center or cloud computing facility. This approach can increase edge intelligence across a diversity of inputs, increase availability and reliability, improve real-time response through reduced delays and system overloads, reduce costs of networking, and increase security and privacy of the data. At the same time, cloud computing can support and supplement such edge deployments of AI by running the AI model during training and retraining, managing the latest versions of the AI model and application, and processing more complex requests.

AI at the edge holds immense potential for companies to navigate the IoT data deluge and unlock its value. By addressing key challenges and making strategic investments, companies can leverage AI to transform their operations, optimize processes, and gain a competitive edge in the data-driven future.

3. The Cloud Grows Up: From Cloud-First to Cloud-Smart

For years, the cloud was touted as the be-all and end-all for companies’ IT needs. The cloud has become an operating principle driving the data platforms we operate on and grow our businesses. The lure of cloud scale, speed, flexibility, and simplicity has driven both large as well as smaller, growing businesses to invest in massive digital transformations. But that ‘cloud-only’ or ‘cloud-first’ approach has run into its own wall of reality. IDC data indicates that 70 to 80 percent of companies are ‘repatriating’ at least some of their data back from the public cloud. This includes both large enterprises that made wholesale migrations to the cloud without sufficient preparation, as well as cloud-native startups that have achieved scale.

Said differently, the future of IT infrastructure lies in the Hybrid Cloud, a balancing of workloads between the public cloud (even multiple public clouds), on-premise and co-location environments. Behind this Great Rebalancing are concerns about the costs of the cloud in terms of data transfer fees, concerns about security and privacy, requirements for data sovereignty given country-specific data residency laws, and even performance considerations for latency-sensitive and mission-critical applications.

The need of the era is to be ‘cloud smart.’ Where data sits requires a thoughtful approach and depends on a number of factors: the nature of the applications, the contents of the data, the profile of its users, and the requirements and constraints of the geographies involved, among other things. A hybrid cloud foundation supports robust data management, which in turn enables more effective data use through powerful AI applications that feed business decisions and actionable insights. IDC data indicates that more than 50% of investments in GenAI projects in the near term are being allocated to digital infrastructure.

The right choices in infrastructure can be critical to maximizing the return on data for businesses. Investments in hybrid cloud management platforms, including FinOps capabilities to manage cloud costs, will grow in importance as businesses establish the foundation they need for future data-driven growth.

Cloud-like consumption will be a part of that story: expect a continuing drumbeat towards "everything-as-a-service". A focus on outcomes and delivery against SLAs, with the ability to pay for what you get, will bring the simplicity of cloud payments and usage to the emerging hybrid cloud setups. Tech leaders who embrace as-a-service (aaS) models in 2024 stand to gain significant advantages in terms of agility, efficiency, security, and cost optimization. By adopting a strategic approach to aaS adoption, they can unlock new opportunities for growth, innovation, and competitive differentiation in an increasingly digital world.

4. Cyber Resilience Requires Business Transformation

The threat of ransomware is not new. There is no question, though, that these attacks are getting more sophisticated and opportunistic, including the use of AI to identify and target system vulnerabilities. The potential fallout, too, is more severe. Data theft and extortion incidences are increasing, and Ransomware-as-a-Service platforms have begun to proliferate, even as compliance requirements grow in rigor. A report from Sophos stated that ransomware affected 66% of all organizations in 2023, with the severity of claims also reaching a record high, according to the “Coalition 2023 Cyber Claims Report.”

The evolving threat landscape and the rising cost of breaches will require tech leaders to rethink their approach to security and recovery. Cyber resilience in 2024 will go beyond just preventing data breaches and maintaining business continuity. With the integration of technology and business in all aspects of business operations, additional risks emerge that call for a more comprehensive approach to cyber resilience. Businesses will significantly invest in enhancing their cybersecurity capabilities through advanced threat detection, machine learning and adaptive behavioral analytics, and organizational policy and process upgrades.

Collaborative approaches will be essential: from information-sharing across companies, vendors, and government agencies, to threat databases and other shared resource development, to joint cybersecurity preparedness planning.

Promising trustworthy outcomes to their own customers and stakeholders, companies will increasingly place trust at the top of their own criteria when seeking strategic partners. So, the ability for infrastructure and data solution providers to guarantee an ‘unbreakable’ data foundation will become table stakes. Cyber resilience is no longer just a technical challenge, it is a business imperative high on the management’s radar. Reactive measures must give way to the proactive. Go-it-alone approaches need to cede to the strength that can be built with a trusted ecosystem.

5. Sustainability

According to the International Energy Agency (IEA) and 8 BillionTrees, data centers today use 200 TWh of electricity annually, with 3 to 5 million gallons of water a day (enough for 30,000-50,000 people), and generate nearly 4% of global greenhouse gas (GHG) emissions, more than the GHG emissions from the aviation industry. And all that is before the impacts of AI. The Synergy Research Group reports that hyperscale data center capacity alone will almost triple in the next six years, driven by AI. Gartner believes that AI may consume more power than the human workforce by 2025, offsetting carbon zero gains. With the increasing adoption of AI and growth in complexity of machine learning models, the consumption of data, power and compute resources will only grow.

In the face of all this, the good news is that AI can also be part of the solution. Sustainable AI practices can create a real dent and drive improvements in efficiency, even overcoming the footprint of AI. These include hardware optimized to reduce energy consumption, federated learning, energy efficient coding, etc. Gartner indicates that this impact could even be between 5 and 10 percent reduction in carbon dioxide emissions.

This is what we expect to see in 2024: companies increasingly wrestling with the balance between doubling down AI and digital investments and stepping up their focus on minimizing their environmental impacts.

The choice of storage and infrastructure hardware providers is crucial for companies planning their digital transformation investments: the right choice can determine the footprint of these companies for years to come. In addition, we hope to see additional data center investments in renewable energy such as solar panels and wind turbines, cooling techniques such as free cooling, and managing server utilization during off-peak hours.

The road ahead for 2024 may look daunting, with this swirl of intersecting and often competing forces creating a fog of uncertainty. Even so, the possibilities outweigh the challenges, and we anticipate a year where bold investments in data begin to truly pay off. Businesses that take this journey need trusted partners to tackle this adventure together – and we look forward to being that partner, helping you along each step of the way.

Additional Resources