Tracing Our First GenAI Steps

This is the second story in the “Our GenAI Journey” series.

Sometimes in the world of tech and engineering, you simply have to seize upon a good idea. You enter the fray – understanding that the inevitable disruptions will bring new risks – but you enter, just the same. You resolve to build the plane as you fly it.

This, of course, runs counter to what we’re accustomed to in an enterprise world. We like to study every angle, scrutinize every decision, and pit functionality against cost analyses along the way. We may not even discuss or share an innovative idea for months, until we can do so with a body of research to back up the decision-making, including information on the target market and the competitive analysis.

These are some of the things that flashed before my eyes when I finished a brief presentation on innovation at our company’s Executive Leadership meeting this past summer. Upon closing my deck and asking if our leaders had any questions, our CEO and now Executive Chairman of Hitachi Digital Services, Gajen Kandiah, stood and announced to the team that I would be leading our company’s Generative AI (GenAI) charter.

Even though GenAI was clearly already on the minds of everyone in the meeting, the pronouncement was as surprising as it was exhilarating. To be tasked, on the spot, so decisively and in such a setting was as much of a vote of confidence from Gajen, as it was a statement of direction. Gajen has been a thought leader for our company in this space and his involvement and passion on the topic and its potential has been well known.

Gajen would explain to me later that he and the board selected me for several reasons. My being with the company less than a year, for example, was a key element. They wanted the leader of this work to be both deeply capable with experience, yet new enough to the company to be able to drive disruption within the organization and challenge the norms. This, he said, was going to be critical.

As I smiled and nodded in acknowledgment and agreement, my head was spinning, crafting to-do lists and scribbling on the white board in my brain steps we would need to take.

Drafting Our GenAI Team

I wasted no time. When I returned to the office the next day, I discussed the prospect with our Chief Transformation Officer, Santhosh Sreemushta, and began the journey by drafting what I thought would make the most effective constellation of five cross-functional teams: Policy, Product and Engineering, Services, Internal IT/CIO, and Marketing.

In order to get the ball rolling and avoid the pitfalls of traditional bureaucratic delays that can accompany new business plans, I decided that team members would have to tack this new responsibility onto their “day jobs.” (Let the disruption begin, I thought.) So, I put a list together of who I thought would make good leaders and emailed each of them separately asking for their support. In doing so, I made a point of letting them all know that this work required a trailblazer approach and the kind of people who were comfortable dealing with ambiguity.

This was critical, because as part of this journey I wanted to find ways to make GenAI part of our company’s DNA. If I could do that, it would ensure a long-term commitment, something that can be embraced by those who come after me and the teams. It’s a critical component to creating structure as well as sustainable momentum.

Convening the Team

I had the leaders put their teams together and send me their names. I wasn’t surprised when they told me the engineers, researchers, and marketers were all eager to join. By the afternoon our organization was set.

That was mid-June. Looking back, it’s no understatement to say that we moved on this mission as fast as the industry has around GenAI, which rocketed from the rise of OpenAI’s ChatGPT a year ago to the rollout of the impressive Chat GPT platform last month. From that momentous leadership pronouncement to our first official meeting as a GenAI strategy team, a grand total of two weeks transpired. That’s agile in anybody’s book.

I convened the troops a week later. We used the time to craft our blueprint for how we would operate, assign roles and responsibilities and also goals; not KPIs yet, but goals. We’re not at the KPI stage.

Drafting Our North Star

We agreed that our blueprint must do several things simultaneously: establish the guardrails, as Gajen has stressed; align with the values of our company; and provide flexibility for evolution and growth as we all learned more.

To do this most effectively, we had to start from the business standpoint and work inward, giving all stakeholders an equal voice. The discussion included members from the Chief Information Security Office, the CIO’s office, Corporate Legal, and product development and explored every corner of Hitachi Vantara that has or could have a GenAI touchpoint. From the usage of public GenAI apps, like ChatGPT, Bard and now Gemini, to private apps, to co-pilots in software development, we set out to craft a policy paper that officially dictates the approved usage.

As a first action item we agreed to draft a policy paper that could be used by all employees for how and when to use public and private GenAI tools. From product development all the way through to marketing, we set out to lay out the parameters for company-approved practices. It would be the north star for the company. The paper would be visionary as well to help guide development today and into the future. Remember my mission – make it part of the DNA.

This seemingly practical approach actually turned out to be a critical juncture for us. As we dug into the issues for the policy paper, we soon realized it was going to be an endeavor in and of itself. From crafting rules and restrictions around general use and data privacy, to putting guidelines around issues of regulatory compliance, cybersecurity and even policy training – this was going to take time; the kind of time that could stall if not derail the rest of the effort.

But these are unprecedented times. The pace of innovation around GenAI is so fast that it required a heightened sense of urgency. Our people needed the freedom to explore at the same rate at which the tech was moving. We needed to press on, so we agreed to craft the policy paper in parallel with the execution of the rest of the blueprint. We would indeed build this plane as we were taking off.

Bringing My White Board to Life

From there, we outlined our market perspective breaking it down into two distinct paths: 1) determining how to approach integrating GenAI into existing products and, 2) putting creative thought around what we might build from scratch.

We literally white-boarded a healthy list of great, creative ideas and then set about paring that list down. With so many ideas on the board, we had to cut it way down and zero-in on two or three ideas that could have the most impact out of the gate. Boiling the ocean was not an option. In fact, casting a wide net was a recipe for distraction and delay.

Establishing this philosophy early has proven to be quite productive. It has enabled us, as a team, to crystallize our priorities with speed.

So, in addition to the policy paper, we settled on developing two initial projects: GenAI agents, called “companions,” to speed problem resolution for our enterprise support teams (and ultimately integrate them directly into our portfolio to improve system configuration, management, troubleshooting, etc.); and an “out of the box” appliance we would use to fine-tune LLMs and develop companions. 

Applying What We Know to What We Learn

We’re excited to share our stories. There’s a lot going into the thousands of decisions being made within the walls of Hitachi around our GenAI work. But we know we’re not alone. We speak with customers and partners every single day, so we know that enterprises are having the same discussions around the same topics and challenges. And if they’re not, they will.

For example, when it was time to put our blueprint to practice, we realized we needed to first settle a philosophical debate – should we adopt single large language model for our work, or a small model approach that comprised multiple agents that could collaborate with each other?

Like many enterprises out there, we weighed the differences, and for us, our decision to adopt the small model approach came down to at least two key attributes: 1) smaller models have the ability to generate more accurate outcomes, and 2) with far less risk of hallucinations. On the other hand, leveraging one single large language model would be simpler to manage, relatively speaking, yet more expensive to train.

After some healthy discussion, we agreed to adopt the small model approach to create our copilots. This was a great example of applying our technical expertise to what we’re learning as we work through our GenAI journey.

We’re going to extend this idea of learning in Part 3 of the series. While I’m writing from within our internal development work, my colleague, Prem Balasubramanian, of Hitachi Digital Services, will put pen to paper next to share what he’s been working on with a range of customers as they build new models and tackle new problems together in real-time.

GenAI is a journey for all of us and we need to get immersed. 

Related

• Gajen Kandiah: Introducing Our GenAI Journey
• A Generative Approach to AI