March 01, 2022
The historian and philosopher Will Durant once paraphrased Aristotle when he wrote, “excellence, then, is not an act, but a habit.” It’s a poignant commentary on process and behavior and one that comes to mind during these nascent days of the data-driven enterprise. Too often organizations repeatedly expend resources on ad-hoc efforts to try and improve data quality for decision-making, which is simply inefficient and costly.
If a business’ future is to be truly data-driven, then providing people, processes and systems continuously with accurate, relevant, timely data is critical. As Gartner noted recently, “Poor data quality destroys business value.”
Much is written about the high cost of poor data quality, but perhaps a more productive perspective is to discuss substantial ways of achieving it and the opportunities they unearth.
Here’s is a general framework founded in principles of DataOps that best describes the areas of data management to be optimized on the way to achieving the highest standards of data quality:
Data ingestion: These efforts will focus on bringing in data from various sources and transforming it into a usable form that is searchable, curatable, and readable. Aligning these efforts with business requirements and goals will help to improve the access, fitness, and accuracy dimensions of data quality.
Data curation: Cataloging the data so that it can be found and searched for, as well as to ensure it complies with regulatory requirements. Combining AI, ML, and automation techniques speed the processing of data for users and make it possible to evaluate progress toward improved data quality. Using automation to identify gaps in data quality and recommendations to addreess those is critical to engineer quality by design. Accuracy, fitness, access, and timeliness are all enhanced by the work done in this area.
Data publishing: Contextualizing data so that it is enriched with information that makes it more relevant for self-publishing users. The objective is to improve both the fitness and accuracy dimensions by using automation and integration measures that reduce the amount of work required of data consumers.
Data quality is a pursuit and to be good at it requires commitment and diligence. The only way to drive more accurate data-driven decisions, is to leverage the most accurate, clean and complete datasets. Make this a habit and unleash the power of your data to grow your business.
Be sure to check out Insights for perspectives on the data-driven world.
Ajay Vohora is Vice President, Business Development, Hitachi Vantara.