The image of a snowball getting bigger and bigger as it rolls downhill usually appears in cartoons for comic effect. However, it isn’t very funny if your company is sitting at the bottom of that hill paralyzed by a deluge of data. Too many companies are unable to navigate between the demand to accelerate analytics and operational initiatives to reach business goals and the pressure to slow down to ensure compliance with evolving privacy regulations.

Why analytics projects fail 

• The tension between trying to support revenue growth and lower operating costs.
• A lack of insight into what types of data reside where, and who can access it for what purposes.
• The data pipeline bottleneck created by the lag time between data collection and availability for use.
• The increasingly justified fear that noncompliance with privacy regulations will lead to large fines and brand damage.
• The complex array of point solutions that are supposed to help with all this. 

No wonder VentureBeat reports 87 percent of data science projects never make it into production. MIT Sloan finds 81 percent of organizations don’t understand their data because it’s still locked in silos. And a recent HBR survey reveals 69 percent of companies have yet to create a data-driven organization.

The challenges to becoming data-driven organization

Companies that want to overcome paralysis while taking the potential for calamity seriously have prioritized data management, with many appointing a data czar or chief data officer (CDO). Still, obstacles abound, including a perceived conflict between rapid value creation and the risks associated with privacy compliance. Organizations also face resistance to change, an inability to drive collaboration across departments, and the failure to recognize mature information governance (IG) as a core business function required for the management of the organization-wide data lifecycle. Gartner predicts only half of CDOs will be successful. 

As these challenges suggest, the problems organizations face have as much to do with people and processes as with technology. Here are four tips to help your company become a data-driven culture: 

• Add business value through compliance. We cannot accept a conflict between rapid business value and ensuring compliance. These must be part of a single equation.
• Engage information governance (IG) and compliance teams. IG and compliance professionals must collaborative to help business users rapidly launch their analytics initiatives. It makes no sense to be a compliant organization that can’t compete.
• Embrace compliance as an opportunity. To ensure their companies remain competitive, business users must embrace compliance requirements. Rolling out data too quickly and putting the organization at risk of fines and brand damage will do more harm than good.
• Acknowledge the need for ethics. In the era of increased awareness by consumers of the threat to their data, mere compliance may not be enough. To protect their brands and reduce the potential for blowback, organizations now need to ask if their practices are “ethical.”

Solving data challenges with a DataOps Approach

In providing enterprises with a methodology and set of best practices, DataOps is similar to successful operational initiatives in other industries, including, FinOPs, Marketing Operations and DevOps. DataOps borrows best practices from DevOps, data management and data governance and creates a framework for collaborating on developing and maintaining trusted data flows and pipelines across multiple stakeholders. 

A key goal is to make business-ready data available fast by ensuring data is secure, high quality, compliant and easily accessible to business users and data scientists. To do this, specific DataOps strategies include: 

• Breaking down data silos to create a single, comprehensive view of data, so organizations can understand what data they have and where it is. 
• Rolling out easy-to-use self-service capabilities.
• Ensuring data can be trusted through effective governance, including tracking data lineage.
• Enabling collaboration across stakeholders and subject matter experts.

DataOps can unlock the value of your data 

DataOps achieves these goals by utilizing artificial intelligence (AI) and machine learning (ML) to streamline the data pipeline and automate the following processes, which are still currently manual and very labor-intensive for many organizations:

• Data curation services, including auto-discovery and classification, sensitive data detection, quality analysis, and auto-assignment of business terms.
• Core governance and master data management services, including automated data lineage creation, policy management and enforcement.
• Open metadata management, which becomes the knowledge catalog for the enterprise for any type of assets.
• Hybrid cloud data integration, movement and virtualization, including multicloud optimization and replication.
• Self-service capabilities for search, data preparation, workflow and collaboration.
• Enforcement of data privacy and governance policies.

DataOps creates value by shrinking the time between when data from a variety of sources can be accessed and when business users and data science experts can confidently consume it. This allows companies to quickly launch new analytics initiatives with reduced risk of regulatory noncompliance and improve operations faster by rapidly making trusted data available. Today, CGOC sees this growing in importance across industries and will look to provide more resources and opportunities for education and sharing, including case studies. For example, companies that have launched DataOps initiatives report some impressive results, such as an 85 percent reduction in the time to create a business glossary, a 90 percent  reduction in the time to discover meta data and assign terms, and 200,000 technical assets discovered across multiple clouds in less than five minutes. More details to follow.

If you’re already moving toward a DataOps framework and have anecdotes or resources you think the membership can benefit from, we’d love to hear from you. And as always, share your questions and experiences on twitter @CGOC_Council or LinkedIn.

Additional Resources:

• How DataOps helps organisations make better decisions
• DataOps challenge – the complicated art of making things simple
• Data Ethics: Balancing Risk and Reward
• Information Governance eBook
• Information Governance Process Maturity Model 
• What is Metadata Management? [Video]