At first glance, successfully leveraging data within your organization may seem to only be a question of investing in cutting-edge technologies and hiring enough data scientists. However, simply grafting a data team onto an existing organizational structure, no matter how skilled and equipped that team is, can only take you so far. While hiring more analysts and data scientists can help achieve islands of excellence within sections of the business, building a sustainable enterprise-wide data culture is a very different ballgame.

A 2018 McKinsey Analytics survey, for instance, found that only 8% of companies achieve analytics at scale, where analytics is embedded into all areas of the organization.¹ Even more generous estimates suggest that less than a third of companies have managed to create a data-driven organization.²

Such high failure rates arise because data is often an afterthought for most companies, resulting in ad-hoc and siloed approaches that fail to deliver the full value of data. Going beyond hiring one or more data science teams, here are five challenges that have to be overcome to achieve more than isolated success with data and analytics:

The Challenges

1) Lack of data literacy and democratization:

Extracting the full use of your organization’s data is rarely a question of a few big-ticket decisions. Rather, it depends on a series of everyday decisions taken at different levels and across different departments of the organization. This also means that access to data, together with the skills and tools to understand it must be available across the organization.

Business and product teams are typically unaware about what data is being captured and used. Analysts and data scientists constantly face the challenge of having clear documentation on certified and reliable data sources in enterprises. To mine through data sources and Wiki/Confluence links to get a sense for available and reliable data requires a process of constant discovery. We see this across enterprises. Typically, only a select few tenured team members possess knowledge and are data literate.

However, most organizations are unable or unwilling to spread access and literacy to data across the organization, particularly to frontline employees. A 2020 survey by MicroStrategy, for instance, found that less than 45% of organizations provide data and analytics access to more than half of their employees.

Source:³

Even where access is provided, costly bottlenecks occur when diverse sets of employees aren’t able to rapidly and agilely use data to develop hypotheses, test ideas, and design decisions.

2) Lack of Data Ownership

While a lack of data democratization can hobble organizational data initiatives, a situation of every department for itself can be just as harmful. A data-driven organization is a complex collection of moving parts.

When a clear structure of data ownership isn’t set in place, the organization’s data remains siloed in multiple places, leading to multiple inefficiencies and harms:

i) Since data access is limited within siloes, different parts of the organization only see partial or incomplete pictures that limit the quality of their analytics.

ii) Different departments or business units would not agree on key definitions of the data, leaving the organization with disjointed data sets that cannot be connected together.

Massive amounts of wastage occur because different units have to duplicate efforts since repeatability cannot be ensured across initiatives.

3) Inadequate Data Infrastructure

A data-driven organization depends critically on an adequate level of technological infrastructure to collect, process and use data. However, infrastructure can be unsuited to the specific needs of the organization, lacking scalability, involving overcomplication that retards agility and rapid decision-making, and lacking self-service options for frontline users.

For instance, one survey by ASG Technologies found that while 88% of respondents said that they had the necessary tools to manage and glean insights from data, 60% reported spending over 10 hours per week manually finding and managing data.

Source:⁴

Such shortfalls in infrastructure mean that many organizations are not poised to process data in a timely and efficient manner, and can’t gain insights that allow them to respond effectively to evolving business conditions.

4) Lack of Alignment and Prioritization on Business Outcomes

Take a quick look at Gartner’s Top Data and Analytics Predicts for 2019, made in January 2019.⁵ Amid much of the optimistic forecasting of the spread and growth of data and analytics, the list also includes, “Through 2022, only 20% of analytic insights will deliver business outcomes.”

This sobering prediction is revealing of the dangers of the green field nature of data science, where ideas with potential can quickly turn into costly and unviable detours. Such failures can easily result if the organization doesn’t align its data initiatives with concrete business outcomes. Without such alignment, the data lacks crucial context within which it can be processed into insights.

Data initiatives can be untenable not only at the strategic level, but also at the operational, when the organization’s data capabilities and infrastructure don’t match the ambition of initiatives. Thus, failing to prioritize the viability in production of data initiatives can also lead to their failure.

5) Lack of Executive Support and Buy-in

McKinsey research shows just how important the culture aspect of data culture is.

Source:⁶

Importantly, as the graphic shows, the greatest lag between high-performing organizations and all others involves data leadership inclusion in the C-suite team. This is because a data-driven culture begins at the top, as organizational leaders normalize the use of data in decision-making through practical example, as well as by introducing initiatives and incentives designed to transform the organizational culture. Without such support and buy-in, data initiatives are destined to flounder as the crucial shifts needed to transform insights into business actions and outcomes will inevitably derail due to a lack of widespread acceptance.

A Long Road to Data-Driven Culture

A transformative shift from intuition to data-driven decision-making at an organizational level cannot be the responsibility of isolated sections or departments. And while individual data projects can produce much-needed efficiency and productivity gains, the deeper business problems in an age of data cannot be resolved with such operational tweaks and adjustments. Rather, they call for significant shifts in business processes in response to changing market conditions, customer behaviors, and competitor capabilities. Such shifts cannot be achieved with just a few additional data scientists, but only by the development of a data-driven culture at the organizational level.