Is there a challenge related to having too much data? One, among many, interesting findings pointed out in one of the reports emerging from Knowit Insight’s ongoing collaboration with KTH (The Royal Institute of Technology in Stockholm) suggests that this is the case. Too much data may in fact pose a great challenge to companies having come rather far in terms of collecting, processing and analyzing data in their quest of becoming data-driven businesses. In particular as it may significantly aggravate the ability to make good decisions based on data. We find this of special interest, as it is something we are recognizing as an increasingly common factor our clients struggle with.
Working in a data-driven manner has, for some time now, been the topic on everyone’s lips. As competition intensifies and companies are pressed to sustain growth rates, enabling data-driven growth is described as a key enabler for business leaders aiming to transform and futureproof their business models. Google and Amazon are often described as role models, who early on were able to create powerful business models based on an ability to efficiently exploit large amounts of data. The power of data is though, not only recognized by companies such as Google and Amazon. Companies of a variety of sizes and across industries have had data and information management at the top of the agenda for almost over a decade now.
Yet, far from everyone succeed in realizing the full potential of using data. A recent article by Harvard Business Review reports that leading companies appears to be failing when in comes to becoming data-driven, and the percentage of firms that identifies themselves as data-driven businesses has decreased over the past years. Moreover, a majority of executives report that business adoption of data-initiatives is a major challenge.
While the reasons for this are many; according to the article from Harvard Business Review, people and process issues, cultural aspects and lack of organizational alignment are recognized as major obstacles; we believe that access to too much data is also one part of the problem. As digital tools for gathering data are becoming increasingly efficient in collecting extensive amounts of data, data supply soon comes to outweigh demand. As a result, it is becoming more and more common for companies to find themselves at the verge of drowning in data which they are not able to absorb. When faced with more data than can be handled, ensuring that the right decisions are made based on data, becomes a real struggle. In such a situation, companies must be very careful not to fall victims for decision-making biases. While there are numerous biases which we risk falling victims of within decision-making in general, we recognize a few of them as increasingly important to be observant of when it comes to decision-making regarding data.
- Firstly, overconfidence in data, which comes from the tendency of believing we have more accurate and complete data/information than we actually have. As a result, we systematically overestimate our ability to make predictions based on the data available to us, thus causing our decision-making to be skewed as we assign too great of importance to data.
- Secondly, confirmation bias, referring to when data is interpreted in alignment with the interpreter’s own personal beliefs, views or opinions. As we tend to search for or assign greater importance to information that confirms what we already believe in, it may cause our decision-making to rely too heavily on a subjective interpretation of data than is objectively reasonable.
- Thirdly, overfitting and underfitting, which refers to the potential misconception that an overly complex model attempting to incorporate multiple data trends is bound to produce the best results. However, when combining a great number of parameters, unnecessary noise and minor fluctuations occur, thus leading to an enhanced risk of underlying trends becoming ignored. As a consequence, the data interpretation becomes faulty.
- And lastly, availability bias, which refers to the perception that inferences can be based on readily available or recent data only. In other words, we have a tendency to believe that immediate data is relevant data. However, recent or readily available data does not necessary reflect the full picture. Instead, there can be great disparity been what appears to be happening and what actually happens and base our decisions solely on the most recent data, chances are great that they might not be the most accurate ones.
So, what can be done to ensure that extensive amounts of data are successfully incorporated into an organization while avoiding exposure to decision-making biases? At Knowit Insight we believe it’s a combination of technological adjustments, organizational readiness and transformational leadership. Contact us and we will tell you more about how we can utilize our proven methodologies and tools to help your organization successfully transform into a data-driven business.