The accumulation of data is increasing fast – from wearables, the widespread deployment of sensors in physical locations and the ever increasing use of the Internet by people.
And someone somehow has to figure it all out. As a result, data scientists are in demand and analytics is a hot new field of study. On top of long standing statistical methods, there has been impressive progress recently in machine learning, artificial intelligence and new computer system architectures.
Yet, the use of analytics itself has not had as great an impact on many organizations as the data scientists have hoped. Some of the failures of analytics were really failures of implementation.
Perhaps the most public of these was the great Netflix million-dollar prize for a new recommendation engine. From a purely technical viewpoint, the winning team did exactly what was asked for – create a significantly better engine than the one Netflix had been using. Nevertheless, Netflix ended up not using their work. That’s an issue of implementation and integrating the product of analytics into an organization.
Being able to predict behavior or even generate new insights from all this data is one thing. As with Netflix, having people and organizations actually use that knowledge is another. Like many other new technologies, adoption is as much a question of managing change as it is developing the technology itself.
This surely bothers some data scientists. After all, they have a better mousetrap – why aren’t people using it? Being able to think quantitatively, they can prepare quite convincing business cases with impressive ROI statistics and yet even that isn’t enough to get executives to budge. But changing an organization isn’t simple no matter how good your arguments.
Despite this background, there has been very little overlap between the courses that prepare data scientists and the courses that prepare change agents in organizations.
Later this year, I’ll be doing something to help align these two fields to improve the success of both. I’ll be teaching an online course on analytics and leading change. It will be part of Columbia University’s new executive graduate program in Applied Analytics.
We’ll be reviewing what is known about successfully introducing changes into an organization from the classics on the subject that were written as much as twenty years ago to more recent research. The course will, of course, help its students understand how to get an analytics initiative started. More important, it will focus on how to sustain, over the long run, both analytics and the changes it informs.
Thinking about the long run, there are three facets of the relationship between analytics and organizational change.
- The use of analytics as a part of everyday decision making and the rules of operation in a business – this is the obvious first thing everyone thinks of.
- The use of analytics to help better implement the changes that its insights imply – a kind of a meta-analysis.
- The continuing interaction between analytics and change to finally achieve the long desired goal of an organization that learns how to continually optimize itself – this is something of great strategic value to any business.
As the course develops, I’ll be posting more about each topic.
© 2016 Norman Jacknis, All Rights Reserved
[http://njacknis.tumblr.com/post/136679679430/analytics-and-organizational-change]