Big Data is all about process

It sounds like a trivial point really but it is something that I have seen project teams get wrong time and time again.

The logic is simple:

1. Advanced Analytics (and particularly Big Data Analytics) is all about gaining insights you would not normally get from regular reporting dashboards and descriptive analytics.

2. These insights are seen to be most useful when they result in action.

3. Action is a doing word and therefore associated with the steps in a business process.

4. Hence to deliver real business value from Big Data Analytics you need to start with the process.

I think most people get this and they genuinely try to start in the right way but somehow the fascination with the data takes over.

Let me give you an example.

The starting point for a recent project was the notion that propensity to churn for mobile phone customers could be predicted based on their observed behaviour. So we started to analyse data that represented behavioral patterns in order to correlate that with customers who churned and customers who did not. We produced some interesting insights … for example it led us to highlight an illegal narcotics trafficking ring due to their odd calling patterns. And the data did help us produce a strong correlation between certain patterns of mobile usage and customer churn. But a strong correlation is not necessarily a predictor and we struggled with finding that until someone came up with the clue.

Obvious really … they said, “why don’t we look at their behaviour around significant events, like contract renewal dates, payment due dates, and so on”.

We thought about it for a while … and decided to draw up some process/workflow diagrams around the customer buying/usage/payment cycle (actually today you might describe this as a customer journey map). And then we mapped the data we were collecting and started analysing against this, giving different weights/importance to the data depending on where in the process the customer was.

My simple point is this.

Before even collecting data, let alone trying to do simple statistics and analysis, start with a blank sheet of paper and map out the process that you hope to change. From that process map you can gain insights into what data you need, when you need it and what sort of analysis you will need to do on it. Think in terms of big picture – meaning think in terms of what is also happening in parallel (like the weather changing, people’s work and employment scenarios changing, sporting events that might influence behavior, political influences and so on).

Map your hypothesis against a business process that can have an actionable result. That way your Big Data project will at least start with a purpose and be framed within a business need.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s