A Harvard Business Review (HBR) blog entitled The Embarrassment of Complexity concludes that we need to:
“Draw on the faculty of human judgment to focus on the smaller picture in order to comprehend the larger one”
But why, at a time that Big Data grabs so many headlines, should we focus on small data? Well, for several reasons, small turns out to be really interesting:
- As people interact over time, small differences between them can create novel patterns across organisations, industries and communities. And these can never be predicted in advance. In a similar vein, who’d have thought that flocking starlings could resemble a whale chasing a dolphin across the sky?
- Small events can have large consequences, known as the Butterfly Effect. Anyone remember the throwaway comment which led to the demise of Gerald Ratner’s business empire?
- Outliers – small numbers of odd, unexpected, unusual things, cannot safely be ignored. They deserve our attention. We can ask ‘why are they there?’ and ‘what do they mean?’, just as Malcolm Gladwell, or Sir Alex Ferguson have done.
- Small things can serve as weak signals – giving us early warning signs of emerging problems and opportunities. In 2008/9 we found it hard to believe that more questions hadn’t been asked about bankers’ bonuses.
So big data is big, but it’s the computer analytics that help us to see patterns in that data, which are clever. The danger here is that we’re always using our old ways of thinking to make sense. So we may well miss small data and weak signals which are potentially frame changing – heralding completely new patterns that we may never have seen before.
The clever thing about small data is how it can signal new opportunities and emerging problems. If we notice and interpret that data, then we can choose how we respond much sooner – helping us to capitalise on emerging opportunities and head off emerging problems
It’s here, in noticing and interpreting small data, that we need to cast off our reliance on data crunching machines and start using and honing our very human skills of perception and judgement. So that’s why I agree with HBR that using human judgement to focus on the smaller picture in order to comprehend the larger one, is such a crucial skill.
