In a recent blog, marketing guru Seth Godin observed… “Without a doubt, the ability to connect the dots is rare, prized and valuable. Connecting dots, solving the problem that hasn’t been solved before, seeing the pattern before it is made obvious, is more essential than ever before.
Why then, do we spend so much time collecting dots instead? More facts, more tests, more need for data, even when we have no clue (and no practice) in doing anything with it. Their big bag of dots isn’t worth nearly as much as your handful of insight, is it?”
In a world which is increasingly focused on measurement of impact (social, financial and environmental in the world of social enterprise) it’s easy to get sucked in to following the crowd.
A recent publicity blurb for a 2-day course on measuring social impact makes the case for it succinctly… As well as making your organisation more attractive to funders, being able to measure your social impact will enable you to measure the effectiveness of your organisation, to benchmark with your competition and allow you to see where improvements can be made.
Yes – it’s important to demonstrate how well you’re fulfilling your declared purpose/ mission, particularly if you’re spending someone else’s money. But often the record-keeping is for the (questionable) benefit of others and this can tie you in knots and take you away from the activity you’re spending so much time measuring!
We know that most impact is qualitative rather than quantitative and not easily measured. There’s a whole industry around giving such measurement scientific weight eg through social audits and financial calculations that aim to put a figure on the social return on investment – SROI – in organisations. But the people I know who have had this done for their organisations have been disarmingly honest about the dubious reliability of the figures they fed into the process. As they say about computer analysis – ‘rubbish in, rubbish out’.
A wonderful ‘bubble-bursting’ piece by Jeremy Nicholls – a champion and authority on SROI – takes aim at the idea of measuring social impact. He says it assumes we make rational decisions based on evidence (whereas we base them more on preconceptions) and that we have a belief that a shared view is, by definition, correct. Then there are the limitations of predicting the future based on the past.
I bow to Nicholls’ knowledge about the science behind the SROI process (he’s Chief Executive of the SROI Network after all) but I’m not as confident that we don’t sometimes misplace our trust in science and wilfully or otherwise select so-called facts and figures to suit our narrative.
It’s easy to knock the whole process of measurement and counting – and many people do (I suspect because they consider it unnecessary extra work) – but it may not be a matter of all or nothing.
Squawkpoint blogger James Lawther advises caution when counting (mushy peas is his example) but, in a linked blog, he has suggestions for interpreting the figures once gathered. He recommends looking at the bigger picture – patterns and trends over time – and putting your energy into making sense of the biggest and smallest changes over a period. And in a ‘seeing-wood-for-the-trees’ attempt at making sense of all the figures available – use the 80:20 rule to identify which numbers you really need to worry about.
Many years ago, the New Economics Foundation was influential in my thinking about appropriate measurement. They advocate that you decide what measurement is relevant, meaningful, and achievable in your particular community or organisation. So if dog poo in a public park is a measure of a community’s respect for public spaces, put flags in the poo each month and take photos and compare the numbers of flags over a year. That was in the days before hefty fines and public disapproval – an updated version of this measure might be the number of poo-filled bags left hanging on branches or the number of bags strewn around overflowing bins.
Returning to the ever insightful Seth Godin, he sums up for me the importance of cultivating a healthy scepticism, or at least a questioning attitude, towards measurement in another recent blog.
“We keep coming up with new things to measure (like processor speed, heat output, column inches) but it’s pretty rare that those measurements are actually a proxy for the impact or quality we care about. It takes a lot of guts to stop measuring things that are measurable, and even more guts to create things that don’t measure well by conventional means.”
When measuring to satisfy outside rather than internal interests, at least be bold enough to ask them why?
Seth on dots http://sethgodin.typepad.com/seths_blog/2014/04/connecting-dots-or-collecting-dots.html
Jeremy Nicholls takes aim at social impact measurement www.pioneerspost.com/comment/20140404/the-wild-wild-west-of-social-impact
James Lawther on counting mushy peas www.squawkpoint.com/2014/01/operational- and the limits to counting www.squawkpoint.com/2013/06/management-information
Seth on ‘measuring nothing with great accuracy http://sethgodin.typepad.com/seths_blog/2014/01/measuring-nothing-with-great-accuracy.html
For an excellent ‘Prove and Improve’ toolkit from the New Economics Foundation, go to www.proveandimprove.org