Monday, 1 April 2013

Counting what counts

A recent report entitled Counting What Counts by Anthony Lilley and Prof Paul Moore encourages us to use what they call Big Data. We don't say that we follow every step of their argument which seems to confuse use of data in decision-making  with data drawn from web usage but we were struck by some of their general comments which echo our views on the use of data generally. Here are some paragraphs from their report.

They start with that brilliant quotation from Einstein: Everything that can be counted does not necessarily count; everything that counts cannot necessarily be counted. Essays should be written on this.

Their two main themes are language ... The sector exists in a context of narrative of subsidy and market failure which means that measurement is too often seen as a burden placed on defensively-minded organisations as a form of accountability and the use of data ... Data and information are most often used in arts and cultural settings somewhat passively, mainly to satisfy the needs of accountability.

Too often the gathering and reporting of data is seen as a burden and a requirement of funding or governance rather than an asset to be used to the benefit of the [organisation]. This point of view ... arises partly from the philosophy of dependence, subsidy and market failure which underpins much of the cultural sector.

The subsidy model tends to trap many cultural organisations in a survivalist, financial mindset and that, in turn, makes it difficult to adopt an expansive, entrepreneurial perspective when under regular implied threat by accountability.

Without the pull of a more enlightened approach to data from funders and regulators, the likelihood of widespread adoption of a more modern approach to data at the level of organisations is low.

A shift in mindset ... is a requirement. Such a shift would match much of the rhetoric of 'investment' which is used in the sector, particularly by policy and turning bodies. ... this rhetoric has largely been just that: a new term to replace the loaded word 'subsidy' rather than a genuine change.

The Arts Council England's recent ten-year strategy, Achieving Great Art for Everyone uses the term 'investment' itself no less that twenty-one times. However, there is considerable mismatch between this use of terminology and the reality of how commercial investment actually works. Too often 'investment' is used as a synonym for 'subsidy' and this linguistic sleight of hand is, in fact, sometimes harmful.

'Subsidy'. ... is a loaded term which indicates an in-built power relationship. 'Subsidy' is given to something which is weak. Conversely 'investment' is something from which we expect returns and which is thus imbued with potential. Framing public investment in culture as 'subsidy' positions it as a weak, dependent activity. When in harmony, investor and investee should be committed to achieving the same ends.

Being data-rational is a skill and not something that can imposed or encouraged by the dull demand for 'metrics' so beloved by funders. You first need to collect the right data - data which is relevant and representative - then to turn this into useful information for decision-making. 

Not understanding the difference between good and bad data, or the way figures can tell a coherent story, our funders frequently ask the wrong questions and seem satisfied with any numbers. 

I suppose you would not expect 'artists' to understand something as rational as 'numbers' which come from the other side of the great arts-science divide. Their use of the word 'investment' is probably the same: a desperate attempt to adopt the language of a business world they so little understand.