Manipulating the numbers

Photo by SeaRick1

We all know this phrase: "There’s lies, damned lies and statistics". I get so fed up when that gets wheeled out in interviews as an accusation against statisticians. But I can see the reason why that is used as a casual dismissal of claims based on statistics because numbers can be manipulated so much. As I’ve said, they do not speak for themselves. I can make a number look big or small if I want to.

Let me give you an example. We had this EU referendum in 2016, and one of the famous claims on the side of the big red bus was: "We send £350 million a week to the EU. Let’s spend it on the NHS". Now, that number is wrong, but I don’t care about that. It was a brilliant use of a claim because it provoked so much discussion. You could argue that that single claim almost won the referendum.

But let’s assume it’s right. Is that a big number? £350 million a week? There are 60 million people in the UK, so that’s about £6 a week. That’s about two cups of coffee per week, but let’s do it per day. That’s about 80 pence a day, about a euro, a bit more than a dollar. That’s about a packet of cheese and onion crisps. They could have put that on the bus: “We each send the EU the price of a packet of cheese and onion crisps every day”. If they had, I don’t think that would have won them the referendum.

How can we trust the numbers?

The way we present numbers can be manipulated to make things look frightening, reassuring, big, small and so on. We have to be very aware of that manipulation, that it can happen. During the coronavirus crisis, there were a lot of numbers being pushed around by politicians. I actually went on television complaining about why numbers were being used at the daily press briefings – big numbers of tests and cases – without any context or explanation of how good they are, their quality or what they meant. I call this number theatre because it was just a performance that was going on. And I found it deeply irritating.

This is very strongly related to ideas of trust. Whenever I hear the word “trust” I go to Baroness Onora O’Neill, who’s a wonderful woman. She’s a philosopher of Kant from Cambridge, and she’s written so much about trust in a very clear, accessible way. One of the first things she says is that organisations and politicians should not try to be trusted. That’s getting it the wrong way around. She says the onus is upon the organisation or the individual to demonstrate trustworthiness, and then trust may be offered up because you’ve earned it.

The ethical responsibility of statistics

This is such a simple and powerful idea that’s had a lot of influence. I’m proud to be on the board of the UK Stats Authority as non-executive, and that oversees the statistics in this country. There’s a code of practice for statistics. The first pillar of the code of practice of statistics is trustworthiness.

This idea – are the statistics trustworthy? – is enormously important. It goes along with this idea that statistics should not be there to manipulate emotions, to persuade someone to do one thing or another, to reassure or to frighten them. It should be there to inform them, to enable them to be better citizens, to make better decisions, to actually understand what is going on in the world. I feel this deeply, essentially as an ethical responsibility of statistics in the modern world. It’s going to be something that becomes more and more important.

An ecosystem of fact-checkers

Photo by Giovanni di Lorenzo

There’s a complex ecology growing of what you might call fact-checkers, who scrutinise claims made on the basis of numbers, and this is wonderful. I’m a small part of it; I do media appearances, taking apart numbers on radio programmes. There are organisations such as Full Fact – there’s Reality Check on BBC – that do this. There are media organisations that try to do this for newspapers and, of course, bloggers and individuals.

This coronavirus crisis has brought a great blossoming. You might think of the citizen-scientist as being all over the place, but some of them are doing fantastic work. All the time I see really good analyses of the numbers being reported and complaints about any manipulation of those numbers. But we can’t do this all ourselves as individuals. It’s absolutely impossible. We have to do it as part of an ecology of fact-checking and critiquing of claims being made based on data.

Asking the right questions

There are some criteria that we could use when we hear a number. Journalist Tim Harford has got some very nice ideas about this, about how we should first of all examine our feelings. How do we feel about this claim? We should also look at the motivations of the person behind it. I always ask, why am I hearing this? Where is this claim coming from? Why is someone choosing to tell it to me at this time? The other very difficult thing to ask is, what am I not being told? What is being left out of the story?

This is so difficult; it’s like the Sherlock Holmes story, the dog that didn’t bark in the night. What is the evidence from absence of evidence? What am I not hearing? You have to use your imagination when someone tells you, for example, that eating yoghurt when you’re pregnant is related to asthma in children. You have to ask, what am I not being told? Did they look at everything people ate while pregnant, and everything that might go wrong with the children, and did they find this one connection and then tell us that story? It’s very difficult to find out what you’re not being told, but it is absolutely crucial because you cannot judge a claim when it’s based on highly selected evidence.

How to be trustworthy about data

To go back to Onora O’Neill, she’s got these wonderful criteria for what she calls intelligent openness or intelligent transparency. Anybody communicating, if they want to be trustworthy about data, should be adhering to this. It’s also a checklist for anyone receiving a claim. The first thing is that the data should be accessible. We’ve got to be able to find the information. It can’t be buried in some big PDF at the bottom of some spreadsheet. That is not transparency. It’s got to be accessible to people. Then, it’s got to be comprehensible. You’ve got to be able to understand it. For example, I’ve never been able to understand anything about my pension; it’s just not clearly communicated. You need to be able to test whether people understand it if you’re going to be trustworthy.

Next, it’s got to be usable. I love this phrase. Being usable means that it’s got to actually address the issues that the person is interested in, not what the communicator is interested in. It’s what the audience is interested in that you have to address. Now, the final criteria is very important: it has to be assessable. Somebody, if they want to, should be able to check your work. That may be a very informed citizen. It may be a fact-checking organisation. It may be a pressure group. It may be a journalist or anybody else. But somebody needs to be able to go back and say, why are you claiming this? Where did this number come from? What does it mean? You have to provide the means for them to do that.

Tactics to fight polarisation

Photo by Heidi besen

It’s very unfortunate that what is science has become quite politicised, especially in the US, but increasingly elsewhere as well. Whether it’s climate, vaccines or coronavirus, you’re seeing this split in terms of tribes of people who believe different things, as if it’s a matter of faith rather than a matter of knowledge. This is deeply unfortunate, but it’s difficult to counter.

The usual claim was, we need to inform people. We need better understanding. That’s not actually the solution; it’s been shown that people who are rather knowledgeable about these things can be just as split, so essentially it has much more to do with tribal identity. There may be groups of people who are impossible to shift. They are just embedded. But there is quite a movable centre ground.

What are the tactics to try to prevent the bulk of people in the middle moving to these polarised views? How can one counter that? Education is part of it. There’s a role in schools, not to teach what is right about climate, vaccines or evolution, although that is important, but to teach the scientific method. Science is not just a body of facts; it’s in perpetual argument and development. There are always disagreements, and just because scientists disagree doesn’t mean that you can dismiss them all.

Pre-bunking misinformation and fake news

There have been claims that just correcting people doesn’t work – this idea of a backfire effect that if you correct people, they’ll only believe it more. But that’s not actually very reproducible science. The people who developed that idea have really backtracked on its generalisability. So the first thing to know is that correcting people can be useful. Putting people right can be useful. But I like the work that’s being done, for example, by my colleague Sander van der Linden in Cambridge, which says that debunking fake news or misinformation is too late. It’s already embedded. You’re on the back foot; you’re reacting.

You’ve got to “pre-bunk” it. You’ve got to inoculate people against it. You’ve got to be the first one to tell them the misinformation. You give them a little dose of it: do you know there are people going around saying that vaccines are a conspiracy by Bill Gates to put a microchip inside all of us and control us? Do you know that people are saying that coronavirus is being spread by 5G networks? You tell people and say, actually, this isn’t true, and this is why it isn’t true. In doing so, you actually inoculate them against this news, which they will then hear and say, I know this claim. This is because when people are surprised and say, wow, that’s interesting, then they focus on that information. So you’ve got to be the one to give the surprise, to hit them with the story first.

Discover More About

data-based claims and trust

van der Bles, A. M., van der Linden, S., Freeman, A. L. J., et al. (2020). The effects of communicating uncertainty on public trust in facts and numbers. Proceedings of the National Academy of Sciences, 117(14), 7672–7683.

Freeman, A. L. J., & Spiegelhalter, D. J. (2018). Communicating health risks in science publications: time for everyone to take responsibility. BMC Medicine, 16, Article 207.