Subject guides: Data & Statistics guide at the University of Manchester: Understanding statistics

Can you trust statistics?

“Facts are stubborn things, but statistics are pliable.” ― Mark Twain

Statistics help us to understand complex situations and analyse events, but they can also be used to misrepresent and distort our picture of the world. This may be done deliberately, through the selective use of data or misleading language, but it can often be the unintentional result of flawed information gathering and interpretation, leading to unsupported conclusions or oversimplified analysis.

Most of us are not mathematicians and we can struggle to understand the context and meaning of statistics, probabilities and risks. However, there are several useful and reliable guides that can help you take a critical approach to reading statistics and avoid some of the common pitfalls and errors.

Evaluating statistics

Statistics cover methods and procedures for collecting, managing, analysing and consequently presenting quantitative data.

Statistics can be a very important asset when compiling information and supporting your work.

Think about the following questions when you are deciding if a source of statistical data would be useful for your studies.

Authority - Who is the author? Is it a commercial document? Is it from the government? What are the qualifications/authority of the producer of the information?
Date - What is the date range of the data, and is it supposed to be historical or current?
Purpose - Where has the data been published? What type of publication and who is the information for? Can you detect any bias? Are they trying to sell something or emphasise a particular outcome?
Content - Is the data accurate and how can it be verified? Does there appear to be bias?
Coverage - Is the coverage complete? What is the population, or sample size, of the study? How did the researchers select the sample? Was it a random selection or a specific group? Have they chosen a selection of the data to prove their viewpoint?
Presentation - Has the data been repackaged? For example, government data published by a private source might not be as complete as the original study.
Data Source - Is the data from a primary source? If the data is from a secondary source (such as the Statistical Abstract), has it been properly documented so you can find the primary source?

Other University Guides on Statistics

Statistics

Statistical Gateways

RBA Statistics – statistics guide includes many useful links to statistical sources

Evaluation guidelines

There are several ways in which you may want to utilise statistics and data..Evaluating is an important part of this process.

Full Fact

Full Fact is an independent fact-checking organisation that provides information, advice and tools to help anyone check the evidence for claims made by politicians and the media.

The House of Commons Statistical Literacy Guide

Section 4.2 of the guide is entitled “How to spot spin and inappropriate use of statistics” and suggests three questions to ask when looking at statistics:

Compared to what?
Since when?
Says who?

NHS Choices: Behind the headlines

Some of the biggest misuses of statistics are concerned with health scares and medical studies. Ben Goldacre’s book 'Bad Science' challenged many of these falsehoods and the NHS ‘Behind the headlines’ website performs a similar role, looking at the facts and science behind the latest health stories.

Sense about Science and Straight Statistics: making sense of statistics

The Sense about Science guide aims to provide users of statistics with “a source of questions you can ask and pitfalls to avoid”. It features advice from statisticians, journalists and scientists on how to make sense of statistics and how to identify when statistics are being misused, whether by accident or design. It also has an excellent list of recommended further reading.

UK Statistics Authority

Official statistics in the UK are monitored by the UK Statistics Authority, an independent body that is “required to promote and safeguard the quality and comprehensiveness of official statistics, and ensure good practice in relation to official statistics.”