A Field Guide to Lies: Critical Thinking in the Information Age by Daniel J. Levitin Page A

of tampering with their footballs, deflating them to make them easier to catch. Theyclaimed measurement error as part of their defense. Inflation pressures for the footballs of both teams that day, the Pats and the Indianapolis Colts, were taken after halftime. The Pats’ balls were tested first, followed by the Colts’. The Colts’ balls would have been in a warm locker room or office longer, giving them more time to warm up and thus increase pressure. A federal district court accepted this, and other testimony, and ruled there was insufficient evidence of tampering.
    Measurement error also occurs when the instrument you’re using to measure—the scale, ruler, questionnaire, or test—doesn’t actually measure what you intended it to measure. Using a yardstick to measure the width of a human hair, or using a questionnaire about depression when what you’re really studying is motivation (they may be related but are not identical), can create this sort of error. Tallying which candidates people support financially is not the same as knowing how they’ll vote; many people give contributions to several candidates in the same race.
    Much ink has been spilled over tests or surveys that purport toshow one thing but show another. The IQ test is among the most misinterpreted tests around. It is used to assess people’s intelligence, as if intelligence were a single quantity, which it is not—it manifests itself in different forms, such as spatial intelligence, artistic intelligence, mathematical intelligence, and so forth. And IQ tests are known to be biased toward middle-class white people. What we usually want to know when we look at IQ test results is how suitable a person is for a particular school program or job. IQ tests can predict performance in these situations, but probably not because the person with a high IQ score is necessarily more intelligent, but because that person has a history of other advantages (economic, social) that show up in an IQ test.
    If the statistic you encounter is based on a survey, try to find out what questions were asked and if these seem reasonable and unbiased to you. For any statistic, try to find out how the subject under study was measured, and if the people who collected the data were skilled in such measurements.
    Definitions
    How something is defined or categorized can make a big difference in the statistic you end up with. This problem arises in the natural sciences, such as in trying to grade cancer cells or describe rainfall, and in the social sciences, such as when asking people about their opinions or experiences.
    Did it rain today in the greater St. Louis area? That depends on how you define rain. If only one drop fell to the ground in the 8,846 square miles that comprise “greater St. Louis” (according to the U.S. Office of Management and Budget), do we say it rained? How manydrops have to fall over how large an area and over how long a period of time before we categorize the day as one with rainfall?
    The U.S. Bureau of Labor Statistics has two different ways of measuring inflation based on two different definitions. The Personal Consumption Expenditures (PCE) and the Consumer Price Index (CPI) can yield different numbers. If you’re comparing two years or two regions of the country, of course you need to ensure that you’re using the same index each time. If you simply want to make a case about how inflation rose or fell recently, the unscrupulous statistic user would pick whichever of the two made the most impact, rather than choosing the one that is most appropriate, based on an understanding of their differences.
    Or what does it mean to be homeless? Is it someone who is sleeping on the sidewalk or in a car? They may have a home and are not able or choose not to go there. What about a woman living on a friend’s couch because she lost her apartment? Or a family who has sold their house and is staying in a hotel for a couple of weeks while they wait for their new