Polls and surveys play a key role in business, politics, and the media. However, they are often misunderstood. This worksheet will help you interpret polls and surveys in a nuanced way following correct logical principles.

Complete numbers 1 to 6 and submit your work as a Word document. Before starting, review the Lecture Video - Inductive Arguments, Lecture Video – Polls and Surveys, and Handout - Assessing Statistical Generalizations.

COMPREHENSION

  1. Why is it important for statistical generalizations such as polls and surveys to have an adequate sample size and a non-biased sample? What is random sampling and how does that strengthen a statistical generalization? Answer in 6-8 sentences with two direct quotes from the text by Van Cleave. Quotes go in quotation marks with the in-text citation (Van Cleave, 2016, p. ___). (15 points)
  2. What does "margin of error" mean as applied to a poll or survey? What does "confidence level" mean? If a survey says that 65% of Americans enjoy biking with a margin of error of plus or minus 3% and a confidence level of 95%, what does this survey tell us? (Hint: These concepts are not fully discussed in the readings but are present in the video lecture.) (15 points)

DESCRIBING POLL OR SURVEY RESULTS

Find two articles or websites that present poll or survey results related to your research paper topic. These can be newspaper reports, peer-reviewed articles, polling organization reports (Pew Research (Links to an external site.) and Gallup (Links to an external site.) have a lot of material), or other types of sources. Choose sources with enough context and discussion of a poll or survey to let you answer questions 3 to 6 in a detailed way.

  1. Describe the first poll or survey results, linking to your source. Discuss the population being studied, the sample size, what researchers concluded about the group, how participants were selected for the survey (random or non-random), the margin of error, and how survey questions were phrased. If one of these details is not available in your source, that is OK, but if two or three are missing, switch to a different source with more information about a poll or survey. (15 points)
  2. Describe results from a second poll or survey outlined in a different source, linking to the source. Discuss the same survey elements described in the previous question. (15 points)

EVALUATING POLLS AND SURVEYS

  1. Assess the poll or survey described in question 3. Overall, would you say it is a good and reliable statistical generalization about the population considered? As part of your evaluation, touch on the survey’s sample size, whether the sample is biased or non-biased (as these terms are used in the lesson), the margin of error, and whether the survey questions are appropriately phrased. (20 points)
  2. Assess the poll or survey described in question 4. Overall, would you say it is a good and reliable statistical generalization about the population considered? Discuss the same survey elements described in the previous question. (20 points)

EXAMPLE SOLUTION

Describe poll or survey results:

The Yale Climate Opinion Map (Links to an external site.) presents information about Americans' views on climate change. It shows that, as of 2019, 67% of Americans believe climate change is happening and 53% believe this is primarily caused by human activity. It also has state and city level data and survey results about Americans' views on the risks of climate change and different proposed climate policies.

The population being studied is Americans and the sample size for the survey is very large (over 24,000 people were surveyed). While random sampling is not described on the survey methodology page, the survey administrators describe their efforts to achieve a non-biased sample by seeking responses from across the country and comparing them to other existing study results. At a 95% confidence level, the margin of error for the national results is +/- 3%, and for the state and local results is between +/- 7% and 8%. The questions asked appear to be straightforward with results reported for prompts such as "is global warming happening?" and "is global warming caused mostly by human activities?"

Evaluate the poll or survey results:

This appears to be a thorough and reliable poll. The sample size is larger than that of many national studies and several statistical methods are described that helped the researchers achieve a non-biased sample. The margin of error of +/- 3% is relatively small, but people using the survey results should be aware of the large margin of error (as high as +/- 8%) for state and local results. This means that, if a state has 54% of people believing that global warming is mostly caused by humans, the actual value for the population could be anywhere between 46% and 62%. If someone needs precise information about climate views in a specific state or city, they may want to look for another poll or survey with better fine-grained data. The survey questions seem to be presented in a clear, fair, and unbiased way. It is interesting that they refer to "global warming" (an older term) rather than "climate change." This could be an area for improvement in the poll, but the use of an older term may also have value if it allows for more consistent comparison with similar surveys from past decades.