3 ways opinion polls (deliberately) get it wrong—and what you can do about it

roger scime | content creation | scribe site | polls

"Er . . . ?

It may be a little early for this, but probably not. As the political field begins to form for the 2012 general elections, we can expect to see more and more polling results released favoring one candidate or another for any particular office. And, it will continue to get worse in the weeks and months leading up to Election Day.

With that in mind, I think a short examination of how public-opinion polls can be manipulated to reflect whatever results the organization paying for the poll wants it to.

Over the years, three polling tactics have been proven to be particularly effective by providing misleading results; and, while the first of these could be attributed to sloppy preparation, the second and third can justifiably be considered deliberately deceptive.

The Unrepresentative Sample

The Loaded Question

The Push Poll

Let’s take them in order:

First, the Unrepresentative Sample

An unrepresentative sample is one that does not accurately reflect the population one wishes to survey. As implied by the illustration above, the man answering the door isn’t your typical, er, normal human being. So, if you wanted to know what normal humans (the ones who do not wear bird cages on their heads, for example) thought about something, this fellow would not be the one to ask.

The real-life example that is cited most often is a poll commissioned in 1936 by the Literary Digest regarding the presidential race between Franklin Roosevelt and Alf Landon. The sample names were selected from telephone directories and auto-registration lists. The results indicated that Landon would win easily, and events of the Landon presidency are discussed in civics classes to this day.

Huh?

As we all know, Alf Landon actually lost to Roosevelt in a landslide, which is why this particular instance of polling error is so well known. Remember where the sample came from? Telephone directories and auto-registration lists. Well, in 1936 few American had either, and those who did were mainly upper-class. But the less-well-off could vote, too, and there were many, many more of them. Literary Digest‘s sample was unrepresentative of the population, and the election results proved it.

Continue reading

Agnotology: Ignorance by design

roger scime | scribe site | content creator | agnotology

Doctors & Cigarettes: A 20th Century Love Story

I learned a new word the other day. I was reading a story on Salon.com the other day, when I came across a word with which I was unfamiliar: agnotology, defined by WordSpy as n. The study of culturally-induced ignorance or doubt, particularly the publication of inaccurate or misleading scientific data.

As one who makes his living from content creation, I was intrigued: culturally-induced ignorance or doubt? Strong stuff!

The authors of the article, Bill Moyers and Michael Winship provided some examples:

  • Believing that global climate change is a myth
  • The insistence by the tobacco industry that harm caused by smoking is still in dispute.
  • The conviction that Pres. Barack Obama is a closet Muslim, who was born in Kenya.

To which I was able to add an example of my own:

  • The 9/11 hijackers were all from  Iraq.

Agnotology: origin of the term

The word agnotology was coined a few years ago by Stanford researchers Robert Proctor and Londa Schiebinger, who have since written a book on the subject,
Agnotology: The Making and Unmaking of Ignorance.

Actually, though, there is nothing new in the use of agnotology in the furthering of some political or social agenda:

Continue reading

Can logic save journalism? (a LinkedIn conversation)

cross talk | roger scime | critical journalism blog

A conversation between Alexander Rheeney & the Author

I recently posed that (admittedly pretentious) question to the Journalist and Journalism Group on LinkedIn, and yesterday Alexander Rheeney, who describes himself as a self-employed writer and editor in Papua, New Guinea, posted a comment.

Because the exchange encapsulates one of the fundamental purposes of this blog, I am posting the original question, Mr. Rheeney’s comment, and my response.

NOTE: I’ve edited it a bit for clarity and [my] typographical blunders:

March 3, 2011 (my original post):

I’m exploring applying the techniques of informal logic (aka critical thinking) to journalism research, analysis, writing, and reporting.

I believe that restoring rigorous credibility to our stories will help burnish our tarnished image (clichés, anyone?). On the face of it, I’d like some opinions. Anyone?

March 4, 2011 (Mr. Rheeney):

Roger, interesting indeed. For journalists I think colleagues will be torn between
‘objective writing vs subjective writing (critical analysis)’. But hey I sometimes don’t mind stepping outside the box and looking at issues from a different perspective :-o

March 5, 2011 (me):

Thanks for the comment, Alexander.

I take your point, and it’s a good one. It also  prompts me to ask: “Is it time to revisit the ‘objective vs. subjective writing’” argument?

With the increasing use of social media in reporting, are gatekeepers becoming irrelevant? And, if so, what does that do to objectivity?

And, either way, does that change the *perception* of objectivity? Or, is the question itself even relevant?

My very humble opinion is that: 1) careful and critical examination of sources for credibility; 2) context and background be provided when necessary and appropriate; and , 3) that care and thought be given to stories, so that all of the points in a piece lead to a clear and unambiguous conclusion based on the facts of what has preceded it.

Most of my opinions regarding this topic have been informed by the writings of Neil Postman and 5 semesters teaching Reasoning & Critical Thinking to community college students at Southern Nevada College, [Note: It's actually the College of Southern Nevada. My bad.] so I’m certainly not an objective observer:

I firmly believe that applying the techniques and mindset of informal logic to journalistic research, analysis, writing, and reporting—what I’ve taken to calling “Critical Journalism” (shameless plug alert: see my blog at www.rogerscime.com)—will help us provide the public what Carl Bernstein calls, “the best obtainable version of the truth,” and thus restore our craft to the public respect it deserves. I know that was an awfully awkward run-on sentence but—hey—this is only social media . . . right? 8^)

BTW, unless you have any objections, I’m going to post this thread on my blog. Er . . . did I mention it’s at www.rogerscime.com?

I’ve posted this exchange as an example of how individuals can often see things differently—and how both can easily be right: Mr. Rheeney saw my question as asking about subjective journalism vs. objective journalism, while my point was more directly aimed at the informal-logic side of the paradigm; however, objectivity-vs-subjectivity will play a significant role in upcoming entries in this blog, something I had—admittedly—back-benched as a topic.

I thank Mr. Rheeney for reminding me of its importance to Critical Journalism.

Comments on this post are welcome.