Internets Polls and other Cons
The book I’m currently reading has a whole chapter on opinion polls. Specifically, it focuses on how systematic errors in the polls can cause error bars so broad that the data is completely worthless. Unless your goal in the first place isn’t to measure public opinion but to shape it, then they aren’t worthless at all.Yesterday on my facebook feeds I got a request to answer a survey about where I get my news. Sounds good on the surface, but then the problems start popping up.
What’s wrong with this kind of a survey? Well first off, it’s voluntary. They aren’t gonna get any mediocre opinions. People don’t log on to a volunteer survey to say that they really don’t have an opinion. So right off the bat the survey will be artificially polarized, since it will only take responses from people passionate enough to participate.
Second, I didn’t see this same survey come across any other media; radio, TV etc. This isn’t a problem by itself. They may have been specifically looking for the opinions of facebook users. It’s only a problem if they then try to extrapolate from there out to the general population. Many surveys often do exactly that.
But the big death nail in this survey’s credibility is the surveyed audience. This came across my NPR feed. Yup, this survey was only sent out to people who are already self declared fans of NPR. Are you kidding me? You’re taking a survey of people who are already fans of NPR and want to know where they get their news? Gee, I wonder how that will turn out.
Of course this is nothing new. Fox news can’t seem to go a whole hour without asking you to log in and tell them what you think. Then they come back with some ridiculous misinterpretation of the data like, “55% of Americans think Obama is Muslim.” As if the opinions of their viewers makes it reality. I’ve grown to expect this kind of meaningless polling from most news outlets. I was just a little bit surprised the see if from NPR. In fairness to them, I don’t think they were being partisan. They were just trying to create a poll that disproportionately favored NPR itself.
So If you’re ever around me when somebody tells me about a recent poll, you’re liable to hear sigh or a snicker and then a series of follow up questions about things like statistical errors v systematic errors, controlling for sample bias, error bars, etc. You see polls themselves aren’t news. At best, they are what news organizations talk about while they are waiting for real news to happen. At worst they are an attempt to manipulate opinion or politics.
Internets Polls and other Cons
The book I’m currently reading has a whole chapter on opinion polls. Specifically, it focuses on how systematic errors in the polls can cause error bars so broad that the data is completely worthless. Unless your goal in the first place isn’t to measure public opinion but to shape it, then they aren’t worthless at all.Yesterday on my facebook feeds I got a request to answer a survey about where I get my news. Sounds good on the surface, but then the problems start popping up.
What’s wrong with this kind of a survey? Well first off, it’s voluntary. They aren’t gonna get any mediocre opinions. People don’t log on to a volunteer survey to say that they really don’t have an opinion. So right off the bat the survey will be artificially polarized, since it will only take responses from people passionate enough to participate.
Second, I didn’t see this same survey come across any other media; radio, TV etc. This isn’t a problem by itself. They may have been specifically looking for the opinions of facebook users. It’s only a problem if they then try to extrapolate from there out to the general population. Many surveys often do exactly that.
But the big death nail in this survey’s credibility is the surveyed audience. This came across my NPR feed. Yup, this survey was only sent out to people who are already self declared fans of NPR. Are you kidding me? You’re taking a survey of people who are already fans of NPR and want to know where they get their news? Gee, I wonder how that will turn out.
Of course this is nothing new. Fox news can’t seem to go a whole hour without asking you to log in and tell them what you think. Then they come back with some ridiculous misinterpretation of the data like, “55% of Americans think Obama is Muslim.” As if the opinions of their viewers makes it reality. I’ve grown to expect this kind of meaningless polling from most news outlets. I was just a little bit surprised the see if from NPR. In fairness to them, I don’t think they were being partisan. They were just trying to create a poll that disproportionately favored NPR itself.
So If you’re ever around me when somebody tells me about a recent poll, you’re liable to hear sigh or a snicker and then a series of follow up questions about things like statistical errors v systematic errors, controlling for sample bias, error bars, etc. You see polls themselves aren’t news. At best, they are what news organizations talk about while they are waiting for real news to happen. At worst they are an attempt to manipulate opinion or politics.
Merchants of Doubt
So the other day I was trying to convince one of my kids to stop playing video games and get studying one of the school subjects in which they aren’t doing very well. No response. So I looked at their grades so far and did my best to persuade them that if they didn’t do a remarkable job in the last few weeks of the semester that they likely would not pass the course. Again, no response. At this point I was getting more than a little irritated at the lack of action. “Why aren’t you doing anything about this?” The response I got, “You don’t know for sure that I’ll fail if I don’t study today and you can’t guarantee that I’ll pass if I do study right now.” This little logical fallacy is one that has bugged me for years. While it is true that I could not know for sure the outcome of doing homework it’s ridiculous to argue that playing video games is a better use of that time.We see this type of fallacious reasoning all the time. Sometimes it is accidental. I know people who avoid the interstates because don’t know if there will be any construction work going on and they can’t be 100% sure that the off ramps will be open. Other people turn off all passenger side airbags because they can’t be 100% sure that theirs won’t be the one that goes off accidentally. I have even heard of one friend of mine who never wears his seat belt because he can’t be 100% sure that he won’t drive off a bridge and drown because he can’t get out of his car. All of these situations are based on an emotional response to something that had happened to them or a persuasive story they heard or saw on the news. Despite ample evidence to the contrary they still stick to that emotional assessment of risk and a desire for 100% surety.
What really bothers me is when others recognize this fact that you can never be 100% sure and exploit it for political and personal gains. Merchants of Doubt is the history of just such political exploitation of science and the public’s misunderstanding of certainty, statistics and risk.
Industry funded scientists focused on and magnified the uncertainty when dealing with the link between cancer and cigarette smoking. The implication being that since they can’t prove 100% that smoking is what gave this guy cancer then we don’t know what did. And therefore smoking is safe. Later on Industry funded scientists focused on and magnified the uncertainty when dealing with the consequences of the arms race. Then after that it was the link between industries and acid rain. Etc, etc, etc. Time and time again Industry funded experts have used the same tired script to justify their in action. What I found most surprising in this book is that time and time again it is the very same scientists pushing this uncertainty on the public, even when the topics are far afield of their area of training and expertise.
I took this book as a warnign to be skeptical anytime somebody encourages action or inaction just based on the fact that we can't be 100% certian. Do the research and weigh the risks. Sure, absolute certianty is rare, but relative certianty is much more common. I many not no for sure if I'm gonna get driven off a bridge, but I'm far more likely to get into an accident that does not involve a bridge and so I'm gonna continue to wear my seat belt. And the same goes for the other controvertial issues detailed in Merchants of Doubt.
Fark
So shortly after I got back from lunch I decided to check a couple of news. I’ve made it no secret that I’ve been very disillusioned with the crap that masquerades around as news lately. But I still feel some pathological need to check in with the big sites periodically just to see if I’m missing something important. Well what do I find today at 12:15pm? CNN.com has a front page, biggest font and a picture and story with video of the fashion accuracy behind the series “Mad Men”. MSNBC.com has an article about the 10 day old non-news event about Falcon Heene AKA“balloon boy”. And FOXNEWS.com has a story about another hoax, the Latvian meteorite.This is just sad. Did nothing important happen today? Or have they just completely forgotten what classifies as news? The top stories on three of the biggest news sites today had nothing to do with news at all. Who cares about any of these events?
This first story, “Mad Men” fashion accuracy, are you serious CNN? Who cares? Shouldn’t reporting of this calibre be reserved to a fan site on AMC.com? How in any measurable way will the accuracy of inaccuracy of the suits these actors wear affect my life? It won’t.
Next we have two hoaxes that wouldn’t have existed at all if the media didn’t have a predisposition to air anything at all without checking the facts first. Even the local authorities played the media to help gain the trust of the Heene family and get them to slip up and admit the story was a hoax. What does that say about the condition of the media in this country when law enforcement can bank on the fact that they won’t try to follow up on the story and do any kind of accuracy check at all before running the story? Even a cursory check with anyone who had taken 8th grade physics would be able to tell a critical thinking reporter that there was no way in the world a balloon that small could have lifted itself and 37 pound Falcon Heene. But who cares right? They pay good money for that news helicopter so let’s air the stupid footage without any kind of critical review. And 10 days later we can still talk about it as if something new has happened even though it really hasn’t. Give it a rest. The sooner the Heene family falls back into obscurity the better.
I haven’t read much on the whole Latvian Meteor yet. The one picture I saw was obviously created with buried explosives and not the relatively slow moving mass of a meteor. I’ll wait for the scientist to debunk this story completely. Until then I’m sure all the media is perfectly content to continue giving the attention that was planned from the beginning to this complete non-event.
I’ve just finished reading It’s Not News it’s Fark is written by the creator of the website fark.com. The author, Drew Curtis has spent a decade running a website that makes fun of the crap that we continue to call news. The book is a riot. It’s irreverent and frequently potty-mouthed, but always right on the mark. Each chapter is dedicated to one of the many ways that the media puts crap in print, online or on the air.To hear conservative talk show host lambaste the modern media you’d think that they were controlled by the some liberal conspiracy organization. I’ve been personally analysing the news for several years looking for the liberal bias that is so frequently trumpeted. The only way you could get a liberal bias out of the crap that gets aired is if you define anything that isn’t conservative bias as a liberal bias. But in fact much of what make headlines is neither. It doesn’t even deserve to be called news.
Who cares about John and Kate? It’s only news to about 12 people on the whole planet. It’s just entertainment to some of the rest of us. And most of the country couldn’t’ care less. The fashion accuracy of “Mad Men”? Puhleeze. How about a story about the math accuracy of the latest spending bill? That would be news that really affects me.
My only criticism of the book as that I wish he would have dedicated a whole chapter to the media’s impotence. Michael Eisner has gone on record that he didn’t think it would be appropriate for ABC news to report on any of Disney’s business dealings. He doesn’t mind, however, an ABC news report about the technology behind the latest Disney movie. So self reporting is okay if it’s positive. You just can’t bite the hand that feeds you. Well considering the size and depth of most of the media conglomerates today playing by Eisner’s rules it becomes very hard to say anything at all. So what are you left with? A news media that is pretty impotent.
I’d really recommend reading It’s Not News It’s Fark. Don’t dismiss it as satire. His critique is right on the mark.
