When Evidence Isn’t Reliable

How do we know what we know? Who can we trust?

It’s bad enough that an American President constantly attacks reputable sources of information; when Trump asserts that reports unflattering to him are “fake news,” those assertions join–and bolster– widely-held doubts about the reliability of contemporary media. Those doubts are understandable; it is increasingly difficult to separate out the conspiracy-theory websites from legitimate digital newcomers, to recognize and discount sources trafficking in spin and outright propaganda, and even to distinguish between objective reporting and satire.

The unremitting assault on fact, on objective reality, makes the reliability of the information we get from government agencies more important than ever. When Scott Pruitt scrubs accurate science from the EPA website, he does more than degrade our efforts to protect the environment–he adds to the Alice-In-Wonderland nature of our shared reality.

Unfortunately, it isn’t just Scott Pruitt. The problem isn’t even limited to the Trump cabinet.

According to the Guardian (a very reputable source)

Over half of all police killings in 2015 were wrongly classified as not having been the result of interactions with officers, a new Harvard study based on Guardian data has found.

The finding is just the latest to show government databases seriously undercounting the number of people killed by police.

“Right now the data quality is bad and unacceptable,” said lead researcher Justin Feldman. “To effectively address the problem of law enforcement-related deaths, the public needs better data about who is being killed, where, and under what circumstances.”

This article underscores the importance of good journalism–the Harvard study used data compiled in the Guardian’s investigative reporting. It also illustrates the consequences of relying upon bad data.

Feldman used data from the Guardian’s 2015 investigation into police killings, The Counted, and compared it with data from the National Vital Statistics System (NVSS). That dataset, which is kept by the Centers for Disease Control and Prevention (CDC), was found to have misclassified 55.2% of all police killings, with the errors occurring disproportionately in low-income jurisdictions.

“As with any public health outcome or exposure, the only way to understand the magnitude of the problem, and whether it is getting better or worse, requires that data be uniformly, validly, and reliably obtained throughout the US,” said Nancy Krieger, professor of social epidemiology at Harvard’s Chan School of Public Health and senior author of the study. “Our results show our country is falling short of accurately monitoring deaths due to law enforcement and work is needed to remedy this problem.”

Interestingly, the researchers found that the accuracy of the data varied wildly by state, “with just 17.6% misclassification in Washington, but a startling 100% in Oklahoma.”

In 2015 the Guardian launched The Counted, an interactive, crowdsourced database attempting to track police killings throughout the US. The project was intended to help remedy the lack of reliable data on police killings, a lack that became especially visible after the 2014 unrest in Ferguson put policing in the national spotlight.

Other federal databases, including the Bureau of Justice Statistics’ (BJS) arrest-related death count and the FBI’s supplementary homicide reports were similarly criticised for severely undercounting police-related deaths. Both programs have been dramatically reworked since The Counted and similar media/open source databases forced officials such as the former FBI director James Comey to admit that newspapers had more accurate data than the government on police violence.

To state the obvious, policymakers cannot craft good laws or regulations in the absence of sound data. Citizens confronted with experiences at odds with government’s descriptions lose confidence in that government. Discrepancies between reality and government reporting feed conspiracy theories.

When we don’t know what we know, we cannot act.

Other than patronizing news sites we know to be trustworthy, there’s not much we can do about the proliferating media wannabes spouting fantasies and disinformation. But we should be able to insist that government agencies charged with compiling and disseminating factual data do so accurately. We aren’t likely to get that done in the Age of Trumpian Fantasy, but when the time comes to clean up the incredible chaos he is creating, a commitment to accurate data collection by government should be high on our cleanup list.


  1. I think our government also prohibits research into gun violence
    and Pot research –
    and Climate Science
    This desire to move toward ignorance is stunning.

  2. What Patmcc says. As a nation, we just got so we value ignorance as the highest good. No society can do this and survive.

  3. If it were just the government withholding facts and information that is inconvenient to the powers in office that would be one thing. Of more concern to me is the deliberate broadcasting of lies by the media, by “think tanks”, by political parties, by industrial organizations, by professional organizations and the like.

    The daily attack on truth and facts has become the norm. We no longer trust each other… and with good cause.

  4. The problem will only get worse, as Trump’s handlers fill the executive agencies with trolls from the fossil fuel and chemical industries. It will take a generation to clean out the new Augean stables they are creating.

  5. Copied and pasted from Wikipedia:
    “The Guardian is a British daily newspaper, known from 1821 until 1959 as the Manchester Guardian. Along with its sister papers The Observer and the Guardian Weekly, The Guardian is part of the Guardian Media Group, owned by the Scott Trust. The Trust was created in 1936 “to secure the financial and editorial independence of the Guardian in perpetuity and to safeguard the journalistic freedom and liberal values of the Guardian free from commercial or political interference.”

    Copied and pasted from Sheila’s blog:
    “According to the Guardian (a very reputable source)”

    “When Evidence Isn’t Reliable” and we must daily go to a British publication from the UK for the truth about what is going on in the United States of America; maybe it is time to accept Queen Elizabeth’s offer to “take us back”…unless that was more unreliable, fake news from our own journalists. We have no “trust” set up to protect us from our own media.

    Copied and pasted from Theresa Bowers’ comments:
    “The daily attack on truth and facts has become the norm. We no longer trust each other… and with good cause.”

    We must constantly separate the wheat from the chaff; our available printed news sources are fast becoming a monopoly owned by Gannett, Inc., and is best used to wrap garbage…in the garbage it provides us as “news”.

    The Trump administration is withholding facts (think of the Niger attack and the deaths of four Americans) and pit that against Trump now wanting to make public thousands of classified documents pertaining to the Kennedy assassination…for what purpose and to what end? We need and deserve the full set of facts regarding the ill-fated Niger attack which was covered up. We also need and deserve the full set of facts regarding the legality of Trump paying for legal services for his staff members who will be questioned in the the investigation into his ILLEGAL connections to Russia. I see that as “hush money” if accepted and an open threat to his staff if they decline his “offer”. Will any “evidence” provided by this administration be “reliable” or will we need another special council to begin another investigation?

    “Who Do You Trust?” was the name of a long defunct TV program; it now describes our view of the United States government.

  6. This from Chris Hedges, today, “Our Ever-Deadlier Police State.” Police forces in thr U.S. have fatally shot 782 people this year…by comparison, police in England and Wales killed 62 in the 27 years between the start of 1990 and the end of 2016. Read his column at “truthdig”.

  7. This discrepancy in the government data and statistics raises the possibility that there is a “puppet master” controlling such data and information and the people responsible for collecting it. It also illustrates that there is no longer effective accountability for non-elected government employees or the unethical/criminal elected officials that currently inhabit the three branches of the federal government and state legislatures everywhere. The agencies are free to make the data fit their version of reality and support the ideology of the political parties.

  8. Well done. The Guardian is a magnificent paper. I follow Breitbart on Facebook. It reminds me of the old supermarket tabloids. It’s followers are delusional. I saw one commenter refer to 45 as “Supreme Leader and President” and she wasn’t joking or being sarcastic. They have 3.7 million followers. We need to make these people the nutty fringe again, and “build a wall” between fact-based journalism and whatever it is they’re peddling.

  9. The best thing we can do is not take anything at face value. Being retired, it’s easier for me to spend time checking out what I’ve read. Being a natural skeptic, it’s easier for me to avoid falling for fake news. Being older, it’s easier for me to sit back and take a deep breath before jumping into controversy. Even if you are not any of those three things, it’s still wise to do those three things.

  10. Peggy; prior to the “Age of Trump”, beginning in 2015, it was interesting and educational to do research on news items and on those in the news. Almost always it was to learn more or learn the source or history of news. Now; we must do research to validate – or HOPE to validate – the truth of what we receive as news.

  11. Jo Ann,

    Part of the problem finds it origin in social media where no one will take responsibility for what is posted, and the providers of that media (who demand openness and accountability from government) refuse to provide the public with information about the sources from which they profit.

  12. So far as I am concerned, evidence that is not reliable is not evidence. It’s indirect propaganda designed to put numbers on ideology. Think NRA and its tentacles that reach into the darkest of the caverns of government statistic-keeping. Conspiracy theory? Hardly, but even so there is no conspiracy theory that can possibly be denied when year to year numbers as unmasked by The Guardian tell us that the so-called “evidence” is evidence. Besides, the mere fact that we have evidence (if it is) doesn’t mean that the evidence is reliable. The stark difference in British deaths by gunplay and the American deaths by gunplay tells us all we need to know; that we are awash in a sea of guns, many of which are in the hands of such as those in the Orlando and Las Vegas massacres.

    Police coverups on the killing of civilians are routine, as we all know in our gut, whatever the “evidence” proffered. What we need is an independent panel of experts who investigate each such killing by police and make their findings public, not some addition of tanks and military weaponry to the arsenal of the police. To their rejoinder that they need such weapons to match their criminal adversaries, the answer is that we are going to severely limit their adversaries’ NRA-approved arsenals by outlawing automatic weaponry for sale to anyone but the military. We need not live either in a police state or a Tombstone environment. There are places on the spectrum between these extremes where good policy can lead us, and we need to elect policy-makers who will defy the NRA and its coffers and propaganda and lead us there. With such reforms, perhaps reliable evidence will become available to us, evidence that shows a drastic reduction in homicides both by and against civilians and the police, accompanied by a rise in public confidence in government.

  13. The Guardian article points out that police killings were “misclassified” which means the police departments had help in covering up or misclassifying the death of a community member.

    In my small community, the police department has a known element of corruption as does our city hall and the court system.

    Therefore, how does Sheila plan on deciphering what our government releases as “propaganda” versus “fact”.

    Good luck with that.

    In watching the PBS series on the Vietnam War, one thing was evident by each POTUS – they lied to the people. They still use the term, “national security” to convince the media to ignore certain unflattering incidences when it’s really, “political interests”.

    The reason we have “fringe and alternative media” today is that our legacy and establishment media notoriously lie to us on behalf of the government and corporate Oligarchs.

    (See Noam Chomsky’s, “Media Propaganda Model”)

    As a media critic, I also know that most Americans create their own echo chamber with media that support their own points of view. Our fragile egos don’t like being challenged.

    #FakeNews and #RussiaGate are smoke covers used by a corporate state to increase censorship.

    It’s already underway…

  14. Todd, I would agree with your comments about the Vietnam War >> one thing was evident by each POTUS – they lied to the people.

    I also agree with your statement > The reason we have “fringe and alternative media” today is that our legacy and establishment media notoriously lie to us on behalf of the government and corporate Oligarchs.

    CNN, MSNBC, and FOX do not provide “News”. They are 24/7/365 editorial commentary spun in the direction of their target audience. CNN, MSNBC and FOX must have significantly reduced their expenses since virtually all of their programming is now in a studio. Sometimes if the story is big enough they may send reporters out in the field, such as for hurricanes, mass shootings or wild fires. Followup- is non-existent by these net works.

  15. “CNN, MSNBC, and FOX do not provide “News”. They are 24/7/365 editorial commentary spun in the direction of their target audience. ”


    It’s always disheartening to read how revered (in some circles) such disingenuous bastards Johnson and Nixon have become.

  16. I won’t use the word s-d because that person in the W/H writes it all the time and I don’t know how to make the image shed a tear

  17. Due to newspapers & TV cutting editorial staff, some media I used to trust have turned into absolute jokes. As a scientist I’m especially critical of the New York Times, which publishes demonstrably “fake news” on topics I know something about (eg GMOs & glyphosate). So I wonder what else, that I don’t know about, is equally garbage.

    Even the Washington Post publishes wildly slanted articles at times. My go-to sources now are ProPublica, the Guardian, New Statesman, Hartford Courant/LA Times, and Reuters. There really aren’t any 100% trustworthy American newspapers anymore.

Comments are closed.