KFF's flawed gun control article which the Sun, CPR, and CTR blithely reprint. Their (mostly non-) response to questions about it. And, to wash it all down, "preference laundering" in gun studies.
Serious omissions and flaws in a Kaiser Family Foundation KFF gun article.
KFF recently put an article out on the market about the recently-passed and now enacted Prop KK. This was the measure that put a state (there already was a Federal) excise tax on guns, gun parts, and ammunition. he sponsor of the measure sold it as a way to help fund victim's services in Colorado (e.g. groups that help victims of domestic violence).
The article, over the course of a few weeks, was picked up by 3 good-sized Colorado media outlets--The Sun, CPR, and Colorado Times Recorder. I link to both the Sun and CPR articles below, first and second respectively, for reference. Both articles are the same save for the headlines (something the outlets themselves choose -- an interesting study in contrast in and of itself).
The article also has some fundamental flaws. Flaws which, seemingly, no one at any of the outlets above picked up on or cared about, giving the article wide distribution without any check on its claims, reminding me of the saying about rumors making it around the world before the truth gets its boots on.
There is the added touch of irony here in The Sun referring to the reporter that wrote this article as a "subject specialist" (see screenshot 1 from the Sun's article header). The below will easily flesh out the hollowness of such a claim.
The first problem pops up quickly without any need for research. Quoting from the article with the links intact:
"Colorado’s new tax is what economists call a 'Pigouvian' tax, which seeks to compensate financially for the societal toll or damage a product causes. For example, people who drive cars pay a tax on gas, which goes toward repairing roads. 'It’s not because you’re a bad driver that we’re taxing gasoline. It’s because we need this money to be able to improve our infrastructure in ways that allow people to continue to use that product,' said Rosanna Smart, an economist who co-directs the Rand Gun Policy in America initiative. She said Colorado’s gun tax is similar: It supports the social infrastructure that’s required in a society with firearms."
It doesn't take an economist to see the mismatch here. Both Pigouvian taxes and gasoline taxes require participants in the act to pay for damages THEY HELP CREATE, this is not something you can sustain for law-abiding gun owners. The law-abiding do nothing (by definition) to cause bad outcomes outside the law.
Digging a little, in addition to reaching out to Ms. Smart directly, pointed me to a further problem. I won't give you the full quote, I didn't ask for Ms. Smart to go on record, but our correspondence made it clear to me that somewhere between the discussion and the printing the point about taxes was muddied.
I.e. a great deal of detail and subtlety from the discussion was lost. What might at first appear to be a thorough look at the nature of taxation, adverse consequences, and poor behavior turned into something with so much lost, so shortened, as to make it inaccurate.
Let's turn now to another quote from the article, again with its links intact.
"The connection between firearms and domestic violence is stark: Nationally, every month an average of 57 women are killed by an intimate partner using a gun. Researchers have also found that 59% of mass shootings between 2014 and 2019 in the United States were related to domestic violence."
The thrust of the KFF reporter's writing is clear: guns are connected to domestic violence, the need for a remedy (a tax for instance put on those that "participate" in the problem) is shown by the numbers.
But how much can we trust the numbers? Do they say what they appear to?
The short answer, the one that gives the KFF reporter the most benefit of the doubt, is that much context is missing in her writing.
The first link from in the quote takes you to a page run by the Center for American Progress (CAP). Knowing nothing of this group, a quick survey of their site makes their purpose clear. They are gun control advocates.
Screenshots 2 and 3 are taken from the very page the link goes to. Look them over and you will have no doubt.
A quick lateral read also fleshes out their political orientation. CAP is strongly tied to the Democrat party and well-funded by all kinds of leftist organizations. Quoting the top line of their Influence Watch page linked fourth below, (links left intact):
"The Center for American Progress (CAP) is a liberal Washington, D.C.-based think tank with strong ties to the Democratic Party establishment created in 2003 as the left-of-center alternative to the right-leaning Heritage Foundation."
Despite this easily uncovered information, not one mention of this group's obvious political leanings appears in the KFF article. Nor is there mention by the Sun, CPR, or Colorado Times Recorder.
Readers, as a way of assessing what they're told, ought to have been informed. They should know that the number used to bolster the reporter's claim comes from a (in addition to other causes) gun control advocacy group.
Turning now to the second linked claim, that 59% of mass shootings between 2014 and 2019 in the United States were related to domestic violence, you see end up at a study whose "Background" section has the following quote (with links intact):
"Mass shooting fatalities account for a small percentage (1%) of firearm homicide fatalities in the United States, but they receive a substantial amount of media attention and may drive political discourse on gun violence (Gun Violence Archive n.d.-a; Centers for Disease Control and Prevention, National Center for Health Statistics n.d.)."
A quick calculation puts the overall percentage of shootings involving both mass shooting and domestic violence at 0.59% of the total. Using the CDC's most recent (2022 -- linked fifth below) statistics that have 48000 gun deaths that year, shows that about 283 deaths in 2022 could be linked to both guns and domestic violence.
Each death is lamentable.
Absolute numbers bring a perspective that the 59% claim alone does not, however. While 283 is too many, 283 out of 48000 paints a less "stark" picture. It suggests room for time and attention put to other causes of mortality, including others that involve guns. It blunts the strength of the connection the reporter is eager to make here.
I am surprised at the lack of attention to detail by the reporter here, not to mention the same by the papers that reprinted the story. It truly leaves me wondering just exactly who is minding the store here.
In the second post today, I share what little I heard back when I tried to get some information from the reporter, CPR, and the Sun (the major outlets that reprinted the article).
** See the third link below for a lengthier reference than the KFF article on the topic.
https://coloradosun.com/2024/12/30/new-colorado-gun-law-aims-to-shore-up-victim-services/
https://www.cpr.org/2024/12/18/prop-kk-new-colorado-gun-tax-effects/
https://www.investopedia.com/terms/p/pigoviantax.asp
https://www.influencewatch.org/non-profit/center-for-american-progress-cap/
https://www.cdc.gov/firearm-violence/data-research/facts-stats/index.html
So if Kaiser Family Foundation (KFF) wrote a flawed article, with the Sun, CPR, and Colorado Times Recorder reprinting it, who owns the problem?
Good question.
And, it's not a new one. The op ed linked first below is one I wrote back in August of this year and it covers this very topic.
The post prior to this one covers a flawed KFF article on Prop KK (if you've not yet, go back and read it for context) which got picked up in the outlets mentioned above.
As part of the research on the first post, and because I was curious, I wrote to the editor of the Sun, CPR, and the KFF reporter herself.
The email I sent is copied in screenshots 1 and 2 attached. I won't put copies of the links below because these are things I have already linked to in post 1.
As of this writing, I have heard from the reporter (Rae Ellen Bichell), and the editor of the Colorado Sun (Dana Coffield).
From Ms. Bichell, I received the following (quoting her email with links intact):
"Hey Cory, thanks so much for reading and for connecting. The figure in the CAP report (reference 3) comes from a U Penn researcher. The direct link to that data isn’t reader-friendly: Jacob Kaplan's Concatenated Files: Uniform Crime Reporting (UCR) Program Data: Supplementary Homicide Reports, 1976-2019"
From Ms Coffield, I received the following (again, quoting her email with links intact):
"Here's an instance when we used Heritage Foundation as a source:
https://coloradosun.com/2020/02/07/colorado-approval-voting-ranked-choice-voting/"
"We tend not to use a lot of national organizations as sources. We do use the Common Sense Institute on the reg.
https://coloradosun.com/tag/common-sense-institute/"
I followed up with Ms. Coffield by thanking her and then asking her for a response to the notion that, in almost all of the links she supplied, Common Sense Institute and Heritage Foundation are referred to as some version or another of "conservative" while the Center for American Progress (the source of one of the claims in the KFF article) was not labeled.
As of this writing I have not heard back.
The use of other outlet's work in nonprofit papers has been something I've dived into a lot over the past few months. I see it as fraught with problems exactly of the kind you see in this and the previous post.
When there are flaws, when there are mistakes, who owns them? Going back further than that, who is responsible for checking for them, or, on an even more basic level, is anyone at the outlets using this shared content bothering to look?
Questions so far without much in the way of answers, and little seeming will to do anything about it.
https://pagetwo.completecolorado.com/2024/08/20/gaines-questions-accountability-shared-news-content/
Preference Laundering in gun policy studies.
A reader sent me the link below after I'd made a comment involving the studies on gun policy and gun violence.
I really liked reading it. I signed up for this substack (for free) and recommend you do the same, especially if gun control and gun violence are interests of yours.
The essay below actually has a bunch of material in it.
There are links to Reason Magazine's video and text articles and reviews of studies on gun policy (as well as an assessment of how good a job RAND--the group that did a metastudy on gun policy studies--did). I have touched on this in the past, and I will leave it to you to read up on this if you'd like.
If you've not familiarized yourself with it, you should. It gives you a good sense of just how meaningless studies like these can be, and knowing this helps you better advocate in this state because gun control advocates and the Democrats that serve them in our legislature love to quote these kinds of studies as if they were the God's honest.
The novel aspect that the Open Source Defense Substack takes on is what they refer to as "preference laundering".
Preference laundering, at least as the author in the essay has it (my research brought up some other definitions in other contexts -- health economics being one), is a way of establishing good old fashioned common sense.
In the context of gun policy studies, that means that the kinds of studies that confirm the efficacy of a particular policy, a policy which we know works, are what all us "thinking people" believe to be true.
Quoting the essay:
"This [the dynamic whereby the studies that confirm what policy makers and influencers believe to be true, particularly the studies that make a big splash, are the ones that get attention and thus money] creates heavy selective pressure towards studies that reinforce that preference, and away from those that disconfirm it. That pressure can be so strong that it turns an entire field from a scientific enterprise into an emergent preference laundering machine. Preexisting preferences and grant money go in one end, peer-reviewed studies with ≤0.05 p-values [in other words, "statistically significant" studies that journals and others have a bias for] come out the other end. Crucially, this doesn’t require any ill intentions among the individuals in the field. They can all be working earnestly on well-designed studies. It’s simple publication bias: the ones who luck into the 'right' results will get published, and the ones who don’t won’t."
The reason that this stood out to me is that I want you (presumably like the author) to be aware of this dynamic and to not be intimidated.
That is, if you take nothing else away from this particular post, I want you to take the following:
--It is perfectly valid for you to question things, even if the person trying to convince you has a lot of letters behind their name, even if they are a well-known politician, even if they hint (or directly state) that your doubts make you dumb or ignorant.
--Be polite and be thoughtful in your doubt, but hold people making claims accountable for what they say. They, and not you, have the burden of proving their claims and asking them to do so by detailing what they've done and what they've not, will often open up all kinds of holes in what was seemingly so ironclad before.
--I don't want to make a blanket statement that every single study about gun policy and/or gun violence is wrong or useless, but (by a long chalk) many of them are. If this is an interest for you, take the time to study Reason's video and article linked in the essay.
--Remember for every study that gets cited or that makes the news, there are many (MANY) that don't make the news that don't support the narrative the news, the influencer, or the politician wants to push. If you wonder at statistics on this, check out the link. The numbers are startling.
I will end with the words that the author of the essay ended with since they said it as good or better than I could:
"A preference laundering machine is powerful, but it’s also fragile. Because it hinges on monopolizing 'this is what smart people are supposed to think', it’s vulnerable to any hint that it is in fact built on a foundation of basic mistakes. So this doesn’t require vitriol or name-calling or an us vs. them approach. All it requires is for you to point out, 'Hey, you care about getting things right. That’s awesome. So here’s what you haven’t been told about guns.'”
