Augmented Synthetic Control: Cargo Cult Science, hocus pocus with fancy words. More Democrat politician troubles that appear as from nowhere. The only instrument I am by now qualified to play.
Augmented Synthetic Control: a variation on Cargo Cult Science, hocus pocus with fancy words.*
This is going to be a two part series because it's a challenging topic and going fast leaves out more detail than I am comfortable with. Part 2 will appear tomorrow.
This may be challenging. It is worth the effort, however. You need to know the words that social science uses so you can properly evaluate what you hear not only from researchers but when their words are parroted by policymakers.
I posted a little bit back about some public health research on concealed carry along with an interview with Dr. Betz, a big name in public health and guns here in Colorado. A link to that post is first below if you'd like to go back for context.
A reader contacted me after reading that day's newsletter and asked what in the world "Augmented Synthetic Control" was, a phrase she'd heard while listening to the linked audio.
It's tempting to be impressed with the jargon researchers toss around. You hear the $10 words and think that the complexity must mean their results are true, that the researchers' credentials make their conclusions unassailable.
If you feel a pull to do this, I would urge caution. Everyone puts their pants on one leg at a time. Truth is not determined by the number of syllables. Stop and ask the simple question: what am I being given and what is its value?
If you're pretty familiar with the basics of science and have a decent amount of math background, skip now to the video linked second below and we'll see you tomorrow. It gives a rundown on what Synthetic Controls are.
If not, don't worry. I'll give you the pre-req's today and we'll continue tomorrow.
Remember that statistics--the bread and butter of the social sciences--CANNOT EVER give you causality. It doesn't matter what the formulae look like, what the letters are behind a researcher's name, the amount of jargon bandied about, statistics do not tell you that A causes B. Experiment is the only valid way to establish that.
And when it comes to experiment, you need to have a control and change only one variable at a time. I attached a simple screenshot about science to give you an idea of what I mean.
Think it through. You must have a control that matches the experimental condition in every respect but one. Let's say you ran out of Happy Plant potting mix filling up your first pot and were in a hurry to set up your experiment. So, you run outside real quick and scoop some dirt out of a flowerbed to toss in the pot.
If the seeds in your control pot never germinated, how could you ever be sure of whether this was due to the lack of water or the soil? Controls are vital in causality.
Another vital concept in science is an experiment's reproducibility and repeatability. These sound the same, but as hinted at in the screenshot, they are not.
For a result to be valid, other researchers must get the same result when they repeat the same procedure. The experiment must also reproduce the same result (done by you or someone else) when repeated multiple times. Failing this test puts a big question mark on your results.
It may not negate the results, and there need not be exact overlap trial to trial, but repeated trials and trials done by other research groups ought to produce results that are so close as to give the scientific community confidence that they are within small random (natural) errors.
It would be nice if we could do experiments on any scientific question we might have.** Unfortunately, there are some questions that do not lend themselves well to experiment.
You might want to know if a particular chemical causes cancer. It's not ethical to give a group of people this chemical and another group a placebo to see what happens.
It's also often not feasible to run experiments with things like public policy because policy is not randomly assigned (we the people decide it), and you cannot randomly assign people to groups because they choose where they live.
As we've seen with concealed carry public health research, as we'll see with tomorrow's post, researchers try in cases like these to exploit what they call "natural experiments" (see the third link below for a quick Wikipedia explainer): a naturally-occurring set up that is arguably a reasonably close thing to an experimental condition.
One of the examples in the YouTube video below is California's 1988 passage of Prop 99, a measure to try and reduce smoking. Researchers later tried to evaluate whether the measure was actually effective in reducing smoking. They wanted to capitalize on the fact that California, as it so often does, led the nation in policies like these; California passed a law to try to stop smoking before other states did.
Is this so-called natural experiment comparing California to other states valid? Are other states good control groups? If they're not, can we somehow make our control group, a synthetic (manufactured) control group?
That is tomorrow.
*See "Related" content below for more on Cargo Cult Science.
**The test of whether something is a question that science can answer is easy. If it is something that can be falsified by experiment, it's a question that science can answer.
https://en.wikipedia.org/wiki/Natural_experiment
Related:
If you needed or wanted the context on Cargo Cult Science, the below is the relevant section of one of Feynman's 1974 Caltech address (which is linked in full at the bottom).
I think the educational and psychological studies I mentioned are examples of what I would like to call Cargo Cult Science. In the South Seas there is a Cargo Cult of people. During the war they saw airplanes land with lots of good materials, and they want the same thing to happen now. So they’ve arranged to make things like runways, to put fires along the sides of the runways, to make a wooden hut for a man to sit in, with two wooden pieces on his head like headphones and bars of bamboo sticking out like antennas—he’s the controller—and they wait for the airplanes to land. They’re doing everything right. The form is perfect. It looks exactly the way it looked before. But it doesn’t work. No airplanes land. So I call these things Cargo Cult Science, because they follow all the apparent precepts and forms of scientific investigation, but they’re missing something essential, because the planes don’t land.
Now it behooves me, of course, to tell you what they’re missing. But it would he just about as difficult to explain to the South Sea Islanders how they have to arrange things so that they get some wealth in their system. It is not something simple like telling them how to improve the shapes of the earphones. But there is one feature I notice that is generally missing in Cargo Cult Science. That is the idea that we all hope you have learned in studying science in school—we never explicitly say what this is, but just hope that you catch on by all the examples of scientific investigation. It is interesting, therefore, to bring it out now and speak of it explicitly. It’s a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty—a kind of leaning over backwards. For example, if you’re doing an experiment, you should report everything that you think might make it invalid—not only what you think is right about it: other causes that could possibly explain your results; and things you thought of that you’ve eliminated by some other experiment, and how they worked—to make sure the other fellow can tell they have been eliminated.
Details that could throw doubt on your interpretation must be given, if you know them. You must do the best you can—if you know anything at all wrong, or possibly wrong—to explain it. If you make a theory, for example, and advertise it, or put it out, then you must also put down all the facts that disagree with it, as well as those that agree with it. There is also a more subtle problem. When you have put a lot of ideas together to make an elaborate theory, you want to make sure, when explaining what it fits, that those things it fits are not just the things that gave you the idea for the theory; but that the finished theory makes something else come out right, in addition.
In summary, the idea is to try to give all of the information to help others to judge the value of your contribution; not just the information that leads to judgment in one particular direction or another.
https://calteches.library.caltech.edu/51/2/CargoCult.htm
More Democrat politician troubles that appear as from nowhere.
Just like Senator Winter's problems with alcohol didn't happen overnight, I am guessing Senator Jaquez-Lewis' staff troubles didn't pop up overnight either.
I KNOW that Secretary of State Griswold's didn't happen overnight.
Yet, more often than not, we only hear about such things long after they started.
If a Republican leaves a loaded gun in a bathroom in a closed Capitol building it's on every outlet that night or the next day.
Lauren Boebert breaks wind in the canned goods aisle of the grocery and we hear about it 5 minutes later. You almost can't NOT hear about it.
I continue to ask why that is. I don't mean that rhetorically either. I have asked in the media more than once about this disparity.
Really never get an answer.
https://www.cpr.org/2024/04/16/former-legislative-aides-workplace-concerns-against-senator-sonya-jaquez-lewis/
https://denvergazette.com/opinion/columns/griswold-pays-six-figures-on-discrimination-claim-jimmy-sengenberger/article_00fcdc02-3dd5-5ea4-8c54-98c9f969d385.html
Related:
The 9News story below is about the NDA's that some Denver Public Schools employees have signed. Worth a watch.
It reminded me of the Democrats' spotty record on NDA's (and not only the Secretary of State's either).
Senator Kirkmeyer was a champion of trying to limit the state's ability to use NDA's for its employees, running bills nearly every year to try and get some reasonable limits put on them.
Her 2021 effort failed (see the second link below) on a party line vote in committee. Her 2023 effort (which expanded protections against NDA's to local government employees not just state employees) passed and was signed into law. That's the third link below.
https://www.9news.com/article/news/local/next/next-with-kyle-clark/dps-superintendent-alex-marrero-uses-ndas-to-prevent-employees-from-speaking-out/73-b813118d-7e43-4ac7-ac85-3705106ebcac?utm_campaign=snd-autopilot
https://coloradofoic.org/bill-prohibiting-nondisclosure-agreements-for-state-employees-dies-in-a-colorado-senate-committee/
https://coloradofoic.org/colorado-senate-bill-would-bar-nondisclosure-agreements-for-state-and-local-government-employees/
The only instrument I am by now qualified to play.
It's that time of the week. Time for something fun and not related to politics.
**NOTE: I will follow up with part 2 of the series on Augmented Synthetic Control in gun control (and other) studies tomorrow. Then on Sunday I'll be back with a variety of topics again.
I forget if I have mentioned, but I played the string bass all through high school. I played in the school orchestra (not jazz), and enjoyed it. I never went anywhere with it. I didn't really give my heart and soul to it. The experience was enriching, and I think (looking back now) that I got what I wanted/needed out of playing.
I was listening to some classical music a few days back and the little clip linked first below came up in my suggested videos.
I had to laugh when I saw it because I was not aware of any orchestral piece involving a giant wooden mallet. I, of course, watched the short and then looked up the piece it's from.
It's (near as I can tell, I could be wrong) Mahler's 6th symphony. I linked to that one second below if you'd like to hear the work in full. I would recommend it; I like Mahler, maybe not everything, but his pieces have a lot going on .
After laughing and watching the clip on the hammer, I was thinking that after the 1000 years that have elapsed since I was in high school that this is probably the only instrument I could now be qualified to play. I mean, it's a single hammer blow.
I just hope that if I were asked to do it, I could replicate the gentleman's look of purposeful intensity as I did so!
That's it for now. Eat a high protein meal today beef up your brain and see you tomorrow!