Propensity Score Matching (PSM), another example of cargo cult science. Fly ash, concrete and unintended consequences. Happy International Sloth Day!
Propensity Score Matching (PSM), another example of cargo cult science.
The CPR article linked first below is about how a recent study found that (unequivocally apparently) DPS's school reforms worked.
In fact, the story quotes the study as saying the reforms were "among the most effective in U.S. history."
We could probably add nigh-on-to-hyperbolic to the unequivocally above; I mean, with zippy quotes like that, how could you go wrong?
You can go wrong when you go outside the bounds of what your research actually is reasonably able to claim.
The CPR article carries another quote from the researchers behind this study: "These effects are among the largest ever observed in educational research using state of the art extremely rigorous methodology that observes individual student achievement over time."
I don't have time to go through the entirety of the study (which is linked second below the CPR story), so I'll limit my attention to their method and how it affects the strength and validity of their conclusions.
I took a screenshot from the report (p 20 in the link) and attached it as screenshot 1.
Here you see part of the method used by the researchers. It's a method called Propensity Score Matching (PSM). I linked to a Wikipedia page which should give you a fair overview and some links to follow up on if you're interested in learning more.
PSM is, as the researchers themselves (though not CPR of course) note, a step down from actual science. To wit, the following quote from their report:
"Choosing school districtsis not a random process and it is reasonable to expect that some of the factors that determinewhere families enroll their students may also influence their students’ academic performance(e.g., parent education level)."
In order to pursue their "state of the art" research and arrive at their conclusions, the researchers here employ PSM, skirting the pesky need for true comparisons and actual control groups.
Put another way PSM is yet another statistician's trick to create a control group and then make inflated claims as to validity.
PSM is, in this way, similar in theme to synthetic control groups. It's yet more cargo cult science: research that mimics science, wanting you to think it's the same, but that only has the appearance of science.
I put links to three previous newsletters discussing synthetic control groups and gun control studies below the Wikipedia link if you want to see more there.
Whether you delve deeply or shallowly, please take the time to do what the press will not do on your behalf. Take the time to note how the study authors arrived at their conclusions.
If you see propensity score matching, if you see synthetic control groups, remember that they are both attempts to skirt the fact that their measurements are correlations, might have influences outside the things the researchers are looking at, and that they CANNOT make any truly valid claims about causation.
https://www.cpr.org/2024/09/23/denver-public-schools-controversial-reform-successful/
https://wp-cpr.s3.amazonaws.com/uploads/2024/09/Denver_Systemwide_and_Intervention-Effects-Technical-Report-Embargo0001.pdf
https://en.wikipedia.org/wiki/Propensity_score_matching
https://coloradoaccountabilityproject.substack.com/p/augmented-synthetic-control-cargo?utm_source=publication-search
https://coloradoaccountabilityproject.substack.com/p/part-2-of-augmented-synthetic-control?utm_source=publication-search
https://coloradoaccountabilityproject.substack.com/p/house-democrat-majority-leader-duran?utm_source=publication-search
Fly ash, concrete and unintended consequences.
There is a product of coal combustion called fly ash. Skipping over a tremendous amount of detail here, coal in power plants is very finely powdered prior to burning. One of the solid combustion products is the very finely powdered ash which goes by the name fly ash.**
Fly ash has a number of uses but the one mentioned when I first heard about this was fly ash's role in concrete.
I put three links below varying in sophistication/technical detail as you go down the list from 1 to 3.
I'll leave it to you to read up as much (and in as much detail) as you would like. I do want to wrap up by mentioning this.
When we start shutting down coal plants, we will lose more than jobs. We will lose more than reliable baseline generation. We will lose things like fly ash, the ancillary products that come out of the process and which did have value.
It's not irreplaceable. Alternatives exist, some which cost more, some cost about the same. I am not sure about the volume availability of fly ash vs. these alternatives (i.e. will we have the same amount at the same price and on the same timetable?), but the switchover will likely carry some disruption.
Meddling in markets and imposing timetables based on desire rather than reality, as Progessive lawmakers in this state have done in shutting down coal fired plants, carries with it just such unintended consequences.
Another casualty to thoughtless policy.
**Note that there are other combustion products from coal that are useful, see in particular the second link below for other examples.
https://www.thespruce.com/fly-ash-applications-844761#:~:text=Using%20fly%20ash%20in%20concrete,a%20tendency%20to%20resist%20shrinking.
https://www.cemexusa.com/products-and-services/fly-ash
https://www.fhwa.dot.gov/pavement/recycling/fach00.cfm
Happy International Sloth Day!
A sloth in estrous**.
What is the purpose of a three toed sloth? To make more three toed sloths of course.
If you've never heard it before, I thought you might enjoy how it is that lady sloths alert the fellas out there in the jungle (sloths are solitary creatures and don't congregate except to mate) that they're ready to make a baby.
They screech. Loudly and piercingly.
See the video linked at bottom to give it a listen. You'll have to be patient, she doesn't give her call til near the end.
You know, for such low-key creatures, I have to say I'm surprised at the gusto of that yell. I would've thought subtler and quieter would be the way. I guess if you're living in different trees and have a whole bunch of chatter from other animals to work around, you gotta do what you gotta do.
Now, this is a family website so we'll leave it there. I'm sure you can find more details on sloths mating on the internet if you want. No judgment.
Enjoy your holiday and whatever it is that you do to celebrate, make it a good one!
**i.e. in heat