What is peer review? Hey Lower South Platte Water Conservancy District, let it the f**k go. The buck on continuing UEI screwups stops with Polis.
What is peer review?
Short answer: it's a good start, but it ain't perfect.
I've been told more than once by reporters, when questioning their sources, that the material they presented was peer reviewed. This always feels as if it's offered up as punctuation, as the period at the end of a discussion.
See, for example, the reporter's response when I asked about a study she presented about water use and alfalfa in the first link below.
Let's first back up a step and discuss what peer review is. Before a scientist or researcher can get an article published in some journals, the journal wants to make sure they don't publish something that would be so patently bad or false as to cause them embarrassment.
This is a bigger and bigger challenge as academia moves ever toward increased specialization and esoterica. Let me give you an example from my area of study.
Physics splits into many categories. One of them is low-temperature physics, the physics of the very cold. This splits further into an area that has been an off and on interest of mine, low-temperature physics in lower dimensions. By the time you get to this point, you're not out on a limb, you're sitting out with the leaves, and you're out with one leaf among thousands.
Imagine how tough it would be to run a journal and assess whether a paper on a sub-sub-sub-topic of a large field is worth its salt. You can't. No one can be that much of a polymath anymore. Thus, a journal might turn to a researcher's peers to read through and vouch for the work's quality.
A journal that publishes only things that have undergone such a process is called a peer reviewed journal and the articles are labeled as peer reviewed.
Vetting someone's idea by peer review is a good thing. By and large, it prevents errors and bad research from getting out in public. As I say above, however, it ain't perfect.
I don't have time for an exhaustive rundown, but I can give you a sense of the more common and egregious problems that arise in peer review.
First, the issue of "predatory" journals. These are a rough academic equivalent to a shady vanity press. Publishing in one of these journals may give the gloss of peer-review and rigor, but they're anything but.
These are open-access journals (a journal anyone can submit to), that do things like accept articles for money, giving submittals a cursory or nonexistent review, taking others work and rewriting without attribution, etc.
Jeffrey Beall, a librarian and library scientist at the University of Colorado Denver made a scholarly study of such journals and ended up cataloging them and making a list. I am not sure if he is still active in the field, but his work lives on. If this is an interest, you'll find his original criteria for a predatory journal linked second along with a list of suspected journals linked third.*
The fourth link below also deals with predatory journals (as well as a review of what peer review is), but it's from the national library of medicine and thus has a focus centered more on medial journals.
The fifth link below is to a paper that gives a pretty good outline of two of the other common problems with peer review: inconsistency and bias. You'll find that author's take on it by scrolling about halfway down the article, watching for the headings by those same names.
Inconsistency is easy. It's tempting to think that journal articles all undergo a review by a thorough and objective group, but you have to remember that it's humans (not machines) doing the review. There is never objectivity with humans. There's better and worse, but never fully objective.
Maybe one of the reviewers was really busy and skipped lunch so she could knock out the review. Would that reviewer give the same kind of comment as one who had nothing else to do and was well-fed?
What about bias?
Bias could, of course, take several forms and at first blush seems simple. It doesn't always manifest in ways that a layman would think of, however.
To give you a sense of ones you may not think of as a layman, I took two that the author mentions and pulled them out as screenshots 1 and 2.
If you aren't from a fancy university, your ideas won't be seen as equally valuable. Oh, and if you publish a study that shows a negative effect (say a study that shows some drug, or, say, wearing a mask doesn't work)? Yeah, probably not going to get past peer review. Not as readily as ones that do show an effect.
Let's wrap up. As with my earlier post on relevance and reading like a skeptic, I'm not here to make blanket statements. Peer review is not worthless and it's not gold. It's a good idea. It should be continued. It should also not be an imprimatur of infallibility for anyone, media included.
The journal something appears in should be checked on.** What is its reputation? What are its standards? What do researchers say about it? Peer review should also be open to question. It is okay to doubt and it's okay to ask about bias and inconsistency.
Be wary of anyone who cites from a predatory journal or who isn't open to discussions about the peer review process. Transparency and openness to questions are two good indicators of honesty.
*Again, if this is a passion or interest, keep the third link safe and use it as a reference if you wonder at any particular journal.
**If you don't want to keep the list of predatory journals and/or don't want to take the time, a quick google search of the journal is a good idea here.
https://open.substack.com/pub/coloradoaccountabilityproject/p/sneaking-a-study-about-cutting-beef?utm_campaign=post&utm_medium=web
https://beallslist.net/wp-content/uploads/2019/12/criteria-2015.pdf
https://beallslist.net/standalone-journals/
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7237319/
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1420798/
Let it go Lower South Platte Water Conservancy District, let it the f**k go.
The Lower South Platte Water Conservancy District lost (in a unanimous ruling) at the appellate level when the National Taxpayers Union Foundation appealed an earlier court ruling that they district violated TABOR by doubling their taxes without a vote.
This (correctly-decided, a rarity for Colorado courts) appellate ruling overturned what I think were some pretty major flaws in the original decision allowing the district to proceed with their tax pummeling, but the district has decided to seek relief from the Colorado Supreme Court.
Two things about this:
--As a taxpayer in this district, as a taxpayer in general, I would repeat what I put above. Let it the f**k go. You lost in a unanimous ruling. If you want/need more tax money, do what you're supposed to and just ask.
--God save us if the CO Supreme Court takes the case. Then we'll have yet one more precedent by them which will almost certainly erode TABOR further.
Updates as I have them.
Colorado is struggling to pay unemployment claims.
I've heard about the issues Colorado has in paying unemployment from colleagues at my school.** I'm surprised they persist. This has been an ongoing problem since COVID!
The CBS article linked below details how the state unemployment office is struggling to pay out claims, and how internal emails show that they're not handling the problem well. It's hinted at in the article, though not directly stated by anyone, that the head of the office has been having some difficulty in getting his house in order.
Not mentioned at all in the article but worth noting, this department, the Colorado Department of Labor and Employment, is an executive agency. As such, it's under Polis' umbrella.
That is, the buck should stop with him. I understand he has a lot to oversee. I know the meaning of the word delegate. I also know that as the state's chief executive officer, he is ultimately responsible for the state executing the jobs it has to do.
This is not a new problem. This didn't happen yesterday. This has gone on for years under Polis' nose.
Let me end with a question. If you ran a business and were late on your UEI (unemployement insurance) payments, how long do you think it would take for the state to knock on your door? Years?
I genuinely wonder. I mean if they're this incompetent at paying out ...
**The folks that work in the cafeteria have told me. Since they get laid off every time the school is out on breaks they are entitled to a seasonal unemployment benefit and some have told me that they're waiting YEARS on that.