So I read this article on how standardized test essays were scored and about had a bird. I sort of figured that if essays were used as part of standardized testing, they would be assessed with some degree of rigor. Ie multiple scorers, who would discuss each one, and from there come to consensus as to how to rank an essay over a given number of domains, ie creativity, grammer, spelling, depth of understanding etc. Such ranking would provide enough resolution such that meaningful feedback could be returned to the students class room teacher.
Rather, it appears its single readers going as fast as possible, who are providing anything but meaningful feedback. It also appears an incredibly biased process which leans towards the least common denominator save for one thing, the grading firms are getting rich off the deal. In a lot of ways such parallels Matthew Crawfords experiences at a research firm which he wrote about in Shop Class for Soulcraft.
The saddest part of this deal, is that school funding is tied to it, and some states are even proposing teachers compensation be related to the scores of such tests. I sort of wonder why taxpayers arent likewise having a bird over such methods… they are paying for it afterall.
The thing is money talks, and such is a way the aforementioned problems can be addressed. I posted the following comment over on the citipages article.
One solution to this debacle is to require clawbacks as part of the contract with standardized testing companies based upon a students future performance. In other words, the testing industry would in effect be accountable for their methods of assessment (as well as fraud detection).
For example, if a student who does very well on the standardized tests, either MC or essay, and then said student proceeds to flunk out,or even get C’s or D’s while in high school or college years later, significant penalties would then be automatically and retroactively imposed upon the grading firm. It likely would be necessary to require bonds to be posted up front as part of said contract, as a firms viability cannot be predicted over the long haul. Granted, there are any number of factors well beyond the control of the assessment firms..but then again insurance actuaries do the same kind of thing nearly every day.
Granted, such an approach might prove to be too expensive to operate on shore, being it would likely equate to highly trained graders working together, and having enough time to achieve a level of accuracy such that their firm would not end up loosing their shirt years down the road (or go out of business today, as they could no longer afford the bond). On the other hand, having the assessment firm being at more than are arms length from those who would wish to influence outcomes might not be such a bad thing either.
I guess I’m half sarcastic,and half idealistic in this… it would throw such a huge wrench into the status quo, and many grading firms would go under, and likewise many would lose their jobs. On the other hand, to hold a school district accountable with teachers, staff, students, and parents at risk, all the while the test firms rake in the dough with so little accountability is not cool either.