I took a look at the preliminary report from the Bill and Melinda Gates foundation concerning Measurements of Teaching effectiveness, and I was aghast. It almost appears as if someone is setting Mr Gates up for another blue screen of death scenario.

Case in point, even a quick view of tables 4 and 5 in the report should raise some eyebrows. Correlation coefficients do not look very good, albeit granted, this is an preliminary report.

Whats most disconcerting, is the textual discussion of those tables is in some cases near reverse of what they indicate. Perhaps the scariest aspect of this, is that so few look at the data tables, but more so just rely on the text!

The only criticism of the report which I was able to find was the following from Jesse Rothstein from UC Berkeley.

The Bill & Melinda Gates Foundation’s “Measures of Effective Teaching” (MET) Project seeks to validate the use of a teacher’s estimated “value-added”—computed from the year-on-year test score gains of her students—as a measure of teaching effectiveness….

As there is every reason to think that the problems with value-added measures apparent in the MET data would be worse in a high-stakes environment, the MET results are sobering about the value of student achievement data as a significant component of teacher evaluations.

The complete abstract of her criticism as well as a her full report are must reads, especially now that so many states are trying to create legislation as concerns testing and teacher evaluation.