The 2007 Coffman Lecture Series: Lindquist was Right!

Living in Iowa City, arguably the mecca of achievement testing, I am privileged and fortunate to be able to attend professional development activities that many of my colleagues located elsewhere can not. Take for instance the 2007 William E. Coffman Lecture Series sponsored by the University of Iowa and the Iowa Testing Program. This year the guest lecture was Dr. Daniel Koretz, Harvard Professor and noted measurement scholar. I have disagreed with Professor Koretz in the past, but I disagree with everyone on virtually everything so this should be of no surprise. However, the good doctor's lecture this year, while nothing new, reminded me and my staff of why we got into measurement in the first place. The title of Dr. Koretz's presentation was:

Test-based Monitoring and Accountability: Time to Take Lindquist's Warning Seriously

Dr. Koretz started his lecture by recalling the words E.F. Lindquist composed as the introduction to the 1951 edition of Educational Measurement, of which Dr. Lindquist was the author:

"The widespread and continued use of a test will, in itself, tend to reduce the correlation between the test series and the criterion series…. Because of the…potency of the rewards and penalties associated…with high and low achievement test scores of students, the behavior measured by a widely used test tends in itself to become the real objective of instruction, to the neglect of the (different) behavior with which the ultimate objective is concerned." (Lindquist, 1951, 152–153)

What this states, simply, is that even in 1951 and prior to the NCLB rage for "education reform," "high-stakes testing" and "accountability," Dr. Lindquist anticipated the consequences warned by many when we moved to high-stakes testing. This consequence is the focus on improving test scores without the improvement in learning required.

As Dr. Koretz points out in his lecture, this does not have to be as blatant as an increased motivation to cheat or otherwise subvert the system, or even the result of teaching to the test. It could be something much more subtle; one example of which he calls "Reallocation." Reallocation means, in its most simplistic, the shifting of educational resources based on assessment results or lack there of. While this might make sense—that is, if the measures say we need help in area X, we should look at improving instruction and learning about area X. But the complications lie in how we pay attention to this area of needed improvement. Or, to quote Dr. Kortez:

"Individual elements of the domain may be measured well, but representation of the domain is undermined."

An example of this—for those of us who have to make sure our corners are square—is the use of the "3:4:5" triangle as a substitution for a clear understanding and application of the Pythagorean theorem. Students are likely to get the test questions correct without knowing the Pythagorean theorem if test builders are not careful about how they ask the questions and students apply this simple rule. In this case, incorrect inferences about the Pythagorean theorem and possibly about the more general domain of geometric shapes will be made when in reality rote learning of a "trick" led to the correct answer.

Professor Koretz ended his lecture showing that efforts to reduce the "reallocation" effect NCLB has brought is failing mainly because educators are not heeding the words Dr. Lindquist posed years ago: namely, instruction should not focus on the sample of skills tested, but rather on the domain being measured. My staff and I argue it is the instruction that the dialog should be about, but clearly we need to ensure that the efficacy of the measures are of value first.

CASMA-ACT Conference Looks Great, Despite It's Name!

The CASMA-ACT Invitational Conference on Current Challenges in Educational Testing seems to offer a very interesting slate of speakers and topics, despite it's rather academic name.

Dr. Dan Koretz, from Harvard and a national assessment expert, will speak about higher education accountability.

Joe Crick, from the National Board of Medical Examiners, has been solving assessment problems almost as long as there have been assessments and will give his perspectives on performance testing simulations.

Dr. Doug Christensen, the current Commissioner of Education in Nebraska, is an outspoken critic of NCLB and to his credit has managed to implement education the way he sees fit for his native Nebraska. All of this despite NCLB being the "law of the land."

These speakers alone would make it worth the trip to Iowa City on a warm and sunny November afternoon (Saturday, November 3rd to be exact). I promise that the fall colors will still be...well, that the leaves will have color...well, some leaves will still be on the trees!

Another interesting part of the conference will be the participation of "national media" as they provide their perspectives. I hope to leave before this part. Oh, yeah, then there is Popham.

Regardless, it looks to be a great conference, and I hope you can find the time to attend. The Hawkeyes will be out of town that weekend, so hotel rooms will be easy to find and inexpensive.