If David Beats Goliath, Just What Role Does the Psychometrician Play?

So I admit it—I, too, like a good, sensationalized feel-good piece of literature, one that particularly has some application to what we do for a living. My very good but departed friend, Ed Slawski, use to say I was such a soft touch.

Regardless, I recently read a piece by that world famous author Malcolm Gladwell (you know the Tippling Point, Blink, and Outliers guy) that did indeed move me to write this blog. His article appeared in The New Yorker (yes, even an old curmudgeon like me actually subscribes to the New Yorker) and is titled "How David Beats Goliath." In this article, Mr. Gladwell parallels how the underdogs seem to win more often than they should because they change the rules on how the game is played. He uses the full-court press in basketball as an example of a strategy that a smaller, less talented team might use to beat a taller, more talented team. He actually cites statistics regarding the success of such a strategy.

Psychometricians, as you may well know, are very methodical people. They like specifications outlining what it is that they do. They like to follow procedures, are often meticulous, and believe in verification, transparency, and replicability. Simply stated, they follow the rules. So, the question at hand is: Will psychometricians be the David or the Goliath of the new assessment reform implicit in the new administration and more explicit as a goal of Secretary Duncan?

Certainly, the brave new world of assessments in the post-NCLB era will be unlike what we have seen to date. Reliability as a measure of internal consistency, and validity as a correlation coefficient with existing measures are not likely to be the psychometric quality mantra moving forward. These new assessments are likely to be driven by needs for problem-solving measures, measures of critical thinking, assessments of our ability to manage large amounts of information (presumably coming from the Internet), and comparability with international benchmarks all implemented and managed in an online and automated way. By definition, these new assessments will violate the rules and will be the David that defeats Goliath. I wonder how the greater psychometric community will react. "Just say no" comes to mind but will likely be a woefully inadequate response.

I certainly hope that under my direction, my staff will embrace the need to see the world differently now, even more so, lest we fall behind and are not in a position to support this next generation of assessments. Pearson plans to be the giant killer in this regard, changing the rules and leading the way into this new generation of learning. What is it that you plan to do?