Success on the Largest Scale

This summer I had the pleasure of working with Heather Klesch, Senior Area Director, and Tracey Magda, Psychometrician, at the Evaluation Systems group of Pearson in Hadley, Massachusetts. As mentors Heather and Tracey designed an internship comprised of three major events: (1) drafting a technical manual, (2) researching contact modes, and (3) assisting at a national benchmark-setting conference.

The general public was the intended audience for the technical manual. Wording needed to be precise and non-technical while describing topics like equating and reported statistics. Many of the technical staff in Hadley kindly discussed test construction and score validation with me, offering me insight into their fields. During this project, I gained a greater understanding of both the practice and the description of operational psychometrics.

Researching survey contact modes was directed by operational questions regarding content validation surveys. After attending meetings and reading relevant literature, I presented a one-page bulleted synopsis of literature highlights, addressing comments and concerns. On an abstract level, both the technical manual and the survey mode research required retaining meaning while summarizing.

At the St. Louis benchmark-setting conference, I saw the positive effects of a well-planned event. From all 50 states, roughly 700 educators converged on St. Louis to set benchmarks for the 31 tests of the National Evaluation Series (NES). Subject matter expert training was standardized, and there was a protocol for all confidential materials. In addition to planned standardized procedures, we were able to demonstrate appropriate protocol during an unexpected fire alarm. I enjoyed the responsibility of assisting at the conference, knowing that our process would become part of the validity evidence for the NES.

On a personal note, and as a baseball fan, I was thrilled about the St. Louis trip. There has been a new iteration of Busch Stadium since 2004, but on my birthday I got to stand in the air space of what I consider to be the most life-changing stadium of all time. In 2004, the Red Sox pulled off a complex coordinated effort. After they won, the Red Sox belonged to the set of teams that accomplished their goals on the largest stage. Red Sox fans could believe that years of suffering had finally paid off. In my summer of 2010, the same could be said of Pearson; I saw Pearson succeeding at a complex coordinated effort that also took considerable talent. After the conference, I saw that Pearson can accomplish goals on the largest scale, and I can believe that my years of school will also have been well-spent.

It was a special event in my life to be at Pearson this summer with Heather and Tracey. I was given responsibility to do exciting work that mattered. Now, in addition to an excellent graduate school experience, I have an empirically-borne confidence that I will enjoy my future in psychometrics.

Amy Semerjian

Psychometric Intern

Test, Measurement & Research Services

Pearson

Professionals Sharing Knowledge

My summer internship at Pearson in San Antonio, Texas, has been a rewarding experience. I worked along side exceptional individuals and gained valuable experience. In addition, I developed new friendships.

Over the eight weeks, I worked primarily on a research project that dealt with the selection of common items in creating a vertical scale. The study investigated how decisions such as the structure of a common-item design, the composition of the common-item set, and the procedure used for selecting stable common items impact the nature of students’ growth from grade to grade. I learned a great deal from my mentors, Michael J. Young and Qing Yi. They taught me how to conceptualize research ideas while using available data. We submitted a proposal of this study for the 2011 National Council on Measurement Education annual conference. As well, this study set the foundation for what I hope will become my dissertation.

I attended regular training seminars and skill-enrichment meetings. Attending these seminars and meetings helped me to further develop my skills with psychometric software programs (e.g., WINSTEPS and SAS training) and recognize how those skills are applied in practice. Also, I broadened my understanding of the current issues in the field of educational measurement (e.g., consequential validity). As I listened and observed, I appreciated learning how serious Pearson research scientists are about their work.

From time to time, I participated in discussions with research scientists and received instruction from them about relevant topics related to psychometrics (e.g., test construction, simulation, equating, dimensionality). Given their busy schedule, they were very accommodating. I would like to express my gratitude to those individuals who took the time to share their knowledge with me, namely Allen Lau, Marc Johnson, Tim O’Neil, Kwang-lee Chu, Thanh Nguyen, Hua Wei, and Daeryong Seo.

Some of the more memorable moments were simply conversing with everyone in the department. Thanks to Agnes Stephenson, Mark Robeck, Serena Lin, Toby Parker, and Stephen Jirka for making me feel welcome and a very special thank-you to Dee Heiligmann and David Shin for looking after me!

For those future fellows, my suggestion would be to take advantage of the vast knowledge and resources available to you and your internship will surely surpass your expectations.

Assunta Hardy
Psychometric Intern
Test, Measurement & Research Services
Pearson