As colleges have faced increasing pressure in recent years to demonstrate that students learn something while enrolled, many have turned to tests of learning outcomes, such as the Collegiate Learning Assessment. In that test — and popular alternatives from ACT and the Educational Testing Service — small groups of entering and graduating students are tested on their critical thinking and other skills. In theory, comparing the scores of new and graduating students yields evidence either that students are or are not learning. Many call the difference between the entering and graduating students' performance the "value added" by a college degree.
These test results may be high-stakes for colleges, many of which need to show accreditors and others that they are measuring student learning. But for the students taking the exams, the tests tend to be low stakes — no one must pass or achieve a certain score to graduate, gain honors or to do pretty much anything.
A new study by three researchers at the Educational Testing Service — one of the major providers of these value-added exams — raises questions about whether the tests can be reliable when students have different motivations (or no motivation) to do well on them. The study found that student motivation is a clear predictor of student performance on the tests, and can skew a college's average value-added score. The study recently appeared in Educational Researcher, the flagship journal of the American Educational Research Association. An abstract for the study, "Motivation Matters: Measuring Learning Outcomes in Higher Education," may be found here.
The ETS researchers — Ou Lydia Liu, Brent Bridgeman and Rachel Adler — gave the ETS Proficiency Profile (including its optional essay) to 757 students from three institutions: a research university, a master's institution and a community college. (The ETS Proficiency Profile is one of the three major tests used for value-added calculations of student learning, with the others being the CLA and ACT's Collegiate Assessment of Academic Proficiency.) The students were representative of their institutions' student bodies in socioeconomics, performance on admissions tests and various other measures.
Click through for full article content.