Found Wisdom: Validity of Student Evaluations

April 4, 2016 / By: Dr. Stephen Brookfield, John Ireland Endowed Chair

For Synergia newsletter

This month's Found Wisdom is courtesy of Dr. Stephen Brookfield, John Ireland Endowed Chair in the College of Education, Leadership & Counseling.

These two articles call into question the contention that student evaluation of teaching (SET) forms, like the IDEA. form St. Thomas uses, are statistically invalid; that is, that they don't measure the teaching effectiveness they purport to document. The authors claim instead that such forms tell us chiefly about students' identities, biases and experiences, and about the continuing influence of patriarchy and racism. Given the importance of IDEA scores for reappointment, promotion and tenure at UST, any analysis that calls their validity into question deserves our attention.

An Evaluation of Course Evaluations

"Student ratings of teaching have been used, studied, and debated for almost a century. This article examines student ratings of teaching from a statistical perspective. The common practice of relying on averages of student teaching evaluation scores as the primary measure of teaching effectiveness for promotion and tenure decisions should be abandoned for substantive and statistical reasons: There is strong evidence that student responses to questions of “effectiveness” do not measure teaching effectiveness. Response rates and response variability matter. And comparing averages of categorical responses, even if the categories are represented by numbers, makes little sense. Student ratings of teaching are valuable when they ask the right questions, report response rates and score distributions, and are balanced by a variety of other sources and methods
to evaluate teaching."

-- This article by Philip B. Stark and Richard Freishtat was originally published in ScienceOpen in 2014.

Bias Against Female Instructors 

"New analysis offers more evidence against the reliability of student evaluations of teaching, at least for their use in personnel decisions."

-- This article was published in 2016 on Inside Higher Ed