From the Director: IDEA -- Ideals and realities
Dr. Ann Johnson, Faculty Development
It’s not unusual for me to field complaints about the IDEA system for student evaluation of courses; every system has its flaws. But over J-Term, I talked with a faculty member who voiced concern of a different sort. His conversations with other faculty had convinced him that IDEA feedback was, in some cases, having a demoralizing impact. We’ve all encountered disrespectful comments and evidence that students, some students, were not taking the task of evaluation seriously. For faculty who deeply identify with their roles as teachers and take student feedback to heart, this can be wounding and difficult. In addition, those who have been teaching for many years likely recall a time when students took the task seriously and communicated more consistently with respect.
Have students changed over time in how they treat the course evaluation process? Evidence suggests that they have. Research on the millennial generation tells us that this group views education with more instrumental aims compared to earlier cohorts, and tends to expect good “customer service” from professors. As a result, some approach the course evaluation task as if they’re reviewing a restaurant on Yelp. I don’t want to paint with a broad brush here; many of our UST students are thoughtful about the evaluation process and approach it with care. But recent research does raise concern about the shift in values and attitudes seen in the millennial group and what it means for faculty in institutions that use student ratings for evaluative or summative purposes, like our own.
In the 1970s, researchers testing the validity of student ratings found strong correlations between those ratings and other measures of student learning, lending weight to the view that student ratings can serve as an effective proxy for evidence of actual learning. But more recent work, summarized by Linda Nilson (2012) undermines confidence in the old claims and suggests that student ratings of courses may not serve as a useful reflection of the learning actually taking place in the classroom.
Reactions? I can think of three. First, when we adopted IDEA, we also instituted regular peer evaluation of teaching for those seeking promotion; this, and the personal narrative faculty prepare every year ensure that student ratings aren’t allowed to tell the whole story during the teaching evaluation process. Second, students are not always meta-cognitively aware of their own learning, and we can help them improve as course evaluators by calling more attention to the issue. Discussing IDEA-based learning goals early and often, and demonstrating how your activities promote those goals is one good way to achieve this.
Last, faculty can be proactive in developing additional evidence of teaching effectiveness by incorporating measures of student learning, e.g., pre- and post-test measures or portfolio-based assessment, that they can include in year-end reports and promotion documents. During Fall semester we offered some workshops aimed at helping faculty implement assessment strategies in the classroom and some of these can be used effectively to tell your own story about your teaching in your various reports. The good news: this kind of data (as many have already found) is invaluable for one’s own formative purposes as a teacher, affirming what is working well and shedding light on ways to improve.
In an ideal world, student evaluations of teaching would provide information for teachers that is both useful and valid. In the less-than-ideal world we live in, many faculty will want to think about supplementing student ratings with other types of information about student learning – to ensure that the story you are able to tell others about your teaching is well-rounded and well-informed.
Nilson, L. (2012). Time to raise questions about student ratings. In J. E. Groccia & L. Cruz (Eds.), To improve the academy: Resources for faculty, instructional, and organizational development (pp 213-227). San Francisco, CA: Jossey-Bass.