Livetweeting the @alisehq @ALISE_MEHC session on Racism and Bias in Student evaluations of teaching with @rdlankes @rchancell and @michellekazmer!

With recorded presentation from Dr. Bharat Mehra!

First question: What is your process for interpreting/reviewing student evaluations of teaching for your courses? #ALISE20

Response by @rchancell : Initially (as an early career faculty) it was devastating. Took everything personally. Numbers were consistently below mean. Then began to analyze comments, and found that comments were often based on student perceptions about her as a person. #ALISE20

Chancellor recalls a conversation w/alum who commented that once they "knew her" they gave her better evals. Said that initially she was "foreign." Now she looks at evaluations more critically. Always room for improvement, but tries to ignore more personal comments. #ALISE20

Response from @michellekazmer : Course evaluations are the only feedback that still cause "butterflies". Now skims for minimum for required annual metrics. Looks for clear indicators. Then creates concept map of open ended responses. #ALISE20

Kazmer approaches course evaluations as research data, tries to ignore personal commentary, and look for trends - implications for practice. #ALISE20

Kazmer mentions that course evaluations are often returned over breaks, meaning that faculty are alone. Would like to review as faculty groups, because sometimes evaluations are about individual instructor, but sometimes they reflect the dynamic of a cohort of students. #ALISE20

Response by @rdlankes: gives mid semester course evaluations, so that students have had a chance to "vent" prior to evaluations. #ALISE20

Question 2: At this point in your career, what role do student evaluations of teaching play in your professional development? #ALISE20

Kazmer: Every 7 years, chance for small salary increase. FSU reviews numbers across units, so individual numbers contribute to comps. across system, etc. "I have not been in the classroom for a very long time." Evals help relay what has changed in course expectations. #ALISE20

Lankes: As an administrator, there is [should be?] a tie between evaluations and professional development. It is important to allow evaluations to inform faculty teaching development (w/center for teaching excellence, for example). Engage support mechanisms. #ALISE20

Chancellor: "I have refined my craft over time." Now do midpoint check, but also allows students to find out if course is working for them - are they getting what they need out of the course? As chair, evals can give some of this information across department. #ALISE20

Chancellor: Sometimes students "needs" might be "outrageous" so we take that with a grain of salt, but evals still tell us what they would like, if they could get it. Helpful for reviewing what they perceive as their needs. #ALISE20

Question 3: Have you noticed a relationship between the type of feedback you receive on your student evaluations and the rigor with which you grad or conduct the classes you teach? #ALISE20

Lankes: No, but he structures courses so students can get an A. Most of the criticism he [personally?] sees is about clarity of expectations. Higher ranked classes often result from strong rel. between instructors & students, even when there is high level of rigor. #ALISE20

Chancellor: Has had situations in which she has received negative course evaluations for not giving As, for not delivering content in the way certain students specifically want to see. [customer service model=bad grade=bad eval] #ALISE20

Chancellor: Has had bad evals because students didn't like the way she spoke, too much rigor, not guaranteed As, course delivery mode. #ALISE20

Kazmer: Always teaches core course - difficult courses. Her sweet spot is "This course is the most work I've ever done but my favorite." #ALISE20

BUT...(she says)

Bias does figure in here.

Kazmer: "I give my students an inordinate amount of me, but I am privileged in that my students are willing to accept the me that I give them." Says that she is aware that she can get away with more than Dr. Chancellor and not as much as Dr. Lankes. #ALISE20

Lankes: Describing experience of a Black fem. faculty member who regularly faced discrimination from students, and who received no support from faculty, and decided to leave. Said other Black female faculty member came to him w/evals to make a plan for dealing with bias. #ALISE20

Question 4: What do you consider the most unfair comment on your evaluation of teaching received so far?

Prefaced with quick rundown of most recent research on bias in student evaluations. #ALISE20

Chancellor: "I didn't learn anything in this class...I wasted my money." + personal attacks on her as in instructor (and a request that she not be rehired). Early in her teaching career. At the time it was devastating. But that was the only bad comment in that group. #ALISE20

Kazmer: Husband is also a faculty member in the same unit as her. Same generic pool of students. Often share students. Student expectations of their work pace/load is drastically different. If he takes 2 weeks to grade, his comments take a grateful tone (for his work/effort). 1/2

Her avg. paper turnaround is 6 days (with twice as many students). Students regularly comment on her inefficiency. Same students regularly comment positively when he allows children to say goodnight, but call her unprofessional for the same interactions. #ALISE20. 2/2

[Telling how much of evaluations are based in gendered/racialized expectations about the instructor, and not actual instructor behavior]

Question5: How do you account for student evaluations in a fair and equitable manner in the tenure process of your faculty, esp. those who belong to minority and/or underrepresented groups? #ALISE20

Kazmer: Need more association statements explicitly condemning use of student evaluations for merit evaluations (promotions, raises, etc.). Increasingly convinced that evals constitute a hostile work environment for faculty of color. #ALISE20

Kazmer: Too many faculty of color come to her and say "look at these evaluations. I don't do anything different than my white colleagues but my numbers are lower." #ALISE20

Chancellor: New at the admin level, but she thinks course evaluations should be eliminated. OR, just for faculty use as a tool for improving teaching. Should not be factored into raises or promotion. #ALISE20

Lankes: As director of school, sees all evaluations for faculty. Also sees as external reviewer. Mostly just the quantitative data that moves forward to tenure. But we also have a strong peer evaluation mechanism. #ALISE20

[I wonder why people think that the numerical evals are somehow unaffected by the same biases that impact the qualitative portions? Plausible deniability? Why wouldn't the same biases be reflected in the quant portion?]

Lankes: Person who reviews tenure cases must compare with other people who teach the same course, so hopefully that accounts for some of the difficulty of core courses. Also, external reviewers have power to say "no I can't evaluate teaching from evals." #ALISE20

Lankes: I don't mind the data, because it is very contextualized.

[I'm not sure that this accounts for the kind of systemic bias we see wrt women and POC]

#ALISE20

Recorded, Dr. Bharat Mehra: As context, has been teaching social justice and inclusion, diversity leadership, community engaged scholarship, etc. Masters & Doctoral students. #ALISE20

American ethic has been, "work hard and you will get rewards." But faculty of color are embedded in a workplace culture where the majority of students prioritize whiteness as comfort. Mostly white female students want, culturally, white Christian women to teach them. #ALISE20

Mehra: Our field's focus on "niceness" encourages an entitled attitude among library students, who want "cookies" for making little effort. Faculty of color who hold these students to account (rigor, discomfort with conversations around race) suffer bc of insecurities. #ALISE20

Mehra: Within a system that prioritizes whiteness, faculty who can align with these cultural, racial, and religious "standards" have a responsibility to not coddle students and reinforce these demands. Faculty of color often have similar experiences wrt this problem. #ALISE20

Mehra: We tell faculty that if they work hard, they will be rewarded, but all of this is outside of the control of faculty of color. #ALISE20

Mehra: Strategies for dealing with this (now as a senior faculty member) - leans into the work of training students to stretch themselves beyond comfort, privilege, whiteness. Also actual results oriented work with agencies/organizations outside the academy. #ALISE20

Mehra: Pushes students outside of their comfort zones to talk about race, gender, sexuality, etc. #ALISE20

Mehra: Many of his students have shown in evals that they want easy As ("cookies and cupcakes" ) and he has had colleagues who give them. This impacts colleagues who teach more difficult courses. [Not sure "talking courses" are always easier, though.] #ALISE20

Mehra: Cupcakes and cookies...fill their brains with sugar.

Pretending that racism and bias don't exist has led us to where we are now. #ALISE20

Mehra: As a senior scholar evaluating junior colleagues, is very sensitive to how bias plays out in course evaluations. Advocates for emphasis on other evidence of effectiveness. #ALISE20

Q to Dr. Kazmer: Are there any professional development resources she'd recommend related to keeping up with student development/literacies/etc.?

A: Largely ad hoc, keeping contact with university Center for Teaching, paying attention to state K-12 environment #ALISE20

Lankes: Faculty does 10 minute teaching discussions at faculty meetings where they share resources. #ALISE20

Q to Dr. Chancellor: What is your advice to faculty of color about how to receive and navigate personal comments, e.g., their accent is terrible, I couldn't understand them. Such comments can be devastating and can prevent us from recruiting and retaining faculty of color. 1/2

Q to Dr. Chancellor contd: Essentially, how do we deal with the microaggressions and racism, some of which is explicit. 2/2 #ALISE20

Chancellor: Have a trusted group of mentors/faculty of color friends to debrief with; bring it to administration; engage discussions about bias in classroom. #ALISE20

Lankes: We also often miss adjunct faculty in this conversation. They are often re-hired (or not) based on student evaluations. Transparency becomes even more important for those positions. Citing adjunct faculty as a way to increase exposure to diversity for students. #ALISE20

Q to Kazmer: How can we encourage our white colleagues to be better allies and accomplices to our faculty of color?

A: On one hand, you can't change people who don't want to be changed. So for some people, you need to blunt their impact. #ALISE20

Kazmer, contd.: Also, we're finding that many trainings don't work. Be consistent - be an accomplice in white only spaces in the same way that you are in public spaces. Tell people what to do - not just what not to do. Practice! Social scripts. Do your background work. #ALISE20

[Social stories - my bad]

Kazmer: Practice intervening when you're not in a high pressure moment so you can intervene publicly.

Colon-Aguirre highlights dynamic in which white "allies" privately express support after the fact, as opposed to speaking up in the moment. #ALISE20

Q to Lankes: David mentioned during Q5 that the quantitative makes it into the faculty evaluation process. That data also be skewed negatively and are as harmful though maybe not as emotionally damaging to the faculty as the evaluation comments. 1/2 #ALISE20

Q, contd.: What would you replace student evaluations with, if anything? 2/2 #ALISE20

Lankes: Acknowledges that quantitative data can absolutely be biased. Is a proponent of highly contextualized data - not quantitative tenure metrics/frameworks. Contextualized student voices are important part of the process. #ALISE20

Kazmer: One of the reasons students get more vigorous in their feedback is that they aren't sure if and how feedback is being used. She acknowledges prior student feedback when she talks through classes. Suggests we do that institutionally as well. 1/2 #ALISE20

Kazmer, contd.: Programs should acknowledge that previous student participation in committees, etc. has influenced current program context. #ALISE20

Lankes: It is important to create mechanisms for students to report very problematic behavior without waiting until the end of the semester. #ALISE20

Q to Mehra: Regarding Bharat's comment about students expecting to get A's. What role might new student orientation play in setting student expectations to not get or even go for an easy A grade? This can be answered by any of the panelists. #ALISE20

Lankes weighing balance between complete standardization of coursework and complete academic freedom (in teaching). Says somewhere around the middle is probably desirable.

[So some level of common expectation of work in core courses but not so far as rubrics, etc.] #ALISE20