Detecting Bias in Language Models

Summary This module introduces the idea of word embedding models and how they encode cultural biases. Students learn about one tool, Word Embedding Association Test, for uncovering these biases. Students apply their understanding to analyze the ethical issues that arise in real-world applications of machine learning.
Topics Ethics in AI, Natural Language Processing, Machine Learning, Word Embeddings
Audience Suitable for any CS course. Ideal for Introduction to AI, Machine Learning, NLP, or Intro CS. Can be used in any course looking to introduce ideas of ethical issues and bias in algorithms.
Difficulty Very low technical difficulty - no programming experience is required although students must be comfortable using the command-line interface. The main difficulty for students is in learning how to frame the ethical issues that arise in the prompts. The entire module can be completed 3-4 hours, of which one or two hours is done in lab/lecture with the instructor.
Strengths The assignment is simple to set up on the backend. No coding is required, though it would be very easy to add some. Students are able to take complex mathematical models of bias and apply them to tangible examples. The exercises are straightforward and students are able to connect real-life examples to abstract philosophical discussions.
Weaknesses As with most ethical discussions, the questions that arise are complex and can frustrate students who desire a straight-forward answer. This assignment is an introduction to the topic and focuses more on identifying ethical issues and bias in algorithms, and not the question of how to address them.
Dependencies The software uses Python and standard libraries (pandas, seaborn, matplotlib, numpy). Students do need to learn how to use the command-line to run the programs, but no programming experience is required (the assignment was developed for a first-year seminar in Philosophy with no pre-requisites). It would benefit students to use this assignment after introducing topics such as bias in algorithms and/or ethics, though this is not required.
Variants This assignment can be adapted for more technical audiences. For example, students could implement the search algorithms and statistical tests given the original paper. Additional ethical case studies could be added and tailored to course content (e.g., facial recognition technology in hospitals for a computer vision course).