Analyzing the COMPAS Recidivism Algorithm

by Raechel Walker, Matthew Taylor, Olivia Dias, Zeynep Yalcin, and Dr. Cynthia Breazeal

For more information about this assignment, you can download our Abstract

Summary First, students will be introduced to the historical bias in arrest data by watching the documentary titled, 13th. Then, students will receive an entry level introduction to Python, so they can engage with the AI bias activities. Next, students will analyze the COMPAS algorithm and relay that information to a non-technical audience. Overall, the activities provide students with the technical and critical thinking skills to use data science to analyze bias embedded in an algorithm. The tools used from this assignment can be applied to other AI domains. As a result, this curriculum should be used to increase diversity. Teaching students how to analyze AI algorithms is one step towards teaching students how to use data science to transform society.
Topics: AI Fairness, Responsible AI, and Data Science
Audience: We geared this curriculum towards high school and undergraduate students.
Difficulty: These activities do not require any pre-knowledge in Python or AI.
Strengths: This assignment teaches students how to understand the historical bias in a data set. Also, students learn how to use data science to measure the fairness of different algorithms. One assignment strength is that it has real-world examples. This activity teaches students how to communicate their knowledge about the harms of AI to people that are not data science experts. Another strength of the assignment is how the cumulative assignment design ensures students are equipped with the proper background knowledge to critically analyze how AI can amplify racism. We have piloted these activities with 12 high school students and incorporated their feedback. After students completed our assignments, more students stated that they believe different forms of slavery still exist today in the US. This is likely due to the course focusing on how the modern prison system is derived from the 13th amendment, which abolished slavery except for the case of imprisonment.
Weaknesses: One weakness of the assignment is that it is geared towards high school students. In the future, this curriculum will incorporate block-based programming languages, so younger students can engage with this material. Also, we plan on developing a more descriptive lesson outline to ensure teachers know how to articulate the purpose of each lesson.
Dependencies: Mathematical competencies in finding the average, such as addition, subtraction, multiplication, and division are required to complete the assignment. Also, the Intersectional Data Analysis activity requires Internet connection. If students already know Python and understand systemic racism, specifically the history of mass incarceration, they do not have to complete the 13th or Intro to Python activities. Activities 1-5 are preparing students to complete the “Introduction to the COMPAS recidivism algorithm” activity.
Variants: Instructors are encouraged to change the datasets. The dataset should represent the input and output values from algorithms that demonstrate AI bias. In order to ensure students understand that different fairness metrics should be utilized based on the contexts, there are several fairness metrics instructors should use. While this class focuses on African American history, these lessons can be applied to other contemporary issues. Moreover, we encourage students and teachers to read these books titled, “Race After Technology”, and “Data Feminism” because the books inspired several of these assignments.


Each activity folder contains the instructions and descriptions for the slides that are required to teach students. Please complete the activities in the following order: