Teaching Algorithm Auditing to Help Students Detect Bias in AI Outputs

As students integrate AI into their learning at an increasingly rapid pace, they need algorithm auditing to critically assess the outputs it provides—and how much they can trust them.

“The question is, since we’re all interacting with AI systems, how do we know whether the output these systems generate is fair or biased?” says Yasmin Kafai, the Lori and Michael Milken President’s Distinguished Professor at Penn’s Graduate School of Education (Penn GSE). “A one-time interaction might raise some red flags, but in order to figure out whether there is a bias in the results, we need a much more systematic investigation.”

A leading learning designer and researcher, Kafai worked with Danaé Metaxa, the Raj and Neera Singh Term Assistant Professor in computer and information science at the School of Engineering and Applied Science, and high school teachers to develop the “AI Auditing for High School” toolkit to help teachers introduce students to algorithmic bias and guide them through hands-on audits of real-world AI applications. Until very recently, she explains, most people thought that AI systems were too complex for the average person to understand how they work, and thus potential algorithmic bias was a fact of life.

Kafai offers advice for teachers looking to implement algorithm audits in their curriculum, which includes keeping small datasets and incorporate students’ perspectives and interests to diversify data.

Read more at Penn GSE.