U of T Engineering alumna Inioluwa Deborah Raji (EngSci 1T9) has been named to this year’s list of Top Innovators Under 35 by MIT Technology Review. Raji was recognized in the Visionaries category for her research on racial and gender bias in facial recognition services.
“It’s been this whole journey, of getting my research published, of fighting against big companies that didn’t want to change, of advocating against this technology, and so this just feels like a cherry on top,” says Raji. “And a celebration of visible work, of being seen.”
As an intern at MIT’s Media Lab, she worked under computer scientist Joy Buolamwini to audit commercial facial recognition technologies made by companies such as Microsoft, IBM and Amazon. The audit revealed algorithms with baked-in biases across the three services.
Due in part to biases in the data sets used to train them, these facial recognition systems could identify the gender of light-skinned men with nearly 100 percent accuracy, but would misclassify darker-skinned women for men upwards of 34 percent of the time.
Misidentifying faces is especially high-stakes if deployed in policing scenarios, where the technology is used to scan faces on cameras against those in criminal databases.
The mounting concerns, and potential dangers, around commercializing biased artificial intelligence (AI) technologies have led to calls for federal regulation in the U.S. and a growing list of research on the subject, led by researchers such as Buolamwini and Raji.
This work, and the Black Lives Matter movement, seem to have moved the needle — IBM recently announced it was withdrawing its facial recognition software.
“My reaction to IBM was pretty mild because our research group, the Algorithmic Justice League, had been talking to IBM for a while and knew it was winding down and was privately addressing the bias in their models,” says Raji. “The remarkable thing about IBM’s statement was that they took a public, ethical stance.”
Amazon and Microsoft soon followed suit. Amazon announced it was putting a one-year pause on letting the police use its facial recognition tool, while Microsoft stated that it will not sell the tools to police until there is a national law in place.
Though it’s a step forward, Raji says she still holds some skepticism. “It feels like a small win. Amazon’s messaging from when we were advocating against them, to this announcement, there has at least been a shift in tone,” says Raji.
“Ultimately, we shouldn’t depend on the companies to make the decision-making. I think there’s a lot of work to be done to change policy, to really push technology-adoption restrictions to where they need to be.”
Until regulations are in place, Raji is dedicated to keeping companies accountable through her work and activism. During her fourth year at U of T Engineering, Raji worked with Google collaborators as a research mentee, introducing internal auditing practices for AI testing, in the hopes of stopping biases in its track.
Raji is currently a Tech Fellow at the AI Now Institute at New York University, where her study in AI ethics continues.
“I’m at a point where this is where I want to be, this is the area I want to dedicate myself to,” says Raji. “I’m really grateful that this work is visible and seems to be so appreciated by the research community as impactful.”
