Joy Buolamwini: The Face Her Software Couldn't See
Joy Buolamwini is a computer scientist and founder of the Algorithmic Justice League, which researches and challenges harmful uses of AI. Her work helped expose large differences in how accurately commercial software classified faces from different groups of people.
A failure in her own art project helped set that work in motion. While studying at MIT, she built the Aspire Mirror to detect a face and display an altered reflection. The software struggled to find hers.
In her account of the experience, she describes how putting on a white mask made the detector respond. She wanted to find out who else these systems were failing, then give people a way to challenge them.
Before the Mask
Buolamwini's route to MIT crossed computer science, education and art. She earned her computer science degree at Georgia Tech, then studied learning and technology at Oxford as a Rhodes Scholar. At MIT, she pursued media arts and sciences, earning a master's degree in 2017 and a doctorate in 2022.
Those interests met in the Aspire Mirror. Writing software was part of the project, but so was exploring what a person might experience while using it. A tool meant to change a reflection first had to notice the person standing in front of it.
The software she was using had failed at face detection: finding whether, and where, a face appears in an image. It wasn't trying to identify her by name. That is a separate task, in which software matches a face to a particular person.
Buolamwini says she'd encountered similar problems before. As facial technology spread into more settings, a failure she'd worked around became harder to dismiss as somebody else's coding issue.
An Average Can Hide a Lot
To investigate commercial facial analysis, Buolamwini teamed up with computer scientist Timnit Gebru. Their 2018 study, Gender Shades, examined another task: assigning a face to one of two gender categories, female or male.
The pair tested products from IBM, Microsoft and Face++. They assembled a benchmark containing photographs of 1,270 parliamentarians, with a better balance of lighter and darker skin than the existing image collections they had examined.
Then they separated the results by both skin tone and the study's gender labels. The highest error rate for darker-skinned female faces reached 34.7%. For lighter-skinned male faces, the highest was 0.8%. A product could perform well for one group while getting answers wrong for another roughly a third of the time.
These figures describe the product versions and photographs tested, not police identification errors. The researchers measured skin tone, which isn't the same as race, and used binary labels that couldn't capture the full range of gender identities.
The study gave other researchers a test they could examine and repeat. It also showed why a company's overall accuracy score could leave an important part of the story untold.
Returning to the Companies
Researcher Inioluwa Deborah Raji and Buolamwini followed up with Actionable Auditing, a study led by Raji. They returned to the original companies' services and added two more to their evaluation.
The original vendors had received advance notice of the Gender Shades findings before being named publicly. Within seven months, all three had released new versions. On the benchmark, the follow-up found smaller performance gaps, with the largest improvements for darker-skinned female faces.
The process borrowed from computer security: document a weakness, tell the vendor, allow a response and make the evidence public. Outside researchers could test a company's claims and then check what changed.
For Buolamwini, better scores were only part of the job. A service could classify a photograph more accurately while being used without the person's consent. Deciding whether a system should be used at all became part of her public work.
Taking the Findings Out of the Lab
Buolamwini calls herself a poet of code, and she has used that side of her work to reach people who may never read a research paper.
Her visual spoken-word piece AI, Ain't I A Woman? examines automated analysis of photographs of prominent women, including Serena Williams, Oprah Winfrey and Michelle Obama. It puts familiar faces alongside the systems' judgments.
Her research and advocacy also reached Congress, where she brought concerns about facial technology in 2019, and the documentary Coded Bias. In 2023, her book Unmasking AI explored how the problem begins before a company tests its finished product.
In an excerpt published by MIT Sloan, she describes how past inequalities can shape the material available for training. Even a decision to gather photographs of public officials brings earlier decisions along with it: who could enter politics, who held office and whose photograph was available.
Adding more images, in her account, doesn't remove the need to ask how the collection was made.
Buolamwini is now founder and president of the Algorithmic Justice League. It combines research with art and public education, and works to give people affected by AI a greater say in its use.
The organization's principles reach beyond a product's test scores. They call for people to have a real choice about whether to interact with AI, to understand how it is used and to have a way to seek redress when it causes harm.
Timnit Gebru Asks Who AI Is Really Built For

Fei-Fei Li: Why Millions of Labeled Pictures Changed AI

Privacy: What Happens to Your Data After You Share It?

QUOTE:
"Amateurs hack systems, professionals hack people."