Threat Picture

Latest Cybersecurity News

Timnit Gebru Asks Who AI Is Really Built For

Her research questions whose data trains AI and whose lives it affects. She founded the Distributed AI Research Institute to give overlooked communities a greater role in the work.
By Charles Joseph · Updated
Pen-and-ink watercolor portrait of Timnit Gebru with natural curly hair, rectangular glasses and a patterned scarf.
Timnit Gebru. Portrait adapted from a photograph by TechCrunch (CC BY 2.0).
Share
Share
Copy Link

Timnit Gebru is a computer scientist who studies how AI can treat people unfairly. She founded the Distributed AI Research Institute, or DAIR, where communities affected by technology help decide what gets studied.

One of her best-known proposals borrowed an idea from electronics. Parts come with datasheets explaining what they do and how to use them. Gebru and her colleagues proposed similar documentation for the collections of data used to train AI, so researchers could understand what they were building with.

The idea connected two parts of her career. Before investigating training data and facial-analysis software, she'd worked on electronic circuits at Apple.

From Circuits to Street View

Gebru was born and raised in Ethiopia in a family of Eritrean origin. She left during the Ethiopia–Eritrea war and sought asylum in the United States as a teenager.

Engineering was familiar territory in her family. In a 2020 interview, she said her father and two sisters had worked in electrical engineering. Her own path included circuit design at Apple and starting a company before she returned to academic research.

At Stanford, she studied computer vision under Fei-Fei Li. Computer vision gives software ways to extract information from images, whether that means finding a vehicle or distinguishing one model of car from another.

Gebru and her colleagues used that ability to investigate a social question. Could photographs of cars help researchers estimate the makeup of a neighborhood?

Their 2017 study analyzed 50 million Google Street View images from 200 U.S. cities. A team including Jonathan Krause, Jia Deng and Li classified vehicles by make, model and year, then examined their relationship to demographic information.

The work explored a possible supplement to conventional surveys. Its estimates described neighborhoods, not the income or voting choices of individual drivers.

Sponsored

Counting Who Was Missing

After her doctorate, Gebru joined Microsoft Research as a postdoctoral researcher. She later moved to Google, where she became co-lead of the Ethical AI team established by Margaret Mitchell.

By then, she was looking closely at who participated in the field as well as what its software did. In her account, reporting about criminal-risk software and the scarcity of Black researchers in AI helped direct her attention toward fairness.

In 2017, Gebru and Rediet Abebe co-founded Black in AI. The organization works to broaden participation in AI research, development and policy, including helping people reach conferences and professional opportunities.

With Joy Buolamwini, Gebru showed how much a testing decision could reveal. Their 2018 Gender Shades study evaluated three commercial products that assigned faces to female or male categories.

Rather than relying on one accuracy score, they divided results by skin tone and those gender labels. On their 1,270-person benchmark, errors reached 34.7% for darker-skinned female faces, while the highest rate for lighter-skinned male faces was 0.8%.

The task was classifying faces, rather than identifying people, and the binary labels couldn't capture the full range of gender identities. Within that task, the study showed how a single accuracy score could conceal much higher error rates for one group.

Giving a Dataset a Paper Trail

In Datasheets for Datasets, first circulated in 2018, Gebru and six co-authors proposed documenting data before anyone built a system with it. Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé III and Kate Crawford helped develop the proposal.

They asked why the data was gathered, what it contained, how it was collected and labeled, and which uses its creators recommended. Maintenance mattered too: someone needed to explain how the collection would be kept useful.

The authors wanted people to think through those choices, not automate the paperwork. A researcher reading a datasheet could discover a poor fit before relying on the data. Its creators, meanwhile, would have to make their assumptions visible.

When Fluent Text Sounds Like Understanding

Gebru's work at Google brought her into another argument about training data and scale.

A language model learns statistical patterns in language. Among its uses is generating text by predicting what may come next. As models grow, they can produce increasingly fluent passages, but fluency alone doesn't tell a reader how well an answer is grounded.

With linguist Emily M. Bender and their co-authors, Gebru examined that problem in On the Dangers of Stochastic Parrots. Bender and Gebru were joint first authors.

The paper raised concerns about environmental and financial costs, poorly documented web data, inherited biases and research effort concentrated on ever-larger models. It also warned that people could mistake convincing language for understanding.

Published in 2021, the paper acknowledged improvements on measured tasks while arguing for more deliberate data collection and attention to affected communities.

Before publication, a dispute over the draft, Google's review process and workplace concerns ended with Gebru's departure in December 2020. In Reuters' reporting at the time, she said she had been fired and abruptly lost access to company systems. Google AI head Jeff Dean said the company had accepted her resignation after rejecting conditions she set about the paper-review process.

Sponsored

Letting Workers Set the Questions

On December 2, 2021, Gebru founded the Distributed AI Research Institute, or DAIR. She continues to lead work there, with an approach that gives communities affected by technology a role in defining what gets studied. The 2026 Right Livelihood Award recognized that work alongside her criticism of concentrated power in AI.

The DAIR-associated Data Workers' Inquiry shows what that approach can mean in practice. People who label data, moderate content and do related work become community researchers themselves. With guidance on research methods from the organizing team, they choose questions about their working conditions and report through essays, films, comics, podcasts and other formats.

QUOTE:

"Amateurs hack systems, professionals hack people."