MIT, Harvard Scientists Discover AI Can Recognize Race From X-rays — And No One Knows How

A doctor can’t tell if someone is black, Asian, or white just by looking at their X-rays. But a computer can, according to a surprising new paper by an international team of scientists, including researchers from the Massachusetts Institute of Technology and Harvard Medical School.

The study found that an artificial intelligence program trained to read X-rays and CT scans can predict a person’s race with 90 percent accuracy. But the scientists who conducted the study say they have no idea how the computer finds out.

“When my graduate students showed me some of the results in this article, I actually thought it was a mistake,” said Marzyeh Ghassemi, an MIT assistant professor of electrical engineering and computer science, and co-author of the paper, who wrote on Wednesday. was published in the medical journal Lancet Digital Health. “I really thought my students were crazy when they told me.”

At a time when AI software is increasingly being used to help doctors make diagnostic decisions, the study raises the troubling prospect that AI-based diagnostic systems could unintentionally generate racially biased results. For example, an AI (with access to X-rays) can automatically recommend a particular course of treatment for all black patients, regardless of whether it’s best for a specific person. Meanwhile, the patient’s human doctor would not know that the AI ​​based its diagnosis on racial data.

The research effort was born when the scientists noticed that an AI program for examining chest X-rays was more likely to miss signs of disease in black patients. “We wondered, how can that be if computers can’t tell a person’s race?” said Leo Anthony Celi, another co-author and an associate professor at Harvard Medical School.

The research team, made up of scientists from the US, Canada, Australia and Taiwan, first trained an AI system using standard data sets from X-rays and CT scans, with each image labeled with the person’s race. The images came from various parts of the body, including the chest, hand and spine. The diagnostic images examined by the computer did not contain any obvious characteristics of race, such as skin color or hair texture.

After the software showed large numbers of race-labeled images, several sets of unlabeled images were then shown. The program was able to identify the race of people in the images with remarkable accuracy, often well above 90 percent. Even when images of people of the same size or age or gender were analyzed, the AI ​​accurately distinguished between black and white patients.

But how? Ghassemi and her colleagues remain baffled, but she suspects it has something to do with melanin, the pigment that determines skin color. Perhaps X-rays and CT scanners detect the higher melanin content of dark skin and incorporate this information into the digital image in a way that human users have never noticed before. It will take a lot more research to be sure.

Could the test results be evidence of inborn differences between people of different races? Alan Goodman, professor of biological anthropology at Hampshire College and co-author of the book “Racism Not Race,” thinks not. Goodman expressed skepticism about the paper’s conclusions and said he doubted other researchers would be able to reproduce the results. But even if they do, he thinks it’s all about geography, not race.

Goodman said geneticists have found no evidence of substantial racial differences in the human genome. But they do find large differences between people based on where their ancestors lived. “Instead of using race, would the machine do just as well if they looked at someone’s geographic coordinates?” asked Goedman. “My feeling is that the machine would do just as well.”

In other words, an AI could determine on an X-ray that one person’s ancestors came from Northern Europe, another from Central Africa, and a third from Japan. “You call this breed. I call this geographic variation,” Goodman says. (Still, he admitted it’s unclear how the AI ​​could detect this geographic variation on an X-ray alone.)

In any case, Celi said doctors should be reluctant to use AI diagnostic tools that could automatically produce biased results.

“We need to take a break,” he said. “We cannot rush to bring the algorithms to hospitals and clinics until we are sure they are not making racist or sexist decisions.”


Hiawatha Bray can be reached at hiawatha.bray@globe.com. Follow him on Twitter @GlobeTechLab

Leave a Comment