A Stanford dermatologist explains how AI systems can reinforce racial bias in health care and underperform on skin cancer detection in darker skin.
Roxana Daneshjou, MD, PhD, assistant professor of biomedical data science and
Offering real-world examples in a conversation with Morayo Adisa, MD, medical director of Dermatology Physicians SC in Chicago and Kenilworth, Illinois, Daneshjou was featured alongside Adisa on the latest episode of Skin of Color Savvy: The Art and Science of Treating Patients of Color. This is a podcast hosted by
Daneshjou, who splits her time between clinical dermatology practice and running an AI-focused research lab at Stanford, said her work centers on evaluating whether artificial intelligence performs equitably across patient populations, spanning both imaging-based systems and large language models. She said findings across these studies consistently show that artificial intelligence tools often reinforce existing inequities within health care rather than correcting them.
To illustrate the issue, Daneshjou pointed to a widely cited study led by Ziad Obermeyer, MD, published in Science, which evaluated an algorithm already in use across multiple hospital systems to determine which patients received additional support following discharge.
The analysis found that Black patients needed to be considerably sicker than White patients to receive the same level of allocated resources. Daneshjou explained that the algorithm relied on health care spending as a proxy for illness severity, a flawed measure given that Black patients have historically spent less on health care due to systemic barriers in access, housing, and socioeconomic status, despite often being sicker.
Daneshjou also referenced her own 2022 research evaluating dermatology-focused artificial intelligence algorithms designed to detect skin cancer. Although none of the algorithms studied were in clinical use at the time, they were being discussed as candidates for future deployment.
Her team found that these systems performed significantly worse at identifying skin cancer on brown and Black skin compared with White skin, raising early concerns about scaling such technology without addressing performance gaps across skin tones.
Editor’s note: This episode was summarized with the help of AI tools.
References
