Pursuing fair artificial intelligence (AI) for healthcare requires collaboration between experts across disciplines, says a global team of scientists at the Singapore-based Duke-NUS Medical School in a new study published in npj Digital Medicine.
While AI has demonstrated potential for healthcare insights, concerns around bias remain. “A fair model is expected to perform equally well across subgroups like age, gender, and race. However, differences in performance may have underlying clinical reasons and may not necessarily indicate unfairness,” explains first author Liu Mingxuan, a PhD candidate in the Quantitative Biology and Medicine program and Centre for Quantitative Medicine (CQM) at Duke-NUS.
“Focusing on equity—that is, recognizing factors like race, gender, etc., and adjusting the AI algorithm or its application to make sure more vulnerable groups get the care they need—rather than complete equality, is likely a more reasonable approach for clinical AI,” says Ning Yilin, PhD, research fellow with CQM and a co-first author of the paper.
“Patient preferences and prognosis are also crucial considerations, as equal treatment does not always mean fair treatment,” Yilin adds. “An example of this is age, which frequently factors into treatment decisions and outcomes.”
The paper highlights key misalignments between AI fairness research and clinical needs. “Various metrics exist to measure model fairness but choosing suitable ones for healthcare is difficult as they can conflict. Trade-offs are often inevitable,” says associate professor Liu Nan, PhD, also from Duke-NUS’ CQM, senior and corresponding author of the paper.
Nan adds, “Differences detected between groups are frequently treated as biases to be mitigated in AI research. However, in the medical context, we must discern between meaningful differences and true biases requiring correction.”
The authors emphasize the need to evaluate which attributes are considered ‘sensitive’ for each application. They say that actively engaging clinicians is vital for developing useful and fair AI models.
“Variables like race and ethnicity need careful handling as they may represent systemic biases or biological differences,” says Liu. “Clinicians can provide context, determine if differences are justified, and guide models toward equitable decisions.”
Overall, the authors argue that pursuing fair AI for healthcare requires collaboration between experts in AI, medicine, ethics, and beyond.
