How Health Apps Are Biased Before Coding Begins

The article argues that many health apps and AI-powered tools can reflect bias before coding begins because they are shaped by the training data and the assumptions of their developers. It says datasets often represent a narrow slice of the population, which can lead to harm for people who are under-represented. The piece cites examples including skin-condition diagnostic algorithms trained largely on light skin, linked to misdiagnosis in people with darker skin. It also describes feedback loops where biased healthcare delivery influences future model development and can worsen inequities over time. A cited study from U.S. hospitals found an algorithm assigned extra care to under 18% of Black patients, when it should have been over 46%, in part because risk scores tied to annual healthcare expenditure. The authors say marginalized communities are often not consulted early enough.