A recent study published in npj Digital Medicine reveals that the performance of machine learning (ML) models in diagnosing bacterial vaginosis (BV) varies significantly across different ethnic groups. Conducted by researchers from the University of Florida in Gainesville, the study focused on assessing the capabilities of four ML algorithms to diagnose BV, particularly exploring fairness in predictions for asymptomatic BV among Asian, Black, Hispanic, and White women using 16S rRNA sequencing data.
The study, led by Cameron Celeste, identified notable variations in the performance of general-purpose ML models based on ethnicity. The research indicated that the models were least effective for Hispanic and Asian women when evaluating false-positive (FPR) or false-negative rates (FNR). Overall, the models exhibited the highest and lowest performance for White and Asian women, respectively.
The authors of the study emphasized the importance of their findings, stating, “Here, we show that several supervised learning models perform differently for ethnic groups by assessing commonly used metrics, such as balanced accuracy and average precision, as well as more clinically relevant metrics, such as FPR and FNR, in a cohort of women with asymptomatic BV. The results provide evidence that there is a discrepancy in model performance between ethnicities.”
