I don’t think I’ve seen this posted up here, but apologies if it has been. Should probably be in the list of fundamental readings for organizations (particularly in healthcare) planning to implement some algorithms as decision aids. This has a wide footprint - including frequently already being used clinical calculators - through to more complex statistical models, applying to either purchased versions or home-cooked.
The first step is to audit the algorithms in use - something I think Waitamata has done - which is not something I believe my organization has done. Then assess the risk of bias. Focus on both calibration to minorities/sub-populations as well as what the authors’ have termed “label bias” - or how closely does the actual objective align with the putative objective. Provides good examples of cases, and how to address issues.
Anyway - attached for reading enjoyment - though does seem a bit of a stich-up for MOH to push across all the DHBs. algorithmic-bias-playbook.pdf (479 KB)