Nothing particularly surprising in this article but it concisely describes the problems, especially in figure 1. A similar exercise in NZ isn’t possible in the absence of a regulator approving the commercial use of AI devices. https://www.nature.com/articles/s41591-021-01312-x.epdf?sharing_token=8BNOnt1UUOf0iPsJ9yU0J9RgN0jAjWel9jnR3ZoTv0M6PlZXWQqbgCrdZtSbNOnPDQlhZJ-fPz8LJ4JqCoxGYshqBh62049hIhMSEfJaE7pKaceG00AD1FUBHLZ5YShokEBQWoF6kBbZitEELPDqWu-9esaFE8DcbdQ1QAgRChw%3D&utm_source=STAT+Newsletters&utm_campaign=fdec4d0d0d-health_tech_4-6-21_COPY_01&utm_medium=email&utm_term=0_8cab1d7961-fdec4d0d0d-152708089
This is interesting, am providing feedback on an assessment model at the moment, but as you say only useful if it is consistently applied.
Jon