From this morning, the FDA has approved an algorithm for use - is an machine vision algorithm that has “96% accuracy”. Detects pneumothorax for inpatients.
Interesting. How has the claimed accuracy been validated (If at all?)
opengraphobject:[360484514955264 : https://www.cnbc.com/2019/09/12/ges-health-unit-wins-first-fda-clearance-for-ai-powered-x-ray-system.html?&qsearchterm=ge%20healthcare : title=“GE’s health unit wins first FDA clearance for A.I.-powered X-ray system” : description=“The FDA grants clearance to GE Healthcare’s new artificial intelligence-powered X-ray device for patients suffering from a collapsed lung.”]
I don’t know, but that is the best question the reporter could/should have asked 
In talking to the ANZ College of Radiologists we thought that physiological algorithms might be more easily standardised and operationlised than those dealing in human-created or “social” data (of course humans create the machine that creates the x-ray but that is another story involving Terry Pratchet and turtles).
The critical difference being that in a physiological sense we might be able to account for the variation in humanity in a more objective way - we do this at the moment in a simple way by having reference ranges for lab tests (or by having diagnostic criteria for radiological images).
opengraphobject:[360484514955264 : https://www.cnbc.com/2019/09/12/ges-health-unit-wins-first-fda-clearance-for-ai-powered-x-ray-system.html?&qsearchterm=ge%20healthcare : title=“GE’s health unit wins first FDA clearance for A.I.-powered X-ray system” : description=“The FDA grants clearance to GE Healthcare’s new artificial intelligence-powered X-ray device for patients suffering from a collapsed lung.”]
Except ironically we realised only in 2016 how crap our understanding is of the variability in blood pressure:
https://blog.withings.com/2016/11/13/blood-pressure-variability-research-scripps/
opengraphobject:[360484514955264 : https://www.cnbc.com/2019/09/12/ges-health-unit-wins-first-fda-clearance-for-ai-powered-x-ray-system.html?&qsearchterm=ge%20healthcare : title=“GE’s health unit wins first FDA clearance for A.I.-powered X-ray system” : description=“The FDA grants clearance to GE Healthcare’s new artificial intelligence-powered X-ray device for patients suffering from a collapsed lung.”], opengraphobject:[360484819386368 : https://blog.withings.com/2016/11/13/blood-pressure-variability-research-scripps/ : title=“Withings Helps Advance Research on Blood Pressure Variability — Withings” : description=“Discover the first phase of a blood pressure variability study in collaboration with Scripps Translational Science Institute at the AHA Scientific Sessions.”]
Its interesting - just searched the FDA site - and no mention of this - no press release, no listing in devise approvals. I would be interested to see the regulatory approval minutes - I can see this is part of a suite of software tools from other references, but don’t know if devise attached or not. If not, it would be one of the first SaMD approvals, and would be helpful to critically appraise how that is going to look. The draft guidance are pretty loose in terms of thresholds for approval.
opengraphobject:[360484514955264 : https://www.cnbc.com/2019/09/12/ges-health-unit-wins-first-fda-clearance-for-ai-powered-x-ray-system.html?&qsearchterm=ge%20healthcare : title=“GE’s health unit wins first FDA clearance for A.I.-powered X-ray system” : description=“The FDA grants clearance to GE Healthcare’s new artificial intelligence-powered X-ray device for patients suffering from a collapsed lung.”]
Unfortunately that is how it is - the media cycle is much shorter than the bureaucratic one…
opengraphobject:[360484514955264 : https://www.cnbc.com/2019/09/12/ges-health-unit-wins-first-fda-clearance-for-ai-powered-x-ray-system.html?&qsearchterm=ge%20healthcare : title=“GE’s health unit wins first FDA clearance for A.I.-powered X-ray system” : description=“The FDA grants clearance to GE Healthcare’s new artificial intelligence-powered X-ray device for patients suffering from a collapsed lung.”]
But the critical appraisal of novel technology and its impact on health should be more conservative than both. Still awaiting the FDA commentary on this (at least for drugs, this has typically had reference to the actual data that supported the approval), but some interesting observations from other data sources. 1) the GE site - PPV of 35-70% - worth noting that this means it will be correct in identifying a pneumothorax only 1/3 to 2./3 of the time, so this has a reasonably high false positive rate. 2) from the GE site - that PPV is in a population prevalence of 4-15% pneumothorax. This tool is really only limited to very high risk environments - because if you are getting chest x rays and there is a ~10% incidence of pneumothorax, you are talking about a post-procedural, or major trauma, or critically ill population. If the event is rarer, one would assume the false positive rate only goes up, possibly rending the algorithm useless in its intent to reduce radiology read time (if everything is flagged by the algo as high risk, then it just puts a new urgent sticker on all the films). 3) from a Bloomberg press release by GE - the bulk of the software is dedicated to flagging technical issues with the film to the technologist - no data has been published around this function. 4) from the same press release - a footnote in the press release (from GE) - “notification is generated after a delay, post exam closure, and it does not provide any diagnostic information, nor is it intended to inform any clinical decision, prioritization, or action” - it is ambiguous if this footnote applies to the whole software suite or to some functionality, but if the whole suite, it basically divests the software of any “responsibility”.
I would say these innovations carry substantive potential risk, have a novel approach to regualtory approval that is very light, and thus the requirement for assessment of safety and efficacy is defaulting to national decision-makers (e.g. MoH) and individual healthcare provider organizations (DHBs). Are we prepared for this?
opengraphobject:[360484514955264 : https://www.cnbc.com/2019/09/12/ges-health-unit-wins-first-fda-clearance-for-ai-powered-x-ray-system.html?&qsearchterm=ge%20healthcare : title=“GE’s health unit wins first FDA clearance for A.I.-powered X-ray system” : description=“The FDA grants clearance to GE Healthcare’s new artificial intelligence-powered X-ray device for patients suffering from a collapsed lung.”]