This is outside of health, but the lessons of this failed long term experiment are highly pertinent for those of us looking to apply AI in the digital health sphere:
Basically, if your problem isn’t a great fit for AI capability, you are almost certainly wasting your time.
I’ve been tinkering with AI (OpenAI via API) to provide meeting summaries here in the eHealth Forum, and have found that it performs really well. However, it is critically dependent upon the quality of the transcription.
And this is where it falls down; the amount of human (i.e. my) effort required to curate the transcription adequately is approximately equal to that of a human simply writing a summary.
A much more effective path is to have a human (better two or three) take notes, and then have the LLM format it nicely, and lastly have it edited / sense checked by the speakers.
The fundamentals of the walk in, walk out approach was a concept that was dependant on the assumption where people were logical and decisive. Too often in supermarkets (as with any shopping experience), a different frame of mind happens when people browse, put it in the cart, then find a better deal, and remove the original item or items.
Without understanding human emotional logic, we fail to comprehend based on linear logic trees that technologists thinks everyone will be following. This then results in billions in investment into failed businesses or workflows.
While my tinkering with meeting summaries and the like has yielded similar results to yours @NathanK, I suspect it won’t be long until improvements (like voice recognition/ transcription) do bounce back to become more efficient (and still accurate), as compared to the alternative.
Among many other things, I think one of the risks here is that if something doesn’t work off the bat, we cast it aside as a failure and don’t revisit it for a long time after the models have improved.
That said, I’m glad I don’t have to worry about someone selling my in-store purchasing habits for a while longer… unless I walk into a countdown facial recognition store and use a loyalty card of course
