This has been running over the last few weeks in the UK. While initiated by NHSX (the soon to be dissolved digital health arm of the NHS), it has been heavily promoted and pushed as a community effort by the Digital Health Networks.
Should we do such a thing here in NZ? It might be extremely helpful to compare ourselves with a UK baseline.
It would take a combination of funding (likely MoH) and community effort (i.e. us) to pull it off.
Looks like a good idea. Will need a bit of careful discussion before embarking on such a big project. If you do pull some possible collaborators, please count me in.
KJ
That is quite an understatement! We are talking about a national EPR audit from the end user perspective. And that has never been done on any scale here in NZ (would love be corrected on this one). Would be an excellent baseline project for HealthNZ @searnshaw @mat.
One quote from Rachel Dunscombe caught my eye. She was reflecting on her experience as a CIO (for a large DHB equivalent) when doing this as a pilot several years ago:
One of the biggest lessons for me was that you can make the vast majority of EPR systems deliver an outstanding user experience; you can also make even the shiniest, most advanced system fall flat if you don’t invest time and effort in the right things. And as we found in Salford (echoed in KLAS’s international findings too), the majority of the factors that make an EPR system a success fall under the organisation’s control.
How would such a project be funded? It would need a team of (relatively expensive) people (researchers and practitioners and senior leaders) to collect, analyse, report on and use the data.
As you say Nathan, a big job - hence one which should not be undertaken if it can’t realistically produce usable results.
A while ago Marcus Baw kindly sent me a ranked list of the various EMRs in the NHS which was extracted from this data… It was very useful when there was discussion about implementing DXCs Lorenzo in the central region.
One fundamental difference we need to consider between the UK and the NZ landscape is that there is much less heterogeneity. i.e. nearly all hospital doctors work on some version or instance of Orion’s Concerto (under various names) while GP practices are nearly all covered by either medtech or Indicie. i.e. it is not reasonable to expect the same ability to rank platforms by user experience like they could in the UK.
It may be worth considering a more pared back attempt - i.e. compare radiology systems
As in the UK, this could only fly with hefty buy-in and lots of $$$ from the centre i.e. the MoH or HealthNZ. It is not something that could be done on the smell of an oily rag.
In theory, a true community-led effort (e.g. CiLN) could possibly achieve such a thing on the cheap. But only if it captures the imagination of the whole community and several individuals donate considerable expertise and time.
Thanks Mat.
The interesting thing about the data generated by previous smaller iterations of this survey is that the platform per se doesn’t seem to be a major factor in user experience.
For example, in the US both the top performing and bottom performing health organisations were both using the Epic platform.
What a survey like this would tell us is who in the country is doing what bits well. And vice-versa. This is likely much more useful than platform comparison anyway.
I think there have been a couple of these surveys in the NHS now. The first one was in 2016 I think (called CSUS) but I never saw the results published. If anyone has a link to the data from the 2016 survey, I’d be interested to see them. It will also be interesting to see the results of the latest survey as well if they are published.
There was an SUS survey of NHS ED systems published: https://emj.bmj.com/content/38/6/410.
The NHS now ask vendors to complete a DTAC with a 10% score attached to usability testing/validation: https://www.nhsx.nhs.uk/key-tools-and-info/digital-technology-assessment-criteria-dtac/ (section 1.3.1). I haven’t seen any reports of DTACs being published although it might be possible to get them on request from NHSX/Digital.
Similarly, the US have a database of certified systems with CIF-reported usability tests (including SUS) for the Safety Enhanced Design (SED) section of the certification process: https://chpl.healthit.gov/#/search
Type in the name of the system you are interested in and you can usually see the CIF usability report if you click through the links: e.g. searching for Cerner - you can find: https://www.drummondgroup.com/pdfs/EHR_Usability_Test_Report_for_PowerChart.pdf
The US have also recently implemented a new system of monitoring safety/usability of medicare implemented EHR systems using the SAFER guides: https://www.healthit.gov/topic/safety/safer-guides
There have been a few research papers about the US system for monitoring EMR usability in JAMA in the last few years:
So it may be possible to get usability data on systems used in NZ if they have either had a DTAC (would probably need a request to NHSX/Digital) or if they are already in CHPL, they should be publicly available.