Hi everyone,
We are currently reviewing our policy around management of digital health information, and I am keen to know how other organizations manage the destruction of health data older than 10 years (after last entry).
Previously most of our older health records were paper-based and easy to destroy but its more complicated with electronic information.
Is it more common to delete information? or permanently de-identify? how do you identify records that need to be kept for longer than 10 years after last entry?
Looking forward to hearing how you manage this,
Victoria
2 Likes
Great questions @VictoriaBrevoort !
Our computer systems have certainly been in place long enough to have required this policy and management process. I am keen to hear where they have evolved to.
This is one of those fascinating multi-layered problems. Thereās a tech layer and cost to both retention and destruction of data. Thereās the legal layer, the clinical layer, the admin layer, and the tech layer. Possibly even more that I had not thought of, including a potential longitudinal research layer. @VictoriaBrevoort, thanks for raising such an interesting issue.
1 Like
Absolutely agree @Greig, there are many layers and also potential options. The potential impact to research is an area we have been discussing a lot which is where permanently de-identifying health information would meet both the legal layer but provide opportunity to use data - especially that wonderfully structured data that would be a shame to lose permanently.
Also aware that any records that have been part of legal proceedings, reportable or adverse events will need to be retained. I did also read there is a different management process for public and private healthcare information, and the question then is how are organizations doing this in the real world?
1 Like
@helen.lunt has done some thinking about this. I was hoping we might be able to talk to OPC about how we create a data donation service. One of the things on the list.
1 Like
Hi Victoria and Jon.
I agree this is a great question with no clear answer that Iām aware of. We talk a lot about the ādata life cycleā but this is usually talking about the beginning and the middle of this life cycle, not the (less exciting) end-of-cycle! It is indeed a very layered question, with aspects related to legal, tech etc. and also kaitiakitanga / clinical governance.
In the context of clinical research (my area of interest), data is typically collected with participantsā permission for a specified time. There is often an individual (ālead researcherā) obligation, rather than institutional obligation to manage the data life cycle. The NZ ethics process has a subsection on destruction of data, within their research data management section. Te Whatu Ora researchers may indicate they will follow institutional guidelines for destruction, which typically occur at 10 or 15 years after data collection. I suspect however that much clinical research data sits around suffering some form of data rot and is āforgotten aboutā.
I donāt know how many people use the Te Whatu Ora destruction list template: Records Management ā Health New Zealand | Te Whatu Ora ?
The idea of putting anonymised ācoreā research data that is deemed to have ongoing value, into some form of public archive or data repository towards the end of its life cycle is appealing. Maybe there are data archive experts within the HiNZ community who can advise about the threshold for inclusion into the NZ archive system?
For personal health data, how much of the data life cycle question is a systems issue, versus a question for individuals/whÄnau? Many social media sites provide a choice between either deactivating or memorialising a deceased personās account. Maybe we will see more of this type of thinking in the future?
2 Likes
In the central region when we were discussing this issue (without resolution I might add) the principle driver of data destruction was cost of storage. Perhaps Iām missing something here, but my understanding is that we have an obligation to keep data for 10 years, but that we do not have an obligation to destroy data after that periodā¦ is that right?
i.e. moving to a lower fidelity version or a cheaper storage option with the ability to retrieve data with a lag-time if it should subsequently be required were parts of what we planned. In this case we were only talking about radiology images though.
As you indicate Victoria, there are a bunch of reasons why old data can have value.
Yes @Mat that is consistant with what I undertand - that 10 years post last patient contact is the minimum period of storage.
In practice how we deal with this in our region for the paper record is that we keep the paper records for 10 years post patient death (rather than last patient contact).
An exception to this I believe is where there is an enquiry into patient care (such as for hisotric patient treatment in former psychiatric hospitals) where there is a moritorium on destroying patient records for a period even longer than this presumably while the enquiry is not completed.
For electronic records I donāt think we have embarked on a destruction policy yet. While Mooreās law still applies and data storage capacity expands exponentially, it has so far not been a problem to store electronic records indefinatley, and it has probably been easier to keep storing them than to develop a process to destroy them. This will not be the case for ever though, especially as formats that use up greater data such as images and videos become more routinely part of the patient record.
In terms of what we do with patient records after death, I would like to see patients themselves involved in this decision, with the ability to indicate if they want access to these records to be made available to others (eg close family, relatives, whÄnau, iwi, researchers). @jon_herries in the future do you think we could have a field attached to the individualās My Health Account that allows us as health consumers to indicate our wishes here? These wishes could then be read and applied to health records kept by various providers.
2 Likes
@Mat I agree with - we are not REQUIRED to destroy it. Also, disposal schedule were designed at the time where institutions only held their own data. More and more we are holding national data - so the 10 years may only apply to data once they have never touched the health system anywhere in NZ for 10 years.
I think it is probably slightly more complicated than a tick box (consent is more than the tickbox) but I agree we should look at this as it could be really useful for research etc.
To be pedantic, ten years is the standard. Some exceptions must be stored longer: paediatric, maternity, radiotherapy, mental health clinical trial data and raw psychometric data. Many of these are primarily collected in specialist subsystems or, in the case of psychometric tests, administered and scored online. I am sure this all makes sense to someone, and it is not in the slightest sense bizarre to them.
The key argument against historical data retention is that it is a highly imperfect record of the system as it was at the time of collection. Depending on the research question, this may be a significant issue. On a practical level for performance modelling, empirically, we found five years was the right answer. Any longer this historical data unduly influenced the predictions, and the predictions of the system performance as it is now become progressively less accurate.
This parks the reality that data quality and breadth were even worse in the past. You would need a very focused research question to extract current value, and whether this would justify the cost feels dubious even though it is a pain to accept that data ages out of relevance.
Please understand that Research does not conform to Public Records Act requirements by Archives New Zealand. All health agencies are required to be compliant over the general disposal authority guidances, and rules.
My Health Account contemplated the health consumer wishes for data donation, similar to drivers licence declaration as an organ donor, but thatās not likely to be implemented until tranche 3 if that ever gets funded. The main reason is individuals may have a right to donate, but family and whanau consent may be required to meet Maori data sovereignty and cultural belief rules. Itās a lot more complicated than something we can easily implement.
The other side of the coin is that ādata is for lifeā (espoused by the openEHR Community, in particular) and āstorage is infinite and relatively cheapā (a technical perspective). There are also numerous >10 year old healthcare events that remain highly relevant to current day healthcare provision. To cite just a fewā¦
- Adverse Drug Reactions
- Immunizations
- Accidents resulting in permanent impairment
- Illnesses that might result in subsequent immunity
- Family history
- Medical warnings
- Procedures with lasting impact
Many will disagree - but IMO the patientās consent should always be sought before their medical record is effectively truncated.
4 Likes
Thanks everyone so much for your feedback, it was great to read through the discussion and see so many common themes. I think what started out as a relatively simple question - is quite clear that this requires a lot more thinking and especially whether destruction is even the right approach. Especially the comment from @Chris.Dever got me thinking about the role of HIRA and how this may impact data retention in the future.
1 Like
Disposal is definietly the right approach. The law states that we should not keep personal information for longer than it is required. That means that neglecting to dispose of data is unlawfull.
What retention period applies? Data is required for lawful purposes connected with our functions or activities. Those purposes are stated on the consent form the patient agreed to. So, read the patient consent form that was used when the data was collected & determine the stated purpose and from there determine the period required to fulfill that purpose. Consent form->purpose->retention period.
Hope that helps.
This is embroiled in data sovereignty; viewed from the perspective of user/patient ownership of data, my data is being borrowed by a 3rd party (e.g. a health provider). In this case, destruction of my data is a form of assault.
Iād much rather it was returned to me (or at least control ceded to me) and incorporated into my lifelong data repository as per:
1 Like
What is the goal here? applying existing laws and policies or inventing new ones?
The law states that we should not keep personal information for longer than it is required. I can imagine at the reason for this law: If it were retained longer, it increases the damage when itās used for purposes other than what was agreed in the consent. For example, hacked and used for identity theft, blackmail and insurance hikes. Prompt destruction reduces the scope of such risks. Perhaps balance that risk against impact of destroying data after is no longer required for the purpose it was collected.
Iād much rather it was returned to me
So request it. The law states that people have a right to ask for access to their own personal information. ref: https://www.privacy.org.nz/privacy-act-2020/privacy-principles/6/
user/patient ownership of data:
The Privacy Commissionerās guidance on ownership is in: HISO 10064:2017.
Disclaimer: I have a shallow understanding of these topics and am no expert. Every problem is different, local policies vary, consent agreements vary, so itās hard to generalise. I recommend you consult your orgās privacy officer and medical records officer for guidance on a case by case basis.
This is a really good debate. When we have paper notes, the pressure for storage space often drives file destruction. But with digital space there is less pressure so using processes to ensure legislative compliance is important. On the hand, a major consideration is does the information need to stay identifiable? How soon can it be de-identified, and what is considered sufficient for de-identification, especially for patient information of rarer conditions or in small communities.
Another important consideration is how to we ensure that identifiable health information when due for destruction is destroys from all storage sites ā especially with shared systems.
Might be a good question to take to OPC. Donating came up again today in the sense that building things like digital twins probably involves very detailed data about individuals times 5million. I think it would be very hard to deidentify it all for any one person, and so hard to justify (and a bit creepy).
Therefore it feels like donated data (say after you die) maybe like a tissue bank would be a good idea.
Jon