But let’s see where it comes from.
The proliferation of unstructured data in healthcare can also pose data retention, scrubbing and destruction challenges. The problem is not how much data needs to be stored and how long it needs to be stored; instead, it’s where it’s been stored and what’s been stored, says Laberge.
For example, organizations often purge medical records that are inactive or delete research data sets after a study has been completed. With these types of unstructured data, he says, “It’s not just a single database that you’re deleting. There’s likely to be more files and there’s metadata associated with them.”
Work with patient-generated health data
Patient-generated health data comes with its own set of concerns. While it may be available in real time from sources such as monitoring devices or digital therapy apps, and may be structured in its own right, most can only be transferred to EHRs as unstructured summary reports, notes Natalie Schibell, vice president and director. Forrester analyst. (The same goes for visit summaries that come from urgent care, retail health, or telehealth providers not affiliated with a health system.)
In these situations, the valuable nuance of the summary document is largely lost. That doesn’t provide a complete picture of a patient’s health, making it difficult for health systems to sift through their vast stores of data and see which patients need the most care, Schibell says. It also contributes to unnecessary expense, as doctors without readily available results will simply order another test. “There is great risk in duplicative and disruptive care,” he adds.
DISCOVER: How modern data platforms can increase healthcare agility.
Six steps to making unstructured data more meaningful in healthcare
The American Hospital Association has suggested that now is the time for hospitals to transform into data-driven organizations. This will improve clinical and business decision-making, the AHA said, while helping hospitals better serve their patients and their communities in times of need.
Becoming a data-driven organization depends on the ability to derive meaning from unstructured data. While this is a difficult task for many health systems, there are some key steps organizations can take to move forward.
- Optimize storage: Organizations should look at where data is stored and how these storage arrays are synchronized and distributed. Anything that can be migrated to the cloud should be. This will free up space on the site for the most recent and relevant data.
- Sort the data: Data should be structured into groups based on how it will be used, who will have access to it, what level of confidentiality it needs and what security policies apply to it. It is also critical to look at the format of the data and determine if it can, in fact, be structured.
- Sort unstructured data: If unstructured data has clinical or business value, it will benefit from normalization, which aims to make it look more like structured data. “Given the sheer volume of this data, you can’t do it manually,” says Schibell, but artificial intelligence and natural language processing can help.
- Find the context: NLP alone is insufficient to normalize unstructured data, says Laberge. A clinical note may include the word diabetes, but this does not automatically mean that a patient has diabetes. The doctor may have recorded that the patient does not have diabetes or that the patient’s parent has diabetes.
- Code according to industry standards: Once the context of the data is understood, organizations should code as much information as possible according to applicable industry standards such as ICD-10 or SNOMED. This helps structure unstructured data, making it readable and useful for machine learning and analytics models.
- Give direction to data science: Many data scientists do not have a clinical background and may not know, for example, that a diagnosis of type 2 diabetes can be expressed using one of nearly two dozen ICD-10 codes. Clinical teams should provide data science teams with the right guidance before diving into a data set, Laberge says.
As with many large-scale technology initiatives, the secret to success with unstructured data in healthcare is a well-defined scope and use case, Laberge says. Instead of trying to boil the ocean, organizations should focus on a key business metric or other quantifiable area of improvement.
“You need clarity about what you want to get out of the data you have,” says Laberge.
UNTIL NEXT TIME: Unlock this set of data practices for modern data platform success.
Data is one of the most powerful resources available to healthcare professionals today, and understanding the differences between structured and unstructured data is key to making the most of it. At Ikaroa, a full stack technology company, we believe healthcare will benefit greatly from taking advantage of both types of data.
Structured data is organized using data models and is typically stored in a database. It consists of records, each with a number of field and value pairs. Examples of structured data include patient information, laboratory test results and general medical instrument data. The data is easy to search, store and use for analysis, making it the data type of choice for analytics and machine learning.
Unstructured data, on the other hand, is not organized and does not follow standard formats. It consists of images, audio files, free-text documents and other types of uncategorized information. Examples of unstructured data in healthcare include patient patient narratives and hospital radiology reports. Despite the fact that it is more difficult to search, store and analyze, unstructured data is still very important for healthcare professionals as it holds valuable insights.
We are excited about the possibilities that structured and unstructured data can bring to healthcare. Structured data allows healthcare professionals to quickly and efficiently use data for analysis and machine learning processes, while unstructured data offers an expanded and more holistic understanding of patient conditions. By taking advantage of the various types of data available, healthcare professionals can gain unprecedented insights into patient health and leverage them to deliver improved outcomes.
At Ikaroa, we are committed to providing healthcare professionals with the tools they need to make the most of their data. To that end, we are developing products and services tailored to structured and unstructured data, and leveraging our expertise to help healthcare professionals make the most of their data. Our mission is to enable healthcare professionals to gain the insights and cost savings that come with data-driven healthcare.