Back to 2017 Program
Harnessing full text pathology data from the electronic health record to advance bladder cancer care - Development of a natural language processing system to generate longitudinal pathology data
Florian R. Schroeck, MD, MS1, Olga V. Patterson, PhD2, Patrick Alba, BA2, Scott L. DuVall, PhD2, Brenda Sirovich, MD, MS3, Douglas J. Robertson, MD, MPH3, John D. Seigne, MBBS4, Philip P. Goodney, MD, MS3.
1White River Junction VA Medical Center, White River Junction, VT, and Section of Urology and Norris Cotton Cancer Center, Dartmouth Hitchcock Medical Center, Lebanon, NH, USA, 2VA Salt Lake City Health Care System and the Division of Epidemiology, University of Utah, Salt Lake City, UT, USA, 3White River Junction VA Medical Center, White River Junction, VT, USA, 4Section of Urology and Norris Cotton Cancer Center, Dartmouth Hitchcock Medical Center, Lebanon, NH, USA.
BACKGROUND: Population-based studies to advance bladder cancer care require longitudinal pathology data that allow for measurement of disease recurrence and progression. The prime data source for population-based studies has been SEER-Medicare, but SEER data is limited because pathologic information is only abstracted at time of diagnosis. We set out to obtain longitudinal pathology data by developing a natural language processing (NLP) engine to automate abstraction of important details from full text pathology reports.
METHODS: We selected a national random sample of 600 bladder pathology reports from the Department of Veterans Affairs (VA) Corporate Data Warehouse. These reports were independently annotated by two reviewers with discrepancies resolved by a third to develop a gold standard. We used Cohen's kappa to evaluate inter-rater reliability for histology, invasion (presence versus absence and depth), grade, and statements regarding presence of muscularis propria and of carcinoma in situ. Next, we iteratively trained, developed, and tested the NLP engine’s ability to abstract these variables from the reports. We assessed NLP performance by calculating accuracy, positive predictive value (PPV, precision), and sensitivity (recall) and then applied the NLP engine to pathology reports from 10,725 bladder cancer patients.
RESULTS: The validated engine was capable of abstracting pathologic characteristics for 99% of bladder cancer patients. Inter-rater reliability was excellent between the two reviewers (kappa ranging from 0.82 to 0.90). When comparing the NLP output to the gold standard, NLP achieved the highest accuracy (0.98) for presence of carcinoma in situ. Accuracy for histology, invasion (presence versus absence), grade, and presence of muscularis propria ranged from 0.83 to 0.96. The most challenging variable was depth of invasion (accuracy 0.68; sensitivity 0.65), likely due to the high variability in the language used to describe findings. Nevertheless, we achieved acceptable PPV (0.82; table).
CONCLUSIONS: We developed an NLP engine to accurately abstract important pathologic details from full text bladder cancer pathology reports. This engine allowed for abstraction of data from the vast majority of the 10,725 patients included, enabling us to develop a population-based cohort of patients with longitudinal pathology data. The resulting unique dataset will be used to examine the extent to which bladder cancer care impacts recurrence and progression of disease.
| Variable | Accuracy | PPV (Precision)
| Sensitivity (Recall)
| | Histology | 0.96 | 0.97 | 0.96 | Invasion
Presence
Depth | .
0.87
0.68 | .
0.86
0.82 | .
0.89
0.65 | | Grade | 0.93 | 0.96 | 0.92 | | Muscularis present | 0.83 | 0.89 | 0.79 | | Carcinoma in situ | 0.98 | 0.95 | 0.91 |
Back to 2017 Program
|
|
|
|
|
