Natural Language Processing and Machine Learning to assist radiation oncology incident learning
Image credit: Felix MathewAbstract
Introduction: Radiation oncology incident learning systems (ILSes) are designed to help radiation oncology professionals to identify and learn from radiotherapy incidents. Clinical staff are encouraged to report minor incidents and often mandated to report major incidents in ILSes. Reported incidents are categorized and labelled according to radiation oncology-specific taxonomies designed to help find patterns that can facilitate follow-up actions to improve patient care and reduce the risk of incident recurrence. But the manual classification of such reports is a time-consuming and resource-intensive process and can hinder learning if not done correctly or in a timely manner. Therefore, strategies to reduce the burden of manual incident classification are of interest to the radiation oncology community. Objectives: The purpose of this study is to develop a Natural Language Processing (NLP) and Machine Learning (ML) pipeline that can be integrated into an ILS to assist radiation oncology incident learning by semi-automating incident classification. Our goal was to develop ML models that can generate label recommendations, arranged according to their likelihoods, for three data elements in a radiation oncology incident taxonomy. Methods: Over 6000 incident reports were gathered from the Canadian national ILS as well as our local ILS database. Incident descriptions from these reports were processed using various NLP techniques. The processed data with the expert-generated labels were used to train and evaluate over 500 multi-output ML algorithms. The top three models that performed were identified and tuned for each of three different taxonomy data elements, namely: (1) process step where the incident occurred, (2) problem type of the incident and (3) the contributing factors of the incident. The best performing model after tuning was identified for each data element and tested on unseen data. Results: The Linear SVR model extended with the MultiOutputRegressor technique performed best on the three data elements. On testing, our models ranked the correct label 1.48 ± 0.03, 1.73 ± 0.05 and 2.66 ± 0.08 for process-step, problem-type and contributing factors respectively. Therefore, our models are capable of placing the most appropriate label within the top three options in the dropdown menu on average. Conclusions: We developed NLP-ML models that can perform incident classification. These models will be integrated into our ILS to generate a drop-down menu. This semi-automated feature has the potential to improve the usability, accuracy and efficiency of our radiation oncology ILS.
Felix Mathew
PhD Student
Hui Wang
Undergraduate Student
Logan Montgomery
PhD Student
