AI Model Predicts Inpatient Surgical Discharges

Kyan Safavi, MD, MBA

A machine-learning model accurately predicted inpatient surgical care discharges and their barriers, new study findings showed.

Kyan Safavi, MD, MBA, from the Department of Anesthesia, Critical Care and Pain Medicine at Massachusetts General Hospital, and colleagues used more than 15,000 hospital discharges to develop an artificial intelligence (AI) model which had an area under the receiver operating curve (AUC) of 0.84 and identified 65 barriers to discharge.

The findings suggest that clinicians can use this method in the future to increase the timeliness of patient discharge.

Investigators developed the AI model and trained it on all adult inpatients at least 18 years old who underwent a surgical procedure in the operating room. All patients included in the data set were cared for by a surgical team on a general surgical care inpatient floor and were discharged between May 2016 and August 2017.

To test the AI against a baseline model, the investigators used a population with identical inclusion and exclusion criteria who were discharged between September 2017 and December 2017. The investigators performed an observational cohort analysis on postoperative inpatients admitted and discharged from 2 inpatient general surgery floors from January 2018 to April 2018.

Safavi and the investigative team used clinical and administrative data from a database linked to patients’ electronic health records (EHRs). Demographic, environmental, administrative, and clinical data were included in the model. Keyword search was used to find words from a predefined list in case manager notes, including “home today,” “discharge today,” “is medically ready,” and “referral placed.”

Clinical data were used if they represented a clinical milestone of recovery—an event that marked progress toward discharge or recovery—or a clinical or administrative barrier to discharge. Some milestone categories included transition from intravenous to oral medications; laboratory results or vital signs within reference ranges; and improvement in functional or mobility challenges.

Barriers to discharge included medication administration; new scheduled surgical procedures; and presence of surgical drains, catheters, and wound devices.

A multidisciplinary team including physicians, nurses, and case managers identified the barriers and milestones. The data were then included in the model based on whether data related to the events were available in the database linked to the patient’s EHR.

The estimated out-of-sample AUC was computed after the investigators conducted 10 random experiments. The cohort was randomly distributed to the training (80%) or validation (20%) set.

The neural network model was compared to a baseline model that captured a discharge planning approach aligned with the practice at Massachusetts General Hospital. The baseline model used length of stay of a given procedure to predict when the patient would be discharged.

The training data set included 15, 201 patients (median [interquartile range] age, 60 [46-70] years; 7623 [50.1%] men), and the validation set included 3,843 patients (median [interquartile range] age, 62 [49-72] years; 1882 [49.0%] men).

The mean AUC for the AI model was 0.84 (SD, .008; 95% CI, .839—.844). The neural network had a sensitivity of 56.6%, a specificity of 82.6%, a positive predictive value of 51.7%, and a negative predictive value of 85.2%. Nearly 34% of patients were discharged later than the median length of stay model predicted, including 12.4% within 24 hours after predicted, 6.5% between 24 to 48 hours after predicted, and 14.8% more than 48 hours after predicted.

In the prospective observational analysis on 605 patients, 136 were discharged later than the model predicted. Reasons for delayed discharge included clinical barriers (30.1%), variations in clinical practice (22.1%), and no clinical reason (47.8%).

The investigators suggested that the model could be used in the future to guide process improvement and increase timeliness of discharges through prioritization of patients and barrier identification. Doing this could help hospital address capacity challenges and reduce delays in patient care.

The study, “Development and Validation of a Machine Learning Model to Aid Discharge Processes for Inpatient Surgical Care,” was published online in JAMA Network Open.