key: cord-0072184-e3mwobyg
authors: Liao, Kuang-Ming; Liu, Chung-Feng; Chen, Chia-Jung; Shen, Yu-Ting
title: Machine Learning Approaches for Predicting Acute Respiratory Failure, Ventilator Dependence, and Mortality in Chronic Obstructive Pulmonary Disease
date: 2021-12-20
journal: Diagnostics (Basel)
DOI: 10.3390/diagnostics11122396
sha: 6aaec3b5ea837cf25c9e55c0e2fb42b2298afbec
doc_id: 72184
cord_uid: e3mwobyg

Chronic obstructive pulmonary disease (COPD) is one of the leading causes of mortality and contributes to high morbidity worldwide. Patients with COPD have a higher risk for acute respiratory failure, ventilator dependence, and mortality after hospitalization compared with the general population. Accurate and early risk detection will provide more information for early management and better decision making. This study aimed to build prediction models using patients’ characteristics, laboratory data, and comorbidities for early detection of acute respiratory failure, ventilator dependence, and mortality in patients with COPD after hospitalization. We retrospectively collected the electronic medical records of 5061 patients with COPD in three hospitals of the Chi Mei Medical Group, Taiwan. After data cleaning, we built three prediction models for acute respiratory failure, ventilator dependence, and mortality using seven machine learning algorithms. Based on the AUC value, the best model for mortality was built by the XGBoost algorithm (AUC = 0.817), the best model for acute respiratory failure was built by random forest algorithm (AUC = 0.804), while the best model for ventilator dependence was built by LightGBM algorithm (AUC = 0.809). A web service application was implemented with the best models and integrated into the existing hospital information system for physician’s trials and evaluations. Our machine learning models exhibit excellent predictive quality and can therefore provide physicians with a useful decision-making reference for the adverse prognosis of COPD patients.

Chronic obstructive pulmonary disease (COPD) is a long-term, systemic inflammatory disease. It leads to the destruction of the lung parenchyma, small airway inflammation, and fibrotic changes [1] . After the lung tissue becomes damaged and scarred, pulmonary fibrosis ensues, resulting in irreversible airflow and airway obstructions even after bronchodilator treatment [2] . COPD is the third leading cause of death in the world with 3.23 million mortalities in 2019 alone [3] . One of the most important treatment goals for patients with COPD is to reduce the occurrence of acute exacerbations [4] . Frequent acute exacerbation results in the rapid decline of lung function [5] , leading to frequent hospital admission, acceleration of disease progression, acute respiratory failure, ventilator dependence, and increased risk for mortality [6] . Previous studies used different factors to predict acute respiratory failure [7] , ventilator dependence [8] , and mortality [9] .

Many healthcare systems and design practitioners need an evidence-based approach to facilitate treatment planning and decision making [10] . Artificial intelligence and machine learning improving access to care, increasing accuracy, decreasing cost, and providing the greatest value enable physicians more efficiency to treat patients [11, 12] .

Many healthcare systems and design practitioners need an evidence-based approach to facilitate treatment planning and decision making [10] . Artificial intelligence and machine learning improving access to care, increasing accuracy, decreasing cost, and providing the greatest value enable physicians more efficiency to treat patients [11, 12] .

Over the past years, artificial intelligence and machine learning approaches have been acceleratingly applied in medicine and the health care system. Machine learning approaches can improve care allocation and risk prediction in breast cancer [13] , stroke imaging [14] , heart failure diagnosis, classification [15] , and readmission risk [16] and assessment of coronary artery disease in cardiac computed tomography [17] . Computer science is drastically improving and being combined with new algorithms based on neural network methods, allowing enormous progress in the development of machines capable of performing tasks for disease research and prediction [18] . Previous studies used artificial intelligence for the respiratory medicine [19] [20] [21] [22] [23] and diagnosis of COPD [24] , while some looked into environmental factors, lifestyle data, and symptoms for the early detection of acute exacerbation in COPD [25] . However, there is currently no study that investigated the possibility of having acute respiratory failure, ventilator dependence, and mortality after hospitalization in patients with COPD using modern artificial intelligence and machine learning. Our study aims to characterize high-risk COPD patient groups and identify the factors that potentially increase the risk for acute respiratory failure, ventilator dependence, and mortality using machine learning for patients with COPD.

This retrospective study collected the data of patients with COPD (pulmonary obstruction) with either emergency, outpatient, or inpatient orders from the three hospitals of Chi Mei Medical Group in Taiwan (1 medical center, 1 regional hospital, and 1 district hospital), from 1 January 2010 to 31 December 2019, whose first six diagnosis codes are ICD-9: 490, 491,492, 496 or ICD-10: J41, J42, J43, J44. Data of patients who were less than 20 years old at the time of diagnosis and those with incomplete records, missing values, and ambiguous values were excluded from the study. Overall, data from 5061 patients were included for the predictive model analysis (Figure 1 ). 

We chose 3 outcome variables to establish the prediction models: (1) mortality (inhospital), (2) acute respiratory failure (in-hospital), and (3) ventilator dependence (continuous use of a respirator for 21 days during hospitalization).

Furthermore, based on literature evidence and clinical experience, this study included multiple factors (features) of demographics and clinical information that affect the three outcome variables, including age, gender, BMI, vital signs of body temperature (BT), pulse, Glasgow Coma Scale (GCS) and respiratory rate (RR; the worst record in-hospital), SPO2 (the worst record in-hospital), WBC, Hb, platelet, BUN, CRP, Na, K, PH, Pao2, Paco2, Hco3, and presence of comorbidities (e.g., diabetes, hypertension, pneumonia, etc.).

For maximizing model performance, we used all the variables (28, usually available in clinical) to build our prediction models without performing any feature selection process in advance. The data were randomly stratified into a training dataset for model building (70%) and a testing dataset for model validation (30%). The overall research process is shown in Figure 1 . Because there were fewer positive classes (outcomes to be predicted such as death, etc.) in the clinic, the SMOTE method (synthetic minority over-sampling technique) [26] was used to improve the data imbalance in the training dataset. Each outcome was paired with 7 machine learning algorithms to build the predictive models. These algorithms include: (1) logistic regression, (2) random forest, (3) Support Vector Machine (SVM), (4) K-nearest neighbor (KNN), (5) LightGBM, (6) XGBoost, and (7) multilayer perceptron (MLP). Python and Scikit.learn machine learning tools were used.

Grid search with 5-fold cross-validation was used for tuning hyperparameters to build the best models based on the training dataset. After a model was built, we then used the testing dataset to validate the models with well-defined model performance indicators of accuracy, sensitivity, specificity, and AUC (area under the receiver operating characteristic curve). The model with the highest AUC value was regarded as the best model and exported as model file (PKL file in Python) for further implementing a prediction system for practical use.

There were 38,480 raw samples collected; after applying the exclusion criteria, a total of 5061 were used for model building and analysis. Table 1 shows that the mean age of patients was 77 years old and that 67% were males while 33% were females. Spearman correlation analysis (see Figure 2 ) identifies the correlation between the outcome and each feature. For mortality and ventilator dependence, the most relevant features were SPO2, BUN, and GCS, while for acute respiratory failure, the most relevant features were Paco2, PH, and GCS. In this study, three outcome prediction models were established for predicting mortality, acute respiratory failure, and ventilator dependence. We used seven machine learning algorithms to build the three models. We used grid search with 5-fold cross-validation for each algorithm to obtain the best hyperparameters and build the final production models. The results showed that the XGBoost algorithm obtained the highest AUC value (0.817) for mortality as its prediction model (see Table 2 and Figure 3 ), the random forest algorithm had the highest AUC value (0.804) for acute respiratory failure (see Table 3 and Figure 4 ), and LightGBM algorithm had the highest AUC value (0.809) for ventilator dependence (see Table 4 and Figure 5 ).

In addition, feature importance graph refers to techniques that assign a score to input features based on how useful they are at predicting a target variable in graphic form. Thus, during the model training process, the feature importance graph with criteria of information gain was utilized in Python to judge the importance of each feature to the model. It showed that the SPO2, GCS, and gender were the three most important features of the mortality model (see Figure 6 ); PH, SPO2, and GCS were the most important features of the acute respiratory failure model (see Figure 7) ; and SPO2, GCS, and RR were the most important features of the ventilator dependence model (see Figure 8 ). In this study, three outcome prediction models were established for predicting mortality, acute respiratory failure, and ventilator dependence. We used seven machine learning algorithms to build the three models. We used grid search with 5-fold cross-validation for each algorithm to obtain the best hyperparameters and build the final production models. The results showed that the XGBoost algorithm obtained the highest AUC value (0.817) for mortality as its prediction model (see Table 2 and Figure 3 ), the random forest algorithm had the highest AUC value (0.804) for acute respiratory failure (see Table 3 and Figure 4 ), and LightGBM algorithm had the highest AUC value (0.809) for ventilator dependence (see Table 4 and Figure 5 ). In addition, feature importance graph refers to techniques that assign a score to input features based on how useful they are at predicting a target variable in graphic form. Thus, during the model training process, the feature importance graph with criteria of information gain was utilized in Python to judge the importance of each feature to the model. It showed that the SPO2, GCS, and gender were the three most important features of the mortality model (see Figure 6) ; PH, SPO2, and GCS were the most important features of the acute respiratory failure model (see Figure 7) ; and SPO2, GCS, and RR were the most important features of the ventilator dependence model (see Figure 8 ). To verify the usefulness and feasibility of our models in clinical, we asked the hospital's information system department to design a web service application with our best models and integrate it into the existing hospital information system (HIS, here, is inpatient physician ordering system). The predictive models were implemented in Python language, while the web service was implemented in MS Visual Studio .NET tool. Figure 9 shows the snapshot of the web service application. To verify the usefulness and feasibility of our models in clinical, we asked the hospital's information system department to design a web service application with our best models and integrate it into the existing hospital information system (HIS, here, is inpatient physician ordering system). The predictive models were implemented in Python language, while the web service was implemented in MS Visual Studio .NET tool. Figure 9 shows the snapshot of the web service application. Figure 9 . A snapshot of AI web service application for predicting outcomes of in-hospital COPD patients.

Once a COPD patient is hospitalized, the physician can press the "AI button" in the existing inpatient physician ordering system when they need AI assistance, and a risk prediction webpage of the patient will automatically appear ( Figure 9 ) without the need to manually enter patient information (features). The webpage graphically displays the risk probabilities of death, MV dependence, and acute respiratory failure; the probability greater than or equal to 50% means the risk tends to occur while less than 50% means tends not to occur-the greater the probability, the higher the risk. As shown in Figure 9 , the AI predicts the patient will not die or have MV dependence and acute respiratory failure while hospitalized because the risks are all below 50%.

We demonstrated this AI web service application to three thoracic physicians for evaluation and trial. They all gave a high degree of acceptance relating to the graphic interface and reasonable risk values and believed that it was a significant help for clinical decision making.

The features in this study included the most demographic data and clinical information, such as body mass index, gender, age, blood pressure, body temperature (BT), pulse, respiratory rate (RR), oxygen saturation level, GCS, and laboratory data (i.e., white blood cell, hemoglobin, platelet, blood urea nitrogen, creatinine, C-reactive protein, sodium, potassium, Alanine transaminase, glucose, PH, PaO2, PaCO2, and HCO3). We also included common and important comorbidities such as diabetes, hypertension, cardiovascular disease, congestive heart failure, and pneumonia. All of the mentioned features were employed to predict the outcome models (in-hospital mortality, acute respiratory failure during hospitalization, and ventilator dependence). These features are commonly used in clinical practice and are important markers in a patient's health; thus, there was Once a COPD patient is hospitalized, the physician can press the "AI button" in the existing inpatient physician ordering system when they need AI assistance, and a risk prediction webpage of the patient will automatically appear ( Figure 9 ) without the need to manually enter patient information (features). The webpage graphically displays the risk probabilities of death, MV dependence, and acute respiratory failure; the probability greater than or equal to 50% means the risk tends to occur while less than 50% means tends not to occur-the greater the probability, the higher the risk. As shown in Figure 9 , the AI predicts the patient will not die or have MV dependence and acute respiratory failure while hospitalized because the risks are all below 50%.

We demonstrated this AI web service application to three thoracic physicians for evaluation and trial. They all gave a high degree of acceptance relating to the graphic interface and reasonable risk values and believed that it was a significant help for clinical decision making.

The features in this study included the most demographic data and clinical information, such as body mass index, gender, age, blood pressure, body temperature (BT), pulse, respiratory rate (RR), oxygen saturation level, GCS, and laboratory data (i.e., white blood cell, hemoglobin, platelet, blood urea nitrogen, creatinine, C-reactive protein, sodium, potassium, Alanine transaminase, glucose, PH, PaO2, PaCO2, and HCO3). We also included common and important comorbidities such as diabetes, hypertension, cardiovascular disease, congestive heart failure, and pneumonia. All of the mentioned features were employed to predict the outcome models (in-hospital mortality, acute respiratory failure during hospitalization, and ventilator dependence). These features are commonly used in clinical practice and are important markers in a patient's health; thus, there was no need for the physician to deliberately arrange the examination. Moreover, this study analyzed systemic diseases and applied them to predict the prognosis of patients with COPD.

To our knowledge, this is one of few studies that used machine learning and big data techniques to practically predict the likelihood of patients with COPD to acquire acute respiratory failure, become ventilator dependent, and have increased chances for mortality after hospitalization. This study was the first to analyze the daily available data on patients with COPD from the system of three Chi Mei hospitals, including the medical center, regional hospital, and community hospital using different machine learning approaches (i.e., logistic regression, random forest, SVM, KNN, LightGBM, ML, and XGBoost) to select the best models to predict a patient's outcome (i.e., acute respiratory failure, ventilator dependence, and mortality). The performance of each model was assessed using sensitivity, specificity, and AUROC metrics. This is by far the most comprehensive study that used machine learning models to predict COPD outcomes.

Goto et al. [27] used the National Hospital and Ambulatory Medical Care Survey ED data to identify patients with COPD exacerbation. They employed routinely available triage data as predictors (e.g., patient characteristic, arrival mode, vital signs, chief complaint, comorbidities, etc.) and four machine learning-based models (i.e., Lasso regression, random forest, boosting, and deep neural network) and compared them with traditional logistic regression. In addition to patient characteristics, vital signs, and comorbidities, our study added more laboratory data, utilized seven machine learning-based models, and focused on acute respiratory failure, ventilator dependence, and mortality as outcomes.

Peng and colleagues [28] used C5.0 decision tree classifier to predict the prognosis of patients with COPD with acute exacerbation. Their overall accuracy was 80.3%, with 95% CI (0.6991, 0.8827) and Kappa of 0.6054. The models established in this study had an accuracy of more than 80%.

In addition, Shah et al. [29] recruited 110 patients and followed them for more than 35,000 days. They used a finite-state machine-based approach to predict the acute exacerbation of COPD during home monitoring. They found that vital signs obtained from a pulse oximeter (i.e., respiratory rate, pulse rate, and oxygen saturation) could predict exacerbation events and that oxygen saturation was more predictive than respiratory rate and pulse rate. Our study also integrated these vital signs and the results showed that they improved the positive predictive accuracy of the machine learning. We summarized the comparison of these works in Table 5 .

Previous studies mostly focused on acute exacerbation of COPD [30] [31] [32] and little attention has been paid to the impact of acute respiratory failure, ventilator dependence, and mortality. All of these outcomes are important for patients, patient's families, and the medical team. The lack of patient-centered care and early prediction of respiratory failure and mortality could contribute to poor outcomes, suboptimal use of medical resources, and deteriorated psychological sequelae of patients and family members [33] . In addition to appropriate diagnosis and management of COPD [34] , it is important to determine the possible outcomes of patients with COPD after hospitalization as early as possible so that prompt and effective treatment could be given to improve their prognosis. One of the most important tasks of a physician is to inform the patient and their loved ones about the seriousness of their condition [35] . The information may alter a patient's view of their future and cause additional stress [36] . Therefore, more precise and early detection methods are needed to ensure that the information about a patient's prognosis is as accurate as possible so as not to add more stress and anxiety to the patient and their family [37] . The current study used common and important features to predict COPD prognosis. Further, the machine learning techniques that we established could provide physicians an opportunity to develop algorithms that integrated complex interaction factors to offer different possible prognoses to patients with COPD [38] . The addition of patient demographics, laboratory data, and comorbidities in this study to predict the possible outcomes of patients with COPD was successfully modeled. From the optimal models, we were also able to identify the important features that could affect the patients' outcomes. The results indicated that SpO2 and GCS were the most important features for acute respiratory failure and ventilator dependence with the addition of gender for mortality; thus, compared with other features, they are the most crucial outcome predictors.

Our study also has some limitations that need to be addressed. First, although our data included different hospital levels, from a medical center to a regional hospital, to represent different disease severity, these only came from three hospitals in Taiwan. Future studies could include more hospitals for a more representative sample. Second, the models were constructed based on the Asian population with COPD; thus, it may not be accurate for the Caucasian population. Third, smoking history and duration were considered important risk factors in COPD prognosis; however, this study was not able to include them because of failure to accurately retrieve this information from the electronic health record.

To build a generic machine model to help physicians and support their diagnosis of disease progression and risk of death for patients with COPD, this study developed a machine learning classifier using patients' features such as basic health indicators, comorbidity indicators, and inflammatory indicators. We also implemented and integrated a web-based predictive application into the existing HIS and obtained high acceptance from physicians after initial use. We believe that predicting the adverse outcomes of patients with COPD using machine learning algorithms is a promising research approach to help physicians assess the severity of the disease after hospital admission at the earliest possible time. These could guide them in choosing the most appropriate treatment strategies to improve the prognosis of their patients. For follow-up study, researchers can include more potential variables and perform a feature selection process to improve the quality of the models. Informed Consent Statement: Informed consent from patients was waived due to the retrospective nature of the study.

The dataset used for this study is available on request to the corresponding author.

The authors declare no conflict of interest.

COPD Guidelines in the Asia-Pacific Regions: Similarities and Differences. Diagnostics 2021

Airflow obstruction: Is it asthma or is it COPD?

The Top 10 Causes of Death

Changes in forced expiratory volume in 1 second overtime in COPD

Severe acute exacerbations and mortality in patients with chronic obstructive pulmonary disease

Acute hypercapnic respiratory failure in patients with chronic obstructive lung disease: Risk factors and use of guidelines for management

Disease severity at the first hospitalization as a predictor for mechanical ventilation dependency in elderly patients with chronic obstructive pulmonary disease

Predictors of mortality in COPD

Advancing evidence-based healthcare facility design: A systematic literature review

Lessons Learned About Autonomous AI: Finding a Safe, Efficacious, and Ethical Path Through the Development Process

Machine Learning for Precision Psychiatry: Opportunities and Challenges

Diagnostics 2021

Improving breast cancer care coordination and symptom management by using AI driven predictive toolkits

Deep into the Brain: Artificial Intelligence in Stroke Imaging

Machine learning in heart failure: Ready for prime time

A machine learning model to predict the risk of 30-day readmissions in patients with heart failure: A retrospective analysis of electronic medical records data

Machine Learning for Assessment of Coronary Artery Disease in Cardiac CT: A Survey

How artificial intelligence could transform emergency department operations

Artificial Intelligence/Machine Learning in Respiratory Medicine and Potential Role in Asthma and COPD Diagnosis

Artificial intelligence and machine learning in respiratory medicine

Applications of artificial intelligence and machine learning in respiratory medicine

Artificial Intelligence and Machine Learning in Chronic Airway Diseases: Focus on Asthma and Chronic Obstructive Pulmonary Disease

Artificial intelligence in pulmonary medicine: Computer vision, predictive model and COVID-19

Machine Learning Methods for the Diagnosis of Chronic Obstructive Pulmonary Disease in Healthy Subjects: Retrospective Observational Cohort Study

Acute Exacerbation of a Chronic Obstructive Pulmonary Disease Prediction System Using Wearable Device Data

Synthetic minority over-sampling technique

Machine learning approaches for predicting disposition of asthma and COPD exacerbations in the ED

A Machine-learning Approach to Forecast Aggravation Risk in Patients with Acute Exacerbation of Chronic Obstructive Pulmonary Disease with Clinical Indicators

Exacerbations in Chronic Obstructive Pulmonary Disease: Identification and Prediction Using a Digital Health System

Characterization Associated with the Frequent Severe Exacerbator Phenotype in COPD Patients

Prediction of exacerbation frequency of AECOPD based on next-generation sequencing and its relationship with imbalance of lung and gut microbiota: A protocol of a prospective cohort study

Genetics plays a limited role in predicting chronic obstructive pulmonary disease treatment response and exacerbation

Family response to critical illness: Postintensive care syndrome-family

The Importance of Appropriate Diagnosis in the Practical Management of Chronic Obstructive Pulmonary Disease

Shared decision-making in the ICU from the perspective of physicians, nurses and patients: A qualitative interview study

Delivering Bad or Life-Altering News

Assessing patients' preferences for breaking Bad News according to the SPIKES-Protocol: The MABBAN scale

Forecast the Exacerbation in Patients of Chronic Obstructive Pulmonary Disease with Clinical Indicators Using Machine Learning Techniques