key: cord-270286-76mrzaxi authors: Kim, Byunghyun; Kim, Kyuseok; Lee, Jieun; Kim, Joonghee; Jo, Yoo Hwan; Lee, Jae Hyuk; Hwang, Ji Eun title: Impact of bacteremia prediction rule in CAP: Before and after study date: 2018-05-31 journal: The American Journal of Emergency Medicine DOI: 10.1016/j.ajem.2017.10.005 sha: doc_id: 270286 cord_uid: 76mrzaxi Abstract Objective In cases of community acquired pneumonia (CAP), it has been known that blood cultures have low yields and rarely affect clinical outcomes. Despite many studies predicting the likelihood of bacteremia in CAP patients, those results have been rarely implemented in clinical practice, and use of blood culture in CAP is still increasing. This study evaluated impact of implementing a previously derived and validated bacteremia prediction rule. Methods In this registry-based before and after study, we used piecewise regression analysis to compare the blood culture rate before and after implementation of the prediction rule. We also compared 30-day mortality, emergency department (ED) length of stay, time-interval to initial antibiotics after ED arrival, and any changes to the antibiotics regimen as results of the blood cultures. In subgroup analysis, we compared two groups (with or without the use of the prediction rule) after implementation period, using propensity score matching. Results Following the implementation, the blood culture rate declined from 85.5% to 78.1% (P =0.003) without significant changes in 30-day mortality and antibiotics regimen. The interval to initial antibiotics (231min vs. 221min, P =0.362) and length of stay (1019min vs. 954min, P =0.354) were not significantly changed. In subgroup analysis, the group that use the prediction rule showed 25min faster antibiotics initiation (P =0.002) and 48min shorter length of stay (P =0.007) than the group that did not use the rule. Conclusion Implementation of the bacteremia prediction rule in CAP patients reduced the blood culture rate without affecting the 30-day mortality and antibiotics regimen. Blood culture before antibiotics treatment has been a routine diagnostic test in patients presumed to be infectious [1] . However, in patients with community acquired pneumonia (CAP), it has been known that the rate of true positive blood cultures is b 10% [2, 3] . In emergency departments (ED), the true positive results from blood cultures rarely affected patient management [4, 5] . Additionally, ED overcrowding may increase the risk of blood culture contamination [6, 7] . As part of an effort to reduce unnecessary blood culture and improve the yield of blood culture in CAP, there have been many studies describing bacteremia prediction rules [8] [9] [10] . Despite those studies, in a recent study, it has been reported that the blood culture rate in CAP patients is still increasing [11] . Therefore, whether those decision rules clinically impacted the reduction of blood cultures is questionable. For clinical decision rules to have an impact on standard treatment, the clinical decision rules should pass three stages: derivation, validation, and implementation [12, 13] . In our previous studies, we derived the bacteremia prediction model in CAP patients and tested the generalizability of the rule by a multicenter external validation [10, 14] . Here, we evaluated the impact of implementing the bacteremia prediction model on the rate of blood cultures. We hypothesized that implementation may reduce the blood culture rate without changing the mortality rate. Also without blood culture, the time interval from patient presentation in the ED to the first intra-venous antibiotics administration could be shortened as well as the length of ED stay. This study is an uncontrolled before and after study based upon a retrospective review of a patient registry database. Before implementation of the bacteremia prediction rule, a prospective registry of patients diagnosed with pneumonia in the ED had been in place since 2008. The registry includes the patient's baseline characteristics and co-morbidities. Time variables such as ED arrival time and ED discharge time were included in the registry as well as the initial antibiotics administration time. The registry also provided the patient's initial vital signs and laboratory findings including the blood culture results if a blood culture had been performed. Finally, the patient's dispositions from the ED including admission to the ward or ICU as well as the 30-day mortality were included in the registry. This study was performed at a 950-bed tertiary academic hospital with an annual ED census of 90,000 patients and was approved by the institutional review board of the hospital. The implementation of the bacteremia prediction rule was initiated in January of 2015. We retrospectively analyzed the patient registry data from January 2013 to January 2017. Exclusion criteria were the same as in previous studies [10, 14] . In short, we excluded patients who were diagnosed with hospital-acquired pneumonia (HAP), health care-associated pneumonia (HCAP), and ventilator-associated pneumonia (VAP). We also excluded patients who were transferred from our hospital because of difficulty in follow-up and limited information on the changes of antibiotics. For ease of use of the bacteremia prediction rule, we created an Excel file in which the recommendations for blood culture appeared with risk-stratified scores, calculated automatically by entry of variables ( Supplementary Fig. 1 ). Because the resident physicians in the ED changed monthly according to a rotation system with other hospitals, the instructions for the Excel file and the explanations of the prediction rule were introduced on the first Wednesday of every month. All emergency physicians were encouraged to record the score calculated with the Excel file on electronic medical record (EMR) of patients suspected of pneumonia regardless of the performance of blood cultures. The prediction rule use records for each emergency physician were reported as a feedback three times a week. The primary outcome was the blood culture rate before and after implementation of the bacteremia prediction rule. The secondary outcomes included the 30-day mortality rate, the ED length of stay, the time-interval to initial antibiotics after ED arrival, and any changes to the antibiotics regimen as a result of the blood culture findings. For the cost analysis of reduced blood culture usage due to this intervention, we compared the mean cost for blood cultures per patient before and after implementation of the prediction rule. For any positive blood culture results, we thoroughly reviewed the results, including preliminary reporting and final reporting times. In preliminary results, only gram staining results of the species growing in culture bottle were reported. In the final report, the exact names of the identified pathogens were reported with the antibiotics susceptibility test results. For changes to the antibiotics regimen following the blood culture results, we reviewed the daily antibiotics administration with the exact times until 24 h after the final result report time. For adverse events due to contaminations, we defined the unnecessary administration of vancomycin as the start of administration after the preliminary reports and its discontinuation following the identification of contaminants in the final reports. Additionally, unnecessary followup blood cultures were defined as blood cultures performed after preliminary reports that were confirmed as contaminants in the final reports and the follow-up results were negative. The following isolates were identified as contaminants: coagulase-negative Staphylococcus, Bacillus species, Micrococcus species, Corynebacterium species, and Propionibacterium species. Continuous variables were presented as the means and standard deviations. Binomial variables were presented as the frequency of occurrence. Wilcoxon's rank-sum test, the chi-square test, or Fisher's exact test was performed, as appropriate, for comparisons before and after intervention. All P values were 2-tailed, and P values b 0.05 were considered statistically significant. We adopted a piecewise regression discontinuity approach using generalized linear models (using odds for binary results) to evaluate the impact of the intervention. For adjusted intervention effects, we constructed multivariable linear regression models using variables with statistical significance (P value b 0.10) from comparisons before and after the intervention. Additionally, we performed Box-Pierce tests to identify statistically significant autocorrelations in time series models we used (P value b 0.05). Because irrespective of prediction rule, the blood culture rates could be reduced only by education introducing the low yield of blood culture in CAP, we performed subgroup analysis only using the patients after intervention. For the subgroup analysis, we used propensity score matched analysis to compare the groups that use the prediction rule and did not. We compared the blood culture rate, time-interval to initial antibiotics, and ED length of stay of both groups. All analyses were performed using STATA (version 13; StataCorp, College Station, TX) and any patients with missing variables were not included in analyses. Of the 5178 registered patients during study period, a total of 4130 patients were included in the study after 1048 patients were excluded (Fig. 1 ). Among the 4130 eligible patients, 2480 and 1650 patients were assigned to the before and after intervention groups, respectively. Among the 1650 patients after intervention, 602 patients (36.5%) had the bacteremia score record on their EMR. Table 1 summarizes the baseline characteristics of the two groups. After implementation of the bacteremia prediction rule, the blood culture rate decreased from 85.5% to 78.1%, representing 7.4% reduction (95% confidence interval [CI] = 2.6 to 12.2, P = 0.003) ( Table 2) . Additionally, the monthly trend of the blood culture rate changed from increasing to decreasing (Fig. 2) . After model adjustment, the blood culture rate decreased from 82.1% to 77.6%, which represents 4.5% reduction (95% CI = 0.1 to 8.9, P = 0.044) reduction. The monthly trend of the blood culture rate still appeared to decrease after implementation of the intervention ( Table 2 ). After the implementation, the 30-day mortality rate decreased from 12.6% (95% CI = 10.6 to 14.6) to 7.3% (95% CI = 5.1 to 9.5, P b 0.001). However, after model adjustment, this difference was not statistically significant (P = 0.182) ( Table 3) . The time-interval to initial antibiotics after ED arrival decreased from 231 min to 221 min but did not reach statistical significance (P = 0.362). After model adjustment, no statistically significant difference was found between the two groups (Table 3) . After implementation, the ED length of stay also decreased from 1112 min to 954 min but the difference between the two groups was not statistically significant (Table 3) . Table 4 shows the blood culture results before and after intervention. The overall blood culture contamination rate was 1.5% (48/3026). After the implementation, the blood culture contamination rate was not changed (P = 0.689). Antibiotics regimen changes based on positive blood culture results were not significantly different between the two groups (41.5%; 49/118 before implementation vs. 38.0%; 19/50 after implementation, P = 0.181). Among the 48 patients with contaminations, 47.9% (23/48) had to undergo unnecessary phlebotomy for follow-up blood cultures and 12.5% (6/48) were administered unnecessary vancomycin treatment. After score implementation, the total adverse event rates decreased from 68.7% (22/32) to 50.0% (8/16). However, this difference did not reach statistical significance (P = 0.297). In the propensity score matched subgroup analysis, we compared two groups that used the prediction rule and did not, during intervention period (Supplementary Table 1 ). The baseline characteristics of the two groups were similar but, the group with the bacteremia score had the lower blood culture rate than the group that did not use the bacteremia score (54.6% vs. 72.5%, P b 0.001). Additionally, the group that use the prediction rule had shorter time-interval to initial antibiotics (198 min vs. 223 min, P = 0.002) and reduced ED length of stay (371 min vs. 419 min, P = 0.007). In our time series analysis study, we reported that the implementation of the bacteremia prediction rule successfully reduced the blood culture rate without any significant 30-day mortality and antibiotics regimen changes. Blood culture technique is time-consuming and resource-consuming considering the need for an aseptic technique and venous access to more than two sites [15, 16] . Especially in the ED, it is well known that blood culture contamination is common, and many EDs have higher contamination rates than Clinical and Laboratory Standards Institute (CLSI) recommendation of 3% [17] [18] [19] . False positive results from blood cultures are associated with increased medical cost and hospital length of stay [20, 21] . Clinically, blood culture contaminations result in unnecessary antibiotics administration [15] . In one previous study, a 20% of additional unnecessary glycopeptide administration in pseudobacteremia cases was observed, which is consistent with our results (Table 4 ) [22] . As multidrug resistance pathogens in pneumonia have been increasing, concerns exist about treatment failure without initial blood cultures [23] . However, in a recent study, multidrug resistance pathogens were uncommon in CAP patients, which accounted for 1.9% [24] . According to our bacteremia score system, blood cultures could be omitted in patients in the low-risk group, of which the true bacteremia rate was b3% [10] . In this study, the true bacteremia in the low-risk group was 2.3% (51/2676) and antibiotics step up due to multidrug resistance organisms isolated from blood cultures was 0.5% (13/2676). Concerning the adverse event rate due to false positive blood cultures of 0.9% (24/ 2676), it seems reasonable to omit blood culture in the low-risk group (Table 4) . Contrary to our hypothesis, the time to initial antibiotics and ED length of stay tended to decrease, but no statistically significant differences were measured between the before and after intervention groups (Table 3 ). According to the sepsis guideline, initial antibiotics should be administered within 1 h after recognition of sepsis [25] . For patients with severe initial presentation, ED physicians may urge the initiation of IV antibiotics and blood cultures without adherence to the bacteremia score. Calculation of the bacteremia score takes N1.5 h because the scoring system includes albumin and C-reactive protein results. This fact may be why the bacteremia score adherence group had mild presentations and lower Pneumonia Severity Index (PSI) scores (Supplementary Table 1 ). For patients with mild symptoms and stable initial vital signs, ED physicians may wait for the laboratory tests before the initiation of IV antibiotics. Those patients may be stratified into the low-risk group for bacteremia and could have more chances to avoid unnecessary blood cultures of low yield. Therefore, by the time a decision had to be made regarding initiating IV antibiotics without blood culture, N90 min had already passed, and the time-saving effect of omitting blood cultures may have been diminished. This study has several limitations. First because this study was a time series before and after study based on a retrospective review of a registry. Missing not at random (MNAR) which occupied a small fraction (116/4130; 2.8%) but not included in analysis, may have created a bias. Also there could be many confounders influencing the results besides the intervention. Second, the baseline characteristics of the two groups before and after intervention were significantly different. The Middle East Respiratory Syndrome coronavirus (MERS-CoV) outbreak in 2015 may have increased the sensitivity of the ED visit population to respiratory symptoms, which may be one of the reasons accounting for the difference [26] . Third, the adherence rate for the bacteremia score was 36.5% which may be too low to expect clinical and statistical significance. Generally, evidence-based changes in the clinical practice of physicians take a substantial amount of time and effort [27, 28] . In addition to those periods of adaptation, the time needed for calculation of the score make a decrease of adherence inevitable in severe patients for whom blood cultures and IV antibiotics were urgent. Additionally, the calculation of the score with multiplication and addition is difficult and the use of Excel file could be another cumbersome step to emergency physicians in a crowded ED. Finally the target population only included patients with CAP and this would limit the broad application of the Table 4 Blood culture results with contamination and antibiotics regimen changes before and after implementation of bacteremia prediction model. ⁎ Adjusted by variables with statistical significance (P value b 0.1): age, sex, history of diabetes mellitus, hypertension, heart failure, cerebrovascular disease, renal disease, liver disease, and known neoplasm, systolic blood pressure, respiratory rate, pulse rate, body temperature, white blood cell count, hematocrit, platelet count, glucose, albumin, BUN, PSI, admission rate. a The numbers of patients with any missing variables used in analysis was 116 and those patients were not included in analyses. rule in the clinical field. Despite those limitations, we could observed a statistically significant reduction in blood cultures after intervention without affecting 30-day mortality as well as decreasing trends in the initial antibiotics administration time and ED length of stay. Considering the low adherence rate to the score system, a more thorough education and feedback is needed to achieve statistically significant clinical impacts on the time of initial antibiotics administration and ED length of stay. In conclusion, the implementation of the bacteremia prediction rule in CAP patients reduced the blood culture rate without affecting the 30day mortality and antibiotics regimen. And bacteremia prediction score was independently associated with a reduction in initial antibiotics administration time and ED length of stay. Further prospective multicenter studies should be performed to evaluate the impact of wide implementation of the clinical prediction rule. None. The clinical significance of positive blood cultures: a comprehensive analysis of 500 episodes of bacteremia and fungemia in adults. I. Laboratory and epidemiologic observations Do emergency department blood cultures change practice in patients with pneumonia? Selective use of blood cultures in emergency department pneumonia patients Clinical impact of blood cultures taken in the emergency department Do peripheral blood cultures taken in the emergency department influence clinical management The impact of overcrowding on the bacterial contamination of blood cultures in the ED Reducing blood culture contamination in community hospital emergency departments: a multicenter evaluation of a quality improvement intervention Predicting bacteremia in patients with community-acquired pneumonia A prediction rule for estimating the risk of bacteremia in patients with community-acquired pneumonia Bacteremia prediction model using a common clinical test in patients with community-acquired pneumonia Blood culture use in the emergency department in patients hospitalized for community-acquired pneumonia Implementation of clinical decision rules in the emergency department Framework for the impact analysis and implementation of Clinical Prediction Rules (CPRs) Bacteremia prediction model for community-acquired pneumonia: external validation in a multicenter retrospective cohort Contaminant blood cultures and resource utilization. The true consequences of false-positive results Guidelines on blood cultures Reducing blood culture contamination in the emergency department: an interrupted time series quality improvement study Reducing blood culture contamination rates in the emergency department Clinical and economic impact of contaminated blood cultures within the hospital setting Impact of blood cultures drawn by phlebotomy on contamination rates and health care costs in a hospital emergency department Clinical significance of potential contaminants in blood cultures among patients in a medical center Risk factors for drug-resistant pathogens in community-acquired and healthcare-associated pneumonia Epidemiology and predictors of multidrug-resistant community-acquired and health care-associated pneumonia The 2016 surviving sepsis campaign sepsis guideline Preventive behaviors by the level of perceived infection sensitivity during the Korea outbreak of Middle East Respiratory Syndrome in 2015 From best evidence to best practice: effective implementation of change in patients' care What drives change? Barriers to and incentives for achieving evidence-based practice None. Supplementary data to this article can be found online at https://doi. org/10.1016/j.ajem.2017.10.005.