The Language of Caring: Digital Human Modeling, Practice Patterns, and Performance Assessment 2351-9789 Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of AHFE Conference doi: 10.1016/j.promfg.2015.07.881 Procedia Manufacturing 3 ( 2015 ) 3788 – 3795 Available online at www.sciencedirect.com ScienceDirect 6th International Conference on Applied Human Factors and Ergonomics (AHFE 2015) and the Affiliated Conferences, AHFE 2015 The language of caring: digital human modeling, practice patterns, and performance assessment J.R. Hotchkissa,b,c,*, J.D. Paladinod, C.W. Brackneya,b, A.M. Kaynarb, P.S. Crookee aVeteran’s Affairs Healthcare System, Pittsburgh, PA 15240, USA bDepartment of Critical Care Medicine, University of Pittsburgh, Pittsburgh, PA 15213, USA cVeteran’s Engineering Resource Center, Pittsburgh, PA 15215, USA dDepartment of Medicine, John A. Burns School of Medicine, Honolulu, HI 96813, USA eDepartment of Mathematics, Vanderbilt University, Nashville, TN 37240, USA Abstract Digital human modeling offers unique potential in educating providers to apply complex, titratable forms of medical care and assessing their cognitive competence in these domains. Mechanical ventilation uses a machine (a ventilator) to support patients who cannot breathe independently, and is a cornerstone of modern intensive and emergency medical care. This cognitively complex, titrated, and potentially harmful therapy saves hundreds of thousands of lives per year. Practical and ethical considerations limit the provision of extensive bedside training, and there are no current mechanisms for assessing operational competence. We constructed a comprehensive digital model of patients undergoing mechanical ventilation that was populated with “virtual patients,” as well as specific guidelines regarding clinical goals for each patient. Individuals ranging from experienced clinicians to trainees were evaluated regarding their performance as they managed the virtual patient population. The training experience was well received and required less than 2 hours. Nonetheless, exposure to the simulator improved provider efficiency, and was accompanied by clear changes in patterns of practice. The ability to test on rigorously standardized cases (entirely unfeasible in the clinical setting) facilitated assessment of competence and more sophisticated quantification of. © 2015 The Authors. Published by Elsevier B.V. Peer-review under responsibility of AHFE Conference. Keywords: Mechanical ventilation; Virtual patients; Practice patterns; Performance assessment * Corresponding author. E-mail address: john.r.hotchkiss@gmail.com Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of AHFE Conference http://crossmark.crossref.org/dialog/?doi=10.1016/j.promfg.2015.07.881&domain=pdf 3789 J.R. Hotchkiss et al. / Procedia Manufacturing 3 ( 2015 ) 3788 – 3795 1. Introduction Mechanical ventilation (MV) is a cornerstone intervention in modern intensive and emergency care that is used to support hundreds of thousands of individuals each year while they cannot breathe independently. Unfortunately, this lifesaving intervention can cause harm: injudicious ventilator settings can promote lung injury, compromise circulatory stability, produce patient distress, stimulate an inflammatory response, and prolong the period of support required. Such adverse consequences of mechanical ventilation add to the burden of patient suffering, increase healthcare resource utilization, and compromise outcomes. The physiologic and engineering foundations of mechanical ventilation are relatively well understood [1-12]. Regrettably, the teaching of mechanical ventilation remains primarily a “bedside” exercise more akin to an apprenticeship than a systematic approach to mastery. New learners cannot practice extensively on actual patients, for ethical and practical reasons, and physiologically realistic alternatives (large animals or physical simulators) are expensive and suffer from limited access. Moreover, the exposure of the practitioner to the full spectrum of possible mechanical or physiologic derangements cannot be guaranteed. Contemporary approaches to assessment of expertise in mechanical ventilation are ill-suited for defining clinician practice patterns or competence in the context of a potentially harmful intervention for which any patient problem may have many possible solutions, the prevailing physiology is highly dynamic, and the clinician is reasoning in the setting of uncertainty. However, there is recent evidence that model-based training can have an impact on the proficiency of clinicians [13-18]. We addressed these issues by refining and extending an existing simulation-based educational model of mechanical ventilation and coupling this software to a state of the art approach to characterizing practice patterns, one based on symbolic dynamics [19, 20]. This ensemble included a computer based micro-simulation training tool and software and algorithms for constructing a database, characterizing provider practice patterns. We explored the evolution of the practice patterns adopted by individual providers as they progressed through a training exercise in which they confronted 100 virtual patients having common clinical derangements of respiratory mechanics. An updated version of the simulation tool freeware can be downloaded from: http://www.math.vanderbilt.edu/ ~pscrooke/CANVENT/upload.html. 2. Methods 2.1. Simulation tool The simulation tool comprises 5 distinct simulation based elements: Element one: mathematical models that faithfully emulate airspace mechanics during mechanical ventilation. The mathematical models that underlie the simulator are based on general models of non-passive (patient active to varying degrees) mechanical ventilation under pressure controlled (PCV) or volume controlled (VCV) mechanical ventilation. The primary model has been parameterized and tested in a large animal model of lung injury, and are based on a representation of the pulmonary pressure-volume curve (lung recruitment) originally proposed and validated in humans [21]. In this approach, lung compliance is represented as a trapezoidal (increasing, constant, and decreasing) function of lung volume [22, 23]. We studied this model in an oleic acid swine model, and found it to faithfully emulate the dynamic behaviors of this large animal model of very severe lung injury [24]. Element two: a model emulating gas exchange during mechanical ventilation. We developed a simple “two- compartment” model of pulmonary gas exchange that captures relevant behaviors based on a “perfused, ventilated” compartment and a “perfused, unventilated” compartment. The “size” of the unventilated compartment is determined by the volumes predicted from the mechanics model above. Element three: models emulating acid-base metabolism and the effects of elevated intrathoracic pressure We developed a simple model of CO2 clearance and systemic pH that captures relevant behaviors based on CO2 production, minute ventilation, and anatomic deadspace . Similarly, we incorporated a simple model of interactions between elevations in intrathoracic pressure and decrements in cardiac output that is used to emulate mean arterial pressure responses to elevated intrathoracic pressure. 3790 J.R. Hotchkiss et al. / Procedia Manufacturing 3 ( 2015 ) 3788 – 3795 Element four: a population of “virtual patients” that faithfully emulate the behaviors of patients managed in everyday clinical practice. Manipulation of patient specific impedance parameters (such as inflection points, gain values, oxygen consumption, etc.) was undertaken to construct a population of virtual patients having physiologic characteristics mimicking common clinical derangements: Chronic Obstructive Lung Disease (COLD) Severe Acute Asthma (SAA) Mean Airway Pressure Responsive Acute Lung Injury (rALI) Mean Airway Pressure Unresponsive Hypoxemia (UH) Restrictive Lung Disease (RLD). Several iterations were performed in which experienced Critical Care clinicians confronted each simulated patient; those displaying grossly unrealistic behaviors, or that were deemed so easy to be uninformative, were replaced by alternate candidates. We sought virtual patients that were both ultimately “solvable,” and non-trivial in the manipulations required for solution. The simulator sequentially presented 100 patients, of which 80 (16 from each pathophysiologic class) were unique and an additional 5 of which (1 from each class) appeared 4 times (at the beginning of the simulation, and after 31, 67, and 95 patients). These “recurring patients” allowed evaluation of user responses to identical patients at different points in the educational experience. Element five: a user friendly interface in which learners confront sequential patients, attempt to satisfy specified goals, and can terminate the simulation when they believe goals have been met. Following (potentially multiple) iterations of ventilator adjustments, mode selections, and fluid management decisions, the user commits to the solution or determines that goals cannot be met. Immediate feedback is provided. 2.2. Assessment tool For each ventilator adjustment imposed on each patient, the simulator archives the patient type and impedance and other characteristics, the current values of each physiologic variable, and requests for ancillary data. Similarly, the software archives the exact values for changes in ventilator settings made by the user. These data, collected for each learner, comprise the inputs of the assessment toolkit. This toolkit provides two broad classes of performance data, detailed below. Analysis of provider solution speeds, success rates, and response patterns. Gross outcomes, such as the number of attempts to solve each patient, number of successful solutions, and complication rates (unsatisfactory physiologic parameters within a trial) are calculated directly. In addition, the complexity of each intervention imposed by the learner is quantified in two ways: average complexity and weighted complexity. Average complexity is simply the number of ventilator settings that the practitioner changes at each attempt, divided by the number of relevant attempts. Weighted complexity “weights” each setting change by the number of outcome measures that parameter can affect—for example, changes in tidal volume can affect pH, plateau pressure, oxygenation, and blood pressure (weight = 4), whereas changes in inspired oxygen concentration typically only affect oxygenation (weight=1). These metrics were constructed to capture the tendency of more experienced practitioners to respond with patterns of interventions, rather than unitary changes. Quantitative analysis of provider practice patterns. For each point in each simulated management problem, the prevailing pattern of derangements of the patient (such as hypoxemia, low pH and high plateau pressure combined with a low blood pressure: the failure pattern) can be assigned a unique numerical symbol. Similarly, the provider responses (such as decreasing tidal volume, increasing PEEP, increasing respiratory frequency, administering a fluid bolus, or combinations thereof) can also be assigned a unique numerical symbol. The simulation can thus be cast as a series of aligned symbols: “the provider saw this pattern of derangements, and responded with the following pattern of interventions.” 3791 J.R. Hotchkiss et al. / Procedia Manufacturing 3 ( 2015 ) 3788 – 3795 Table 1. Attempt/failure patterns. These data are used to construct provider- and population- specific frequency tables depicting the frequency with which a provider (or population of providers) responds to a specific derangement pattern with a specific pattern of interventions. Approaches promulgated by Tang and Daw [25, 26] can be used to construct difference matrices expressing the “distance” between practice patterns. Such difference matrices can provide quantitative measurements of the distance between a provider’s practice patterns and those of other providers or those of a consensus panel of providers. 3. Results We studied 29 subjects ranging from trainees to experienced faculty members. Each managed 100 virtual patients; completion of the training experience took each subject approximately 1.5 hours of interaction time (1.41 ± 0.59 h). User acceptance was good, with 97% of users agreeing that the virtual patients resembled those in their daily practice and 84% indicating that they would apply the knowledge gained in this experience to their daily practice. The resulting database contained 14,503 disorder: intervention dyads. Incomplete pairs resulting from keystroke errors accounted for 0.5% and were excluded, leaving 14,426 dyads for analysis. Two subjects had keystroke errors in the last of the standardized patient panels (96-100) rendering these standardized sets not evaluable. Accordingly, performance on the standardized sets was assessed using patients 1-5 and 68-72 and performance on previously unseen patients was conducted using patients 6-31 and 42-67. When the two individuals with incomplete terminal sets were excluded from the analyses and the remaining 27 individuals were evaluated using patients 1-5 and 96-100 (standard patient panels) and patients 6 through 27 and 73 through 95 (previously unseen patients), the results of the analyses were not substantively changed. All results are corrected for multiple comparisons within the study. Simulation based training increases practitioner efficiency. Following simulation based training practitioners solved a panel of standardized patients with fewer attempts on each patient. Rates of successful solution were similar (as planned- patients were “designed” to be solvable), suggesting increased efficiency (Figure 1). For the 27 provider subset comparing performance on standardized patients 1-5 and 96-100 the corresponding corrected p value was 0.0003. Fig. 1. Evolution of performance on standardized patient sets. Panel A: number of attempts required to solve standardized set before and after training; Panel B: success rates before and after training. Attempt 1 2 3 4 Failure Pattern 14 18 9 17 Provider Pattern 904 83 172 811 3792 J.R. Hotchkiss et al. / Procedia Manufacturing 3 ( 2015 ) 3788 – 3795 Fig. 2. Evolution of performance on novel patient sets. Panel A: number of attempts required to solve individual patients before and after training; Panel B: success rates before and after training. The increased efficiency seen on standardized patient panels generalized to new patients. Following simulation based training, practitioners solved panels of more difficult patients they had not previously encountered with fewer attempts and similar or increased success, suggesting that the increased efficiency seen in the standard panel generalized to “novel” encounters (Figure 2). For the 27 providers subset comparing performance on patients 6 through 27 and 73 through 95 the corresponding corrected p value is 0.003. Providers adopt more sophisticated practice patterns following simulation-based training. Following simulation based training, practitioners implemented significantly more complex patterns of adjustment at each change in ventilator settings. “Complexity” is simply the average number of parameters changed at each step. “Weighted complexity” is the sum of setting changes at each intervention, with each setting change weighted by the number of outcome parameters that are affected by that setting. For example, frequency can affect minute ventilation, plateau pressure, oxygenation, minute ventilation, and mean arterial pressure; changes in FiO2 only affect oxygenation. Practitioners qualitatively changed their patterns of practice (Figure 3). For the 27 provider subset comparing performance on standardized patients 1-5 and 96-100 the corresponding corrected p-values are 0.04 and 0.12. Fig. 3. Average complexity of interventions imposed by each user before and after training. Panel A: Average complexity; Panel B: Average weighted complexity. 3793 J.R. Hotchkiss et al. / Procedia Manufacturing 3 ( 2015 ) 3788 – 3795 Fig. 4. Similarity of individual practice patterns to the patterns adopted by high performing individuals. Panel A: similarity on standardized patient sets; Panel B: similarity on novel patients. Simulation based training leads to providers adopting higher performance practice languages. As previously demonstrated, practice patterns during management of mechanical ventilation display language-like characteristics [20]. Accordingly, we defined a consensus “practice language” based on the patterns of the 5 most effective subjects, and compared the remainder of the subjects to these patterns before and after simulation training. The “most effective” designation was determined as those subjects having the lowest values for: . Following simulation based training, the practice patterns of the remaining subjects converged toward those of highly efficient providers, whether on a standardized panel of patients or on panels of previously unseen patients, suggesting that practitioners were learning a more efficient “language” (Figure 4). For the 27 provider subset comparing practice patterns on standardized patients 1-5 vs 96-100 and 6-27 vs 73-95 the corresponding corrected p-values are 0.02 and 0.001. Although not a primary outcome within the analysis, within the standard sets learners demonstrated a qualitative trend toward increased efficiency across each of the different patient classes. This finding was observed when both the full data and the 27 learner data were examined. Table 2. Attempt/failure before and after training. Class Average Attempts (before training) Average Attempts (after training) p-value (corrected) 1 3.6 2.2 0.02 2 5.7 2.9 0.01 3 2.8 1.4 0.82 4 4.3 2.2 0.23 5 5.2 3.6 0.05 A similar analysis was not possible for the novel simulated patients, as the degree of difficulty of these patients was not constant (by design). Accordingly, patients could not be “matched” for inherent difficulty for comparison before and after training. 3794 J.R. Hotchkiss et al. / Procedia Manufacturing 3 ( 2015 ) 3788 – 3795 4. Discussion Users of this micro-simulation based training tool for mechanical ventilation increased their solution efficiency, implemented more complex patterns of intervention, and converged toward a common “expert practice pattern” as they progressed through the simulations and were confronted with a rigorously standardized testing panel of simulated patients. These changes in performance and practice pattern were mirrored by similar changes as the users confronted “novel-“ not seen before – virtual patients. Of note, learners were exposed to a large volume of cases spanning a wide clinical range in a very short period- on average less than 2 hours. This work highlights unique advantages of digital human models as training tools. First, the trainee can “practice” in an environment that poses no threat to patient safety. Second, an adequate, high volume exposure to the entire range of clinical problems can be assured. Our results suggest that, in the setting of mechanical ventilation, such exposure can be accomplished in a short time frame. The individual cases can be rigorously standardized, facilitating intra-and inter- individual comparisons. In addition, such standardization, combined with complete data capture, facilitates the definition of practice patterns and allows the implementation of practice pattern based assessment tools. As real world medical care requires the apprehension of multi element patterns of patient derangement and multicomponent provider interventions, such pattern based competency assessment is likely more appropriate than currently employed, univariate assessments. Properly deployed digital human models are uniquely suited for training practitioners to manage patients with complex medical conditions requiring titrated care. In addition, such models allow characterization of “expert” practice patterns, facilitating assessment of competence along multiple simultaneous axes. More sophisticated training tools, such as those including user-tailored training (where the cases focus on the user’s weak points) are also readily implemented. The future is bright for such approaches. Acknowledgements This research was supported by NIH R01 HL084113 and AHRQ R18 HS023453-01. References [1] Burke, W.C., et al., Comparion of mathematical and mechanical models of pressure-controlled ventilation. Journal of Applied Physiology, 1993. 74(2): p. 922-933. [2] Crooke, P.S., J.D. Head, and J.J. Marini, A general two-compartment model for mechanical ventilation. Mathematical and Computer Modelling, 1996. 24(7): p. 1-18. [3] Crooke, P.S., et al., Mathematical models of passive, pressure-controlled ventilation with different resistance assumptions. Mathematical and Computer Modelling, 2003. 38(5-6): p. 495-502. [4] Crooke, P.S. and J.J. Marini, A nonlinear mathematical-model of pressure preset ventilation - Description and limiting values for key outcome variables. Mathematical Models & Methods in Applied Sciences, 1993. 3(6): p. 839-859. [5] Hardman, J.G., et al., A physiology simulator: validation of its respiratory components and its ability to predict the patient's response to changes in mechanical ventilation. British Journal of Anaesthesia, 1998. 81(3): p. 327-332. [6] Knorzer, A., C. Schranz, and K. Moller, Evaluation of a Model based Optimization Algorithm for Pressure Controlled Ventilation. Biomedical Engineering-Biomedizinische Technik, 2013. 58: p. 2. [7] Kulish, V. and V. Kulish, Human Respiration: Anatomy and Physiology, Mathematical Modeling, Numerical Simulation and Applications. Human Respiration: Anatomy and Physiology, Mathematical Modeling, Numerical Simulation and Applications. 2006: Wit Press, Ashurst Lodge, Southampton So40 7aa, Ashurst, UK. [8] Morgenstern, U. and S. Kaiser, Mathematical-modeling of ventilation mechanics. International Journal of Clinical Monitoring and Computing, 1995. 12(2): p. 105-112. [9] Schranz, C., et al., Model-based setting of inspiratory pressure and respiratory rate in pressure-controlled ventilation. Physiological Measurement, 2014. 35(3): p. 383-397. [10] Schranz, C., et al., Model-based ventilator settings in pressure controlled ventilation. Biomedical Engineering-Biomedizinische Technik, 2013. 58(Suppl. 1). [11] Batzel, J.J., M. Bachar, and F. Kappel, Mathematical Modeling and Validation in Physiology Applications to the Cardiovascular and Respiratory Systems Preface, in Mathematical Modeling and Validation in Physiology: Applications to the Cardiovascular and Respiratory Systems, J.J. Batzel, M. Bachar, and F. Kappel, Editors. 2013, Springer-Verlag Berlin: Berlin. p. V-IX. 3795 J.R. Hotchkiss et al. / Procedia Manufacturing 3 ( 2015 ) 3788 – 3795 [12] Hotchkiss, J.R., P.S. Crooke, and J.J. Marini, Theoretical interactions between ventilator settings and proximal deadspace ventilation during tracheal gas insufflation. Intensive Care Medicine, 1996. 22(10): p. 1112-1119. [13] Barsuk, J.H., et al., Unexpected collateral effects of simulation-based medical education. Academic Medicine, 2011. 86(12): p. 1513-1517. [14] McGaghie, W.C. and P.M. Fisichella, The science of learning and medical education. Medical Education, 2014. 48(2): p. 106-108. [15] Motola, I., et al., Simulation in healthcare education: A best evidence practical guide. AMEE Guide No. 82. Medical Teacher, 2013. 35(10): p. E1511-E1530. [16] Rosenthal, M.E., et al., Achieving housestaff competence in emergency airway management using scenario based simulation training - Comparison of attending vs housestaff trainers. Chest, 2006. 129(6): p. 1453-1458. [17] Schroedl, C.J., et al., Use of simulation-based education to improve resident learning and patient care in the medical intensive care unit: A randomized trial. Journal of Critical Care, 2012. 27(2): p. 7. [18] Singer, B.D., et al., First-year residents outperform third-year residents after simulation-based education in critical care medicine. Simulation in Healthcare-Journal of the Society for Simulation in Healthcare, 2013. 8(2): p. 67-71. [19] Paladino, J., et al., The language of caring: quantitating medical practice patterns using symbolic dynamics. Mathematical Modelling of Natural Phenomena, 2010. 5(3): p. 165-172. [20] Paladino, J.D., et al., Medical practices display power law behaviors similar to spoken languages. BMC Medical Informatics and Decision Making, 2013. 13: p. 9. [21] Svantesson, C., B. Drefeldt, and B. Jonson, The static pressure-volume relationship of the respiratory system determined with a computer- controlled ventilator. Clinical Physiology, 1997. 17(4): p. 419-430. [22] Kongkul, K., S. Rattanamongkonkul, and P.S. Crooke, A multi-segment mathematical model with variable compliance for pressure controlled ventilation. ScienceAsia, 2004. 30: p. 191-203. [23] Crooke, P.S., J.J. Marini, and J.R. Hotchkiss, A new look at the stress index for lung injury. Journal of Biological Systems, 2005. 13(3): p. 261-272. [24] Crooke, P.S., et al., Mathematical models for pressure controlled ventilation of oleic acid-injured pigs. Mathematical Medicine and Biology, 2005. 22(1): p. 99-112. [25] Daw, C.S., et al., Observing and modeling nonlinear dynamics in an internal combustion engine. Physical Review E, 1998. 57(3): p. 2811- 2819. [26] Tang, X.Z., et al., Symbol sequence statistics in noisy chaotic signal recognition. Physical Review E, 1995. 51(5): p. 3871-3889.