key: cord-0891580-1xqs6gi7 authors: Theou, Olga; Pérez-Zepeda, Mario Ulises; van der Valk, Alexandra M; Searle, Samuel D; Howlett, Susan E; Rockwood, Kenneth title: A classification tree to assist with routine scoring of the Clinical Frailty Scale date: 2021-02-19 journal: Age Ageing DOI: 10.1093/ageing/afab006 sha: 8b504820ba7d5fd1e1a56fef433b6b82291e0b54 doc_id: 891580 cord_uid: 1xqs6gi7 BACKGROUND: the Clinical Frailty Scale (CFS) was originally developed to summarise a Comprehensive Geriatric Assessment and yield a care plan. Especially since COVID-19, the CFS is being used widely by health care professionals without training in frailty care as a resource allocation tool and for care rationing. CFS scoring by inexperienced raters might not always reflect expert judgement. For these raters, we developed a new classification tree to assist with routine CFS scoring. Here, we test that tree against clinical scoring. OBJECTIVE/METHODS: we examined agreement between the CFS classification tree and CFS scoring by novice raters (clerks/residents), and the CFS classification tree and CFS scoring by experienced raters (geriatricians) in 115 older adults (mean age 78.0 ± 7.3; 47% females) from a single centre. RESULTS: the intraclass correlation coefficient (ICC) for the CFS classification tree was 0.833 (95% CI: 0.768–0.882) when compared with the geriatricians’ CFS scoring. In 93%, the classification tree rating was the same or differed by at most one level with the expert geriatrician ratings. The ICC was 0.805 (0.685–0.883) when CFS scores from the classification tree were compared with the clerk/resident scores; 88.5% of the ratings were the same or ±1 level. CONCLUSIONS: a classification tree for scoring the CFS can help with reliable scoring by relatively inexperienced raters. Though an incomplete remedy, a classification tree is a useful support to decision-making and could be used to aid routine scoring of the CFS. Treatment decisions require information about an individual's goals of care, their severity of illness, level of frailty and other health indicators. With COVID-19, such decisions now can include how to allocate access to limited resources, particularly critical care admissions. For this, the Clinical Frailty Scale (CFS) is recommended [1] [2] [3] [4] . Introduced in the second clinical examination of the Canadian Study of Health and Aging, the CFS summarises the level of fitness or frailty of an older adult after evaluation by a health care professional [5] . Now a 9-point scale from 1 ('very fit') to 9 ('terminally ill'), higher scores represent greater risk [2] . The CFS aims to reflect the baseline health state (2 weeks before); scoring it requires clinical judgement. Although still used to summarise a Comprehensive Geriatric Assessment [6] , more widespread uptake requires that CFS scoring be undertaken by people new to frailty assessment [7] [8] [9] [10] . In the UK, outside geriatrics and frailty services, few clinicians receive formal frailty identification training; many lack confidence in this, and desire more frailty education [11] . When used against a background of care rationing more than for the development of a traditional care plan, the results might not always reflect expert geriatrician judgement [12] . CFS inter-rater reliability is generally very good [13, 14] . Even so, some evidence suggests that personal bias may play a role in judgement-based frailty assessment, especially with inexperienced raters [15] [16] [17] . In consequence, we developed a classification tree to improve CFS reliability when employed by inexperienced raters. Our objective was to compare the scoring of the CFS by the classification tree with the scoring of the CFS done by experienced (i.e. geriatricians) and inexperienced (i.e. trainee) CFS raters. As a secondary objective, we also compared inter-rater reliability of inexperienced versus experienced raters. This is a prospective study of 115 patients aged 65+ years seen in the Emergency Department having been referred to Internal Medicine (N = 43), in clinic (N = 40), on the Geriatric Medicine inpatient consult service (N = 21), or at home (N = 11). The CFS scoring followed a Comprehensive Geriatric Assessment, both completed by clinical clerks, residents or geriatricians. To inform their decisions, they used any available multidisciplinary team assessments, and medication and diagnostic data from the patient's health record. This information was then integrated into a care plan. After reviewing the assessment and plan, and interviewing the patient, one of the two attending geriatricians (SDS; KR) assigned the patient a CFS score. For 52 patients, clerks/residents participated in the Comprehensive Geriatric Assessment and assigned a CFS score independently from the geriatrician; residents did so for 38 patients and clerks for 14. Clerks and residents were either entirely new to the CFS or had limited experience. Local training is a 1-hour session, with on-service, case-by-case geriatrician review. After completing the judgement-based CFS scoring, the CFS was scored separately using the classification tree. The classification tree ( Figure 1 ) asks questions based on the descriptions of each of the CFS levels. We also developed a two-page questionnaire that could be used to collect the data needed to complete the classification tree. It asks about basic and instrumental activities of daily living, chronic conditions, self-rated health, energy level and physical activity (Supplementary Appendix A). As these items are often collected in clinical care and do not require specialist training, the questionnaire is not essential for the classification tree. Here, as a process check and for later database comparisons, the clerks/residents or geriatricians administered the questionnaire to all participants. They used the best information (e.g. validated diagnoses) typically with caregiver input. A research team member reviewed each participant's responses and used the classification tree to derive a CFS score. This project was undertaken as a Quality Assurance (QA) initiative and was approved by the local QA Committee. Intraclass correlation coefficients (ICCs) and their 95% confidence intervals (CIs) were used to compare the classification tree CFS scores with the CFS scoring by experienced and inexperienced raters. ICC reflects both degree of correlation and agreement between measurements [18, 19] . As this was a clinical study, it was not possible for the same set of raters to rate all patients, and as such, ICC estimates were based on a single measurement, absolute agreement and a one-way random effects model [20] . Recognising the arbitrariness of any cut-point in the early stages of research, a minimum reliability of 0.70 is sufficient to conclude good agreement [21] . As a sensitivity analysis, we repeated this analysis excluding CFS level 9 (being terminally ill). CFS level 9 focuses on the current health state, less their baseline health; this impacted only classification tree versus geriatrician score comparisons. Spearman correlation coefficients were used to describe the association between the CFS scores and age. Analyses were conducted using SPSS version 26. To detect an interrater reliability of at least 0.70, assuming a null hypothesis of nominal correlation of 0.20, and given a tolerance of ±0.20, with α = 0.05 and β = 0.80, we estimated a need for 40 ratings [22] . The mean (SD) age was 78.0 (7.3) years (range 65-93); 61 patients (53%) were male. CFS scores of 5 or greater were assigned to 86.1% (95% CI: 78.4-91.8%) of the patients based on the classification tree, 80.9% (72.5-87.6%) based on geriatrician scoring and 71.2% (56.9-82.9%) based on the clerk/resident scoring (Supplementary Appendix B) . The CFS scores based on the classification tree were not significantly correlated with age (rho = 0.125, P = 0.183) and the Classification tree to assist with routine scoring of the Clinical Frailty Scale The classification tree scores coincided with the geriatrician scores in 62.6% of cases; 93% received the same or ±1 score. The CFS rating between the classification tree and the clerks/residents were the same in 57.7% of cases, and within one score in 88.5%. Agreement was stronger when comparing the CFS rating between the geriatricians and the clerks/residents (same score in 76.9% of cases; the same or ±1 score in 96.2%) (Figure 2) . The most common discrepancy in CFS scoring was for 14 patients in whom tree classification was 6 and the geriatrician's score was 5. Otherwise, there was no consistent pattern in re-assignment by the geriatrician or the clerks/residents (Supplementary Appendix C). A CFS classification tree afforded good agreement with an expert rating. Most classification tree scores were either the same or differed by one level compared with the other raters. Still, the classification tree classified people with at least a mild level of frailty slightly more often than the raters. In some cases, differing by at most a single CFS level might be acceptable. In other cases, a small difference could determine receiving or withholding care (e.g. where a predetermined CFS level represents a go/no go rule; [1] ). For these cases, a stricter approach to grading risk is needed. Even so, whether the CFS classification tree could improve routine frailty scoring requires further study. Employing the classification tree is not intended to replace the Comprehensive Geriatric Assessment. Inexperienced raters could use the classification tree to screen for frailty using routinely collected data or the questionnaire (Supplementary Appendix A) . We acknowledge that because the raters of the Comprehensive Geriatric Assessment and the questionnaire used to calculate the CFS classification tree score were the same, bias could have been introduced. Future studies should explore this by having a rater complete the questionnaire independently from the Comprehensive Geriatric Assessment. Discrepancies in CFS ratings were in more than one direction. Expert raters necessarily score the CFS based on information particular to the patient. Finely grained considerations, such as the pattern of illnesses and disabilities, or choice in performing some activity, do not readily translate into a classification tree. Less still does patient forbearance in coping with severe illness. This is what judgement seeks to do: provide an individual context for a given piece of information. Inexperienced raters will need to confirm whether their clinical judgement agrees with the classification tree CFS scoring; especially where they are material, differences will require adjudication. Understanding the degree of a patient's frailty is important to prognostication [23] [24] [25] [26] [27] [28] [29] . A judgement-based, assessment-informed CFS aims to better understand likely challenges, such as the risk of common adverse outcomes [29, 30] . Even so, frailty is just one factor that needs to be considered [2, 7, 9] . Outcomes of given CFS levels need to be addressed in relation to the severity of illness, and of course honouring patient preferences. This is a single-site study, the sample was small and many raters contributed only a few ratings. Even so, we have power to detect a significant relationship between scoring sources, and the ICC is robust to using many raters [18] [19] [20] . One rater was an originator of the scale and in such settings agreement tends to be higher [18] . Whether using the CFS classification tree can help inexperienced raters with their CFS scoring is motivating further inquiry by our group. Supplementary Data: Supplementary data mentioned in the text are available to subscribers in Age and Ageing online. National Institute for Health and Care Excellence (NICE) COVID-19 rapid guideline: critical care in adults Using the Clinical Frailty Scale in allocating scarce health care resources Association of frailty with mortality in older inpatients with Covid-19: a cohort study Comparing associations between frailty and mortality in hospitalised older adults with or without COVID-19 infection: a retrospective observational study using electronic health records A global clinical measure of fitness and frailty in elderly people The role of illness acuity on the association between frailty and mortality in emergency department patients referred to internal medicine Frailty in the face of COVID-19 Rationing care by frailty during the COVID-19 pandemic Rationing care in COVID-19: if we must do it, can we do better? Age alone is not adequate to determine healthcare resource allocation during the COVID-19 pandemic Barriers to the identification of frailty in hospital: a survey of UK clinicians Interrater reliability of the Clinical Frailty Scale by geriatrician and intensivist in patients admitted to the intensive care unit Inter-rater agreement between self-rated and staff-rated Clinical Frailty Scale scores in older emergency department patients: a prospective observational study Inter-rater reliability of clinical frailty scores for older patients in the emergency department The Clinical Frailty Scale: do staff agree? Agreement and predictive value of the Rockwood Clinical Frailty Scale at emergency department triage Reliability of frailty assessment in the critically ill: a multicentre prospective observational study Computing inter-rater reliability for observational data: an overview and tutorial Forming inferences about some intraclass correlation coefficients How Many Subjects? Statistical Power Analysis in Research The effect of frailty on survival in patients with COVID-19 (COPE): a multicentre, European, observational cohort study Frailty and outcomes from pneumonia in critical illness: a population-based cohort study Comparison of two frailty identification tools for critically ill patients: a post-hoc analysis of a multicenter prospective cohort study Frailty and invasive mechanical ventilation: association with outcomes, extubation failure, and tracheostomy Implementation of population-level screening for frailty among patients admitted to adult intensive care in Alberta Assessing frailty in the intensive care unit: a reliability and validity study The Clinical Frailty Scale predicts adverse outcome in older people admitted to a UK major trauma Centre A scoping review of the Clinical Frailty Scale The graphical abstract was created with BioRender.com. In addition to his university and hospital appointments, K.R. is Chief Science Officer of DGI Clinical, which in the last 5 years has contracts with pharma and device manufacturers (Baxter, Baxalta, Shire, Hollister, Nutricia, Roche, Otsuka) on individualised outcome measurement. In 2017, he attended an advisory board meeting with Lundbeck and in 2019 another with Nutricia. Otherwise, any personal fees are for invited guest lectures and academic symposia, received directly from event organisers, chiefly for presentations on frailty. He is an Associate Director of the Canadian Consortium on Neurodegeneration in Aging, which is funded by the Canadian Institutes of Health Research, and with additional funding from the Alzheimer Society of Canada and several other charities.