key: cord- -p jb gf authors: kong, qing; mo, shuming; wang, wenqian; tang, zihui; wei, ying; du, yijie; liu, baojun; kong, lingwen; lv, yubao; dong, jingcheng title: efficacy and safety of jia wei bushen yiqi formulas as an adjunct therapy to systemic glucocorticoids on acute exacerbation of copd: study protocol for a randomized, double-blinded, multi-center, placebo-controlled clinical trial date: - - journal: trials doi: . /s - - - sha: doc_id: cord_uid: p jb gf background: systemic glucocorticoids are effective for the management of chronic obstructive pulmonary disease (copd) exacerbation but have serious adverse effects. traditional chinese medicine (tcm) can bring additional benefits to these patients but has few adverse effects. the present study aims to evaluate the efficacy and safety of jia wei bushen yiqi (jwby) formulas in patients who suffer from copd exacerbations and to investigate whether the short-term ( -days) systemic glucocorticoid therapy is non-inferior to the long-term ( -day) regime. methods: in this multi-center, randomized, double-blinded trial, eligible inpatients with copd exacerbation are randomly assigned to four groups (a, b, c, and d). group a will receive placebo plus -day prednisone, group b will receive placebo plus -day prednisone, group c will receive jwby formulas plus -day prednisone, and group d will receive jwby formulas plus -day prednisone. the primary outcomes are the time interval to the patient’s next exacerbation during a -day following up and the copd assessment test (cat) during treatment. secondary outcomes include lung function, tcm syndrome assessment, laboratory tests, and safety. the changes of the hypothalamic pituitary adrenaline axis (hpa axis) and inflammatory cytokine will be measured as well. discussion: by demonstrating the advantages of utilizing tcm and an appropriate duration of systemic glucocorticoids, this effectiveness comparison trial will provide new references to physicians on how to improve the management of copd exacerbation. the results of hpa axis and inflammation cytokine measurements will shed light on the molecular mechanisms and entail further mechanism studies. trial registration: www.chictr.org.cn chictr . registered on may . chronic obstructive pulmonary disease (copd) will become the third leading cause of death worldwide in [ ] . . % of the population over years old suffer from copd in china [ ] , creating a large socioeconomic burden [ ] [ ] [ ] . copd exacerbation is defined as the acute worsening of respiratory symptoms that require additional therapy [ ] [ ] [ ] . acute exacerbations of copd impair pulmonary function and exponentially increase the risk of death [ ] . therefore, effective management of copd is critical to human health. according to international guidelines and evidencebased reviews, systemic glucocorticoids are recommended to treat copd exacerbation [ ] [ ] [ ] [ ] [ ] [ ] . the advantages include shortened recovery time and hospitalization duration, improved lung function and oxygenation, and reduced relapse risk and treatment failure, which have been demonstrated by numerous randomized clinical trials (rct) [ ] [ ] [ ] [ ] [ ] [ ] [ ] . however, the side effects like hypertension, hyperglycemia, gastrointestinal bleeding, psychiatric disease, and hypothalamic pituitary adrenal axis (hpa axis) suppression increase with the extension of treatment duration and the escalation of dose [ ] . controversy over the optional duration continues. on one hand, a dose of mg prednisone (a common oral systemic glucocorticoid) daily for days has been recommended by the global initiative for chronic obstructive lung disease (gold) science committee report based on the reduce randomized clinical trial since [ ] . the trial indicated the efficacy of -day systemic glucocorticoids is noninferior to -day systemic glucocorticoids regarding relapse within a -month follow-up, but significantly reduced glucocorticoid exposure. on the other hand, a dose of - mg prednisone daily for - days [ , , ] was suggested by another academy of china, korea, and europe in . yet, no clinical trials have determined the difference between the -day and -day regimes. in addition, treatment individualization brings benefits. for instance, an inhaled corticosteroid (ics) is more efficacious in patients with high blood eosinophils [ ] [ ] [ ] . however, present pharmacotherapy has failed to reverse the downtrend in pulmonary function completely [ ] . hopefully, traditional chinese medicines (tcm) can expand copd treatment in terms of syndromic difference, also called zheng [ ] . not only has tcm alleviated symptoms such as coughing, shortness of breath, and sputum for thousands of years, but also has demonstrated its efficacy and safety [ ] [ ] [ ] [ ] [ ] . however, there are rarely studies focused on copd patients during the acute exacerbation period, most of them focused on the relatively stable period. we conducted a randomized and placebo-controlled trial enrolling stable copd patients in , which illustrated that tcm formulas called bushen yiqi (by) formulas can improve the lung function, reduce the frequency of acute exacerbation of copd, and modulate the hpa axis [ ] . dr. shen replaced glucocorticoid therapy with tcm formula (by) totally in chronic inflammatory disease [ ] . moreover, several ingredients in by can decrease the inflammatory reactions in copd animal models [ ] . recently, we have observed that by formulae combined with another two chinese herbs-huang qin (scutellaria) and chi shao (paeoniae rubra radix)-demonstrate more effectiveness on the management of acute exacerbation of copd in clinical practice, such as relieving the symptoms including the cough, sputum, as well as shortness of breath. interestingly, the laboratory experiments showed that the main compound of these two chinese herbs benefits the animal of copd model. for instance, scutellaria baicalensis in huang qin significantly improved lung function, ameliorated the pathological damage, and attenuated inflammatory cytokines infiltration into the lungs [ ] . similarly, paeonol in chi shao showed antiinflammatory and antioxidant effects against cs-induced lung inflammation in both in vivo and in vitro experiments [ ] . therefore, we propose that jia wei bushen yiqi formulae (jwby)-bushen yiqi formulae combined with huang qin and chi shao-will benefit patients with acute exacerbation of copd. this study aims to demonstrate non-inferiority of a -day therapy compared with a -day regimen of systemic glucocorticoids based on the copd outcome during the -day follow-up period. it also seeks to determine the relative inferiority of jwby formula as an adjunct treatment to systemic glucocorticoids compared with systemic glucocorticoids alone for copd exacerbation. this is a multi-center, double-blinded, placebocontrolled, randomized clinical trial. this trial will be conducted in two stages: a -day treatment and then a -day follow-up. qualified patients will be randomized to groups: group a will receive placebo plus -day prednisone, group b will receive placebo plus -day prednisone, group c will receive jwby formulas plus day prednisone, and group d will receive jwby formulas plus -day prednisone. assessments will be performed on day and on day during treatment and telephone calls will be conducted on day and on day when patients are discharged (fig. ) . the day-treatment is chosen for two reasons. first, it is because of the two aims that were mentioned above. second, the -day treatment period is based on our investigation result that most copd exacerbation symptoms can be alleviated within days. in other words, days are the common hospitalization time in ten subcenters. therefore, the day-treatment is a good time for patients to complete the study during hospitalization, which will promote the compliance of patients and collect as much data as possible. the -day follow-up time is based on the results from the reduce randomized clinical trial research published on jama in . it is reported in this trial that the median number of days of follow-up was in both the conventional group ( th percentile, ; th percentile, days) and in the short-term treatment group ( th percentile, ; th percentile, days). this trial will be conducted at ten hospitals located in shanghai, yunnan, xinjiang, and jiangsu province in china. five hospitals are selected because they are attached to universities and another five hospitals are selected because they are experienced in rct. also, these hospitals are spread out throughout china ( table ). the principal investigator (pi) work at huashan hospital and is responsible for the steering committee meeting, which includes protocol training, supervision of safety, quality control, feedback of progress, and study reports. pis of other hospitals will organize their clinical physicians and nurses to carry out recruitment and follow-up. patients that are hospitalized with copd acute exacerbation and meet the inclusion and exclusion criteria ( table ) will be eligible to be study participants. acute exacerbation of copd with clinical grade is defined as follows: respiratory rate > times/min, application of assisted respiratory muscles, no mental state change, hypoxemia can be improved by the %- % oxygen concentration in the inner cover of the venturi, and hypercapnia or partial pressure of carbon dioxide (paco ) increases to - mmhg from the baseline value. patients who are diagnosed as having respiratory failure but without the risk of death are appropriate for ordinary hospitalization, as recommended by the chinese expert consensus on the diagnosis and treatment of acute exacerbation of chronic obstructive pulmonary disease (aecopd) ( update) [ ] . in other words, a moderate degree of copd exacerbation does not indicate the need for intensive care unit (icu) admission according to gold guideline [ ] . tcm syndrome differentiation-fei_shen_qi_xu_yu_ re zheng in chinese-specifies people who have lung and kidney qi deficiency mixed with blood stasis and (table ) . study centers are selected from level a hospital in china. the investigators will be selected from attending physician who majors in respiratory disease. prior to the trial, all sub-center physicians, nurses, and other staff will be trained to understand the protocol. attending physician who will take charge of the patients obtains consent from potential participants or authorized surrogates. firstly, attending physician will introduce the trial including the origin of tcm formula, the prednisone effect, what they should do, and what will benefit them if they volunteer to participate in this trial. then, physician will reply to the questions that confuse patients. finally, both the physician and patient will sign the informed consent form to indicate the patient's full understanding of the protocol. in the consent form, participants will be asked if they agree to use of their data should they choose to withdraw from the trial, and if they are volunteer to provide another ml blood for storage, which are used to explore their inflammation level, hpa axis function, and the relationship between effectiveness and gene type. participants will also be asked for permission for the research team to share relevant data with people from the hospitals who take part in the research. as we mentioned in background, prednisone of - mg once daily is recommend for copd exacerbation management since by golg guideline. the evidence is from a clinical trial that compare the efficacy of days of prednisone treatment with days. the participants come from sweden. as in other countries like china, the duration of prednisone treatment is recommended as - days. the differences of the outcomes between days of treatment and days are unknown in the chinese patients. since we choose the relatively mild patients with copd exacerbation, the minimum dose of prednisone mg once daily is decided in this trial. in addition, tcm formula has been used for copd therapy for thousands of years. we have observed the superiority of tcm as an adjunct therapy in copd administration. but there is no evidence to show the exact outcomes. the doses of five tcm herbs are decided by a group of experienced tcm physician who used the principle of tcm in treating copd for many years. the control group is placebo that contains % true herbs with the same appearance and smelling as the drugs. severe impairment of heart, liver and kidney function (heart function - degree, aspartate aminotransferase (alt) and/or alanine aminotransferase (ast) exceeds . times of the upper limit of normal, creatinine (cr) exceeds the upper limit of normal) . received systemic glucocorticoids within weeks or participation in other drug clinical trials within months prior to the trial . other conditions that the investigators consider to be improper all the participants will be provided with standard of care (soc) according to the gold guideline for copd exacerbation during hospitalization and after discharge (table ) . a -day adjunct medication includes systemic glucocorticoids and tcm herbs or their placebo. a basic dose of mg prednisone daily for days will be provided for all participants. the prednisone will be continued in the long-term glucocorticoids arm of the trial in the following days and replaced with the placebo in short-term glucocorticoids arm of the trial. the -day treatment period is based on the fact that most copd exacerbation can be relieved within days. meanwhile, participants will be treated with tcm herbs or placebo. participants will be randomized to four groups with different adjunct medication ( fig. ) . because of the complex and variety in copd exacerbation, variation among patients will be allowed. any variation like another antibiotic used for the indication will be recorded in the case report form (crf). tcm treatment is in accordance with the most common tcm syndromes of copd in a real-world study [ ] . the dosage of jwby formula is selected according to the pharmacopeia of chinese medicine, and the effective ingredient of its granules is determined according to the pharmacopeia of pharmacopeia. jwby formulas contain kinds of herbs: huang qi (astragalus) g, yin yang huo (epimedy) g, sheng di huang (radix rehmaniae) g, chi shao (red peony) g, and huang qin (scutellaria) g, concentrated as . g granules. to use, patients can infuse . g granules into ml of boiling water and ingest orally after breakfast and supper, twice daily. its placebo is identical in appearance, shape, size, and package with jwby formulas, but only contains % real herbs. the granules will be produced and packed by huarui sanjiu pharmaceutical industry in shenzhen, china. granule production will be certified to get the standard certification of the tcm national drug regulatory authority. modification or discontinuation of the intervention will be decided by the pis in each center, according to the requests from participants, or when a participant's disease is worsened to grade which indicates the need for icu admission, or when unexpected adverse effects happen. prednisone and jwby granules are free as study drugs. five-day drugs will be provided to participants at baseline by a sub-center investigator and another -day drug at day . participants will use patient diaries for recording medication and changes in symptoms. all unused packs of drugs and empty bags will be returned to investigational site on day and on day . compliance will be calculated by counting drugs or empty bags for a day course. compliance % of medication = [actual dose/ (specified daily dose × days)] × %. total medication consistency ranging from to % will be eligible for the protocol analysis set. patients enrolled in the trial will be all hospitalized and all the laboratory tests will be performed on standard schedule, which aids in the monitoring of adherence. once the patient is randomized, the investigators will take every reasonable effort to follow the patient for the entire course of the study. all examination and transportation costs in the -day will be covered and the results of symptoms and physical exams will be explained at every visit. messages will be sent through wechat or by phone prior to every visit to remind the patients of the follow-up visits. extra copd-related drugs, such as leukotriene receptor antagonists, antihistamines, immunosuppressants, and antioxidants, will be forbidden during the trial. tcm herbs that are tonifying kidney, benefiting qi, clearing away heat, and promoting blood circulation, whose tcm characteristics are like those within jwby formulas, will be avoided. drug combinations will be recorded in the case report form at each follow-up visit. "tonifying kidney" ("bushen" in chinese) is a tcm term of treatment, which aims at the tcm syndrome "deficiency of kidney" ("shen_xu" zheng in chinese). the chinese herbs used in "tonifying kidney" treatment can relieve "deficiency of kidney" syndrome including shortness of breath, deterioration with movement, fatigue, waist and knee area sore, and their weakness, tinnitus, dizziness, incontinence, or heavy urine volume. patients that are enrolled into the study will be covered by indemnity through the standard national health service indemnity arrangements. the pi will provide the compensation to those who suffer due to trial participation. primary outcomes measurements ) the time to the next exacerbation of copd during the -day follow-up is defined as one primary outcome. the definition of exacerbation is deterioration of the cardinal symptom of dyspnea, increased sputum purulence and volume, and purulent sputum. this may be combined with one of the other symptoms: increased cough and wheeze, sore throat, nasal congestion due to cold, fever (oral temperature > . °c), increased cough, and increased wheezing. the above changes should last for ≥ days at least. a minimum of week between two exacerbations is needed in order for them to be considered as separate events. the duration of exacerbation is measured from the onset of acute exacerbation to a significant reduction which is defined as the symptoms return to the level before the exacerbation per the records in patients' dairies. the diaries are distributed to participants during the treatment and after the treatment. participants record changes of their symptoms and their health status by choosing the right description in terms of feeling. the primary symptom is measured with modified british medical research council (mmrc) and copd assessment test (cat) scores. the days of exacerbation are calculated from the onset date of the primary symptom to the date when all symptoms disappear. the degree is classified as mild (treated with short acting bronchodilators only, sabds), moderate (treated with sabds plus antibiotics and/or oral corticosteroids), or severe (patient requires hospitalization or visits to the emergency room). severe exacerbations may be associated with acute respiratory failure. ) the mean difference of cat scores between day or day and baseline is another primary outcome. the cat involves an -dimension measurement of health-status impairment in copd. cat is universally acknowledged as a reliable and valid measurement in evaluating the changes of copd. ) tcm syndrome assessment will be evaluated from baseline to day and day . according to the guiding principles for clinical research of new drugs in traditional chinese medicine, the syndrome score is calculated as efficacy index n = (pre-treatment score − post-treatment score)/ pre-treatment score × %. in terms of mild, moderate, and severe symptoms, the primary symptoms are given , , and points while the secondary symptoms are given , , and points respectively. total score = scores of the primary symptoms + scores of the secondary symptoms. ) lung ventilation function will be assessed by forced expiratory volume in s (fev ), forced vital capacity (fvc), and peak expiratory flow (pef) from baseline to day and day with standardized equipment (erich jaeger uk ltd., market harborough, uk jaeger master-screen, germany) and per the standard procedure recommended by american thoracic society (ats) [ ] . ) blood gas analyses including partial pressure of oxygen (pao ), partial pressure of carbon dioxide (paco ), infectious indexes including blood eosinophil count in cells per micrometer (eos), creactive protein (crp), and proclamation will be tested by clinical laboratories in the sub-center from baseline to day and day . side effects will be collected at day , day , and day during follow-up. this specifically refers to ( ) the changes in hyperglycemia: fasting plasma glucose ≥ . mmol/l or random plasma glucose ≥ . mmol/l or rise ≥ % in daily doses of insulin or any increase in oral anti-diabetic drugs or initiation of one or more antidiabetic therapeutics, ( ) changes in hypertension: systolic blood pressure ≥ mmhg and/or diastolic blood pressure ≥ mmhg or the addition of one or more anti-hypertensive drugs to previous treatment regimens, and ( ) the number of psychiatric symptoms, asphalt, vomiting coffee samples, and new infection. laboratory tests which include routine blood test, routine urine test, electrocardiogram (ecg), kidney and liver function, and x-ray computed tomography (ct scan/x-ray) of the chest will be conducted at baseline, day , and day during the follow-up. if the results of ct scan/x-ray and ecg are normal at baseline, it will be skipped in the follow-up. the pathology of copd is relevant to the inflammation and the suppression of the hpa axis that follows the treatment with glucocorticoids. therefore, changes in the hpa axis including corticotropin-releasing hormone (crh), adrenocorticotropic hormone (acth), and cortisol and the inflammation cytokines including interleukin- , interleukin- , and interleukin- at baseline and on day and day will be measured. there are four groups with two variables in this trial-tcm treatment and systemic glucocorticoid treatment. therefore, according to primary endpoints collected from previous trial [ , ] , we choose the maximum sample size needed, as calculated by two way on http:// www.powerandsamplesize.com (table ). at the % significance level, a total of patients per group will be required for a -group, -sided calculation to achieve % power and the differences of . ± . and . ± . in cat mean score between the tcm treatment group and placebo group (table ) . meanwhile, a total of participants will be required for a group non-inferiority calculation to achieve the mean difference of the time to next exacerbation ( . , ) and a non-inferiority margin of , under the condition that the standard deviation of the groups is equal to (table ) . a loss of - % to follow-up is predicted based on experience-this increases the sample size to participants per group, resulting in in total. all investigators in the sub-center will advertise and distribute posters in their emergency department and nearby communities. in addition, we will set up a hierarchical medical system in shanghai-communities refer the potential patients to huashan hospital directly where the clinical trial is undertaken. participants will be randomized with equal probability ( : : : ) to receive one of the four treatments that were mentioned above. as the size of each group is predicted to be , the allocation sequence is generated with sample randomization and stratification by trial center. the sequences will be generated by software and in excel format. before the study begins, a series of random numbers will be generated by the computer, and the pharmacists involved in the study place the random numbers in plain, closed envelopes marked with patient numbers. envelopes will be made and stored at the pharmacy and opened by the pharmacist only when the subjects are randomized. the envelopes will be not accessible to individuals directly involved in the study. allocation sequence will be generated by a statistician who will not participate in enrolling participants. participants will be blindly randomized and allocated with an identified number. principal investigators including attending physician and nurses will involve in enrolling participants. pharmacist will distribute an independent emergency envelope for each participant, which contains the treatment assignment. the participants in the placebo group will be given the same number of pills and followed the same medication schedule as the treatment group. to ensure the implementation of the blinding method, the pill and herbs in both the treatment group and the placebo group will be made in the same shapes, smells and tastes. trial participants, care providers including attending physician and nurse, outcome assessors including pi and sub-pi, and data analysts will be blinded after the assignment of interventions. double-grade unblinding will be adopted. first grade unblinding: it will be conducted before the data analysis. after the double input of all the crf data into the computer and blinded review, the data will be locked. afterwards, the personnel who keep the blinded materials will unblind them for the first time, which is to divide the groups corresponding to the case numbers into blinded codes of two groups and to tell the statisticians so as to statistically analyze all the data. second grade unblinding is after the statistical analysis and the completion of clinical trial report. it will be conducted at the wrap up meeting for the clinical trial. the treatment group and control group will be unblinded. place of unblinding will be the unit where the clinical trial is in charged. executive personnel will be the chief researcher and statisticians of the unit that are in charge of the trial. if there is severe adverse event, which impedes the progress of the trial and the selection of the treatment measures, urgent unblinding can be carried out. during the process, all the researcher, sub-pi, and clinical supervisors should take part. the local administrative unit should be informed within h. the reason, time, and place of unblinding should be recorded in detail and all the records should be signed off. afterwards, the clinical supervisors should be informed timely. the case data should be kept intact. prior to the start of the trial, sub-center physicians will be trained. the results of laboratory tests from different hospitals are adjusted per the huashan hospital standards during analysis. demographic information (date of birth, gender, etc.) and medical condition (medical history, concomitant medication, etc.) will be recorded at baseline. all the questionnaires will be answered by patients without inducement. when adverse events that are related to study drugs happen, emergency envelope can be considered as needed to be opened by pis and physician. the investigators will report the reasons and outcome to the pi within h. prior investigation shows that the mean hospitalization duration time is about - days in these hospitals, which matches the trial requirement of days of treatment. after screening and completing baseline evaluations, participants will visit the physician at day and day during adjunctive treatments and day when patients are discharged (fig. ) . we will provide free tcm granules and partial examination reimbursement to participants. the participants and their family member will be informed that standardized treatment is beneficial to reduce copd exacerbation, which will reduce medical expenses the benefits. two telephone calls will be conducted on day and day . the writing and transfer of case report the case report will be written by the doctor who has participated in the trial. every case should have a complete case report. the case report, once completed, should be checked by the supervisor. afterwards, it will be transferred to the data administrator for data entry and management. all the information in crf table will be recorded in a specialized clinical experimental database that is designed by chinese academy of traditional chinese medicine. the format of the database should be close to that of the crf table so as to facilitate the data entry. the variables in the crf table will be encoded and the codes will be kept unchanged during the whole process of clinical research. the crf data will be entered by highly trained specialists from the research centers. the audit of data can be divided into two forms: manual audit and system audit. the former refers that the administrator checks the consistency and logic of the data so as to find the mistakes and to generate the question list. sas software sets the limit of all variables and rules out automatically the unqualified data by running the system program. the question list is sent to the clinical supervisor who transfers it to the researcher for reconfirmation. the related revision should be signed and dated by the researcher. the researcher will correct the data for the last time after the return of all question lists. all the corrections and updates should be recorded and filed. after the data is verified, the data administration meeting will be held so that the corrections and updates can be summarized. at last, the data administrator will announce the locking of database and keep the cipher code. the statistical analysis prospectus will not be changed after the database lock. the data will be transferred to the statistical department for analysis. all the data should be kept according to the requirements of gcp. after the experiment, all the original copy of case reports and records for the administrations of clinical drugs should be checked, signed, and stamped by the supervisors, head researchers, and representatives from gcp office of each clinical center, and finally, these records will be sent to the leading site where the database will be established and the data will processed. statisticians will analyze the data and materials from the participating centers, and the summary of the clinical trial will be completed in the leading site. case report form (crf) collects all the information throughout the trial for every participant. as soon as verification is completed, data will be securely stored and sent to huashan hospital from the sub-centers. a data management group will be established, and the information will be entered into the database provided by http://www.rilintech.comt through independent doubledata entry. the errors and inconsistencies of data will be checked during the entry process. the user identification code and password will be protected by the data management group. the pis will be given access to the cleaned data sets. sub-investigators will only have access to the data sets in their own hospital. original paper forms will be kept in huashan hospital for years. plans for collection, laboratory evaluation, and storage of biological specimens for genetic or molecular analysis in this trial/future use { } the process and collection of blood samples separation of - ml of plasma from ml of whole blood the anticoagulant and aprotinin (for concentration and amount, please refer to the note) will be added to the blood-collection tube, which is placed at °c for precooling, and then . ml of whole blood will be collected. the samples will be mixed in the tube slowly, and afterwards, the mixture will be centrifuged at a low temperature ( °c, r/min, - min). . ml of plasma will be collected and kept at a low temperature (− °c). if the collected blood cannot be centrifuged immediately, it can only be stored in °c freezer for up to h. note: the concentration and amount of anticoagulant and aprotinin. anticoagulant: . medta. na concentration ( ul/ ml) or % heparin ( ul/ml); aprotinin ( iu/ml). there are two kinds of aprotinin: liquid (the concentration will be noted on the label) and solid ( , iu/mg). the solid form of aprotinin can be dissolved in normal saline, so its concentration can be adjusted to iu/ μl. requirements for sample storage the samples should be kept immediately in − °freezer. throughout the transportation, the samples cannot be taken out. in huashan hospital, all of the samples are checked. the samples should be labeled with case codes and collection date. blood serum should be kept in dry ice for transportation. primary and secondary outcomes the tcm intervention arm-jwby (jia wei bushen yiqi formulas)-will be compared against the placebo. the short-term systemic glucocorticoid (ssg) arm will be compared against the long-term systemic glucocorticoid (lsg) arm. four groups will be compared with each other independently. statistical package for social sciences for windows, version . (spss, chicago, il, usa) will be used for analysis. the tests will be -sided, and a p value with alpha ≤ . level is considered significant. p values will be reported to four decimal places with p values less than . reported as p < . . the bonferroni method will be used to appropriately adjust the overall level of significance for multiple comparisons, assuming an exchangeable correlation structure. categorical variables will be summarized by absolute numbers and percentages of total. the difference of categorical variables will be assessed with the generalized estimating equations (gee). gee will also be used to assess the impact of potential clustering of participants in the same hospital. safety outcomes will be analyzed with summary statistics (frequency, count, percentage). the method of analysis of each variable are summarized in table . the score of copd assessment test (cat) will be collected at baseline, day , day , and in the days, days, days after discharge. the mixed effect normal model (menm) will be used to compare each outcome against the tcm intervention group and placebo. the estimate of treatment effect will be presented as unadjusted rate ratio followed by an adjusted ratio with adjustment for a set of pre-specified baseline variables. the list of pre-specified variables is as follows: centers (as a random effect), age (in years), gender (male or female), weight (in kilogram), smoking (pack per year), fev % predicted, the number of copd exacerbation in the previous year, and home-oxygen therapy. fixed effects will include the visit number, treatment, and all the prespecified variables. participant and visit interaction will be fitted as random effects. an autoregressive correction structure will be used throughout. the difference of interval time to next exacerbation during follow-up in the days, days, and days will be compared between ssg and lsg groups using the generalized linear model (glm) with a log-link function, a propriety over dispersion parameter, and length of time as an offset. the numbers will be described respectively in three gradesoutpatient, inpatient, and icu. durations of copd exacerbation will be compared between each two of the four groups with glm in the similar manner as before. specially, the shortness of breath measured by mmrc dyspnea scale ( - degree) in the diary will be undertaken in the logit link function independently. the changes in tcm syndrome score, infectious index, lung function, blood gas analysis, inflammatory cytokine levels, and hpa axis will be collected in baseline, at day , and at day . glm will be used to analyze the change between each two of the four groups as well. none. in this trial, interventions for participants include days of tcm granules and or days of prednisone. these two interventions will be carried out during hospitalization and they are routine treatments in china, so there are no anticipated problems that will be detrimental to the participant. therefore, there will be no interim analyses and there are not anticipated formal stopping rules for the trial. methods for additional analyses (e.g., subgroup analyses) { b} subgroup analysis the potential subgroups have been listed in table . the analysis of primary outcomes will be repeated in the subgroups. methods in analysis to handle protocol non-adherence and any statistical methods to handle missing data { c} analysis will be in accordance with the intent-to-treat principles. the safety set (ss) includes participants that are randomized and have received adjunct treatments and one post-treatment safety assessment at least. the full analysis set (fas) includes participants that are randomized and have received adjunct treatment, and their primary outcomes are available at least in one visit. the per protocol set (pps) includes participants in accordance with all the following conditions: valid baseline values, compliance with the program, no violation of the inclusion and exclusion criteria specified in the program, completion of all assessments, and good compliance (defined as participants taking at least % of expected doses of study drugs as determined by counting). missing data is predicted to appear on day and during the two telephone calls after discharge. the imputation of all the outcomes will be replaced by the mean of the group. plans to give access to the full protocol, participant level data, and statistical code { c} full protocol participant level dataset in chinese will be accessible in the register site. and statistical code will be provided by trial statistician. for sharing purpose, data will be available to outside investigators at the end of the trial. the finding of this trial will be published in peerreviewed journals and presented at conferences. the results of the study will be released to the participating physician and patients. composition of the coordinating center and trial steering committee { d} multi-center trial coordination committee will be established. the huashan hospital affiliated to fudan university will take charge of the committee, and the main researchers of the participating units will serve as the members. the committee will be responsible for the implementation of the whole experiment and resolve problems during the trial process. the head researcher should strengthen quality surveillance of the clinical trial in his own center. composition of the data monitoring committee, its roles and reporting structure { a} the data monitoring committee is unnecessary in this trial, because the drug duration in this trial is short- days. tcm granules and prednisone are routine treatment in china and will be carried out during hospitalization; only minimal risks are anticipated. adverse event report regardless of whether it is related to the study drug or not, any clinically significant abnormalities of medical events or laboratory tests will be defined as an adverse event (ae). for all adverse events, the time, duration, treatment measures and outcomes, the severity of the disease, and the association with the study drug will be evaluated and recorded. it is divided into mild, moderate, and severe according to the following list: conscious symptoms, ability to tolerate, impact on daily activities, duration, whether it is relieved during continued medication, and whether treatment is required. serious adverse events (sae) will be defined as death or life-threatening events. if a sae occurs, the doctor will immediately take emergency measures and report it to the pi and the ethics committee within h. according to the occurrence of adverse events and a reasonable time interval, and alleviation after withdrawal of the study drugs, the correlation between adverse events and study drugs will be evaluated as affirmative (sure), probably related (very likely), may be relevant (possible), may be unrelated (suspicious), and irrelevant (impossible). due to the unsatisfactory treatment effect, the patient will withdraw from the trial. the emergency letter of the case will be opened, and the patient's family will coordinate with the follow-up and report the result to the lead center. the relevant information will be recorded in the case report form. although the formula is optimized to instant granule instead of tcm herbs decoration in our study, some participants who never accepted tcm herbal previously may have gastrointestinal reactions such as nausea and vomiting. they will be suspended for days and evaluated on their abilities to continue to participate in. because the participants are in the acute exacerbation period, their disease may deteriorate to grade at any time with the worsening of clinical symptoms including increase of dyspnea, mental consciousness changes, blood gas analysis of acidosis, and hypoxemia that cannot be improved by oxygen absorption or other treatments. participants will be admitted to the intensive care unit (icu) if it happens. due to the worsening of the disease or the unsatisfactory effect, the emergency letter of the case will be opened. the physician-incharge will communicate with the patient's family if participant needs to withdraw from the study. the relevant information is reported to pi and recorded in the case report form. as for any deterioration syndromes that arise after discharged, participants will be advised to come to the hospital. the investigator will provide free medical services appropriately. the recommend dose of prednisone by chinese consensus is - mg daily for - days. the low dose of mg is chosen. extra management measures were suggested during the initial meeting. first, the participants will be informed that the withdrawal symptoms include fatigue, joint muscle soreness, low mood, poor appetite, and even nausea and vomiting. second, participants discharged from the hospital with adrenal insufficiency will receive instructions on how to take less than mg daily if they cannot tolerate the treatment. finally, participants will be advised to take the following preventive measures against possible adverse events. closely and modulate the number of hypoglycemic agents or insulin. ) investigators will pay attention to whether the patient has abdominal pain, vomiting of coffee-like substances, or tar-like black stool. if this occurs, the patient should promptly come to the emergency department and be treated with acid-suppressing stomach and other drugs. ) investigators will observe the patient's neuropsychiatric symptoms closely, such as euphoria, excitement, mania, and insomnia. if necessary, advise patients to seek medical help. the designated monitor will visit each investigational site once a month. the monitor will check that if the regulatory binder is complete and all that associated documents is stored well or not, including crf, informed consent forms, and adverse events reports. and the monitor will help the investigational site resolve the issues happened in the trial. plans for communicating important protocol amendments to relevant parties (e.g., trial participants, ethical committees) { } any modification to the protocol which may impact the conduct of the study and the potential benefit of the patients will be reported to ethic committee. the amendments will be approved by the ethics committee before it is announced to each investigational site. and an investigator training about new protocol will be held through wechat video meeting. participants will be informed of the new protocol. participant information will not be released outside of the study without the permission of individuals except for monitoring. blood samples, data collection, and administrative forms will be identified with the same code and stored separately in a locked place. all data will be uploaded to the resman original data sharing platform (ipd sharing platform) http://www.medresman. org of the china clinical trial registry, which is available to outside investigators when the trial ends. the result will be published in peer-reviewed journals and shared at conferences. the findings of the trial will be released to the participating physicians and patients. with the design of tcm as an adjunct to systemic glucocorticoids to treat copd exacerbation in this randomized trial, we will test the non-inferiority of two different treatment terms of systemic glucocorticoids in copd exacerbation. the finding will bring new proofs to the controversial applications of glucocorticoids. in addition, we will clarify a pragmatic method to identify the efficacy of classic description based on tcm syndrome differences despite limitations like bias of measurement and the subjectivity of the questionnaire assessment which may be exacerbated by the loss of some participant during follow-up. the difference between the four groups will indicate that tcm reduces the suppression of the hpa axis and strengthens the anti-inflammation effect of glucocorticoids. tcm may strongly support and enrich the management of copd exacerbation. however, there are some limitations in this protocol. firstly, we choose one of the specific tcm syndromes as the criteria. the result is hard to be extended to the whole patients with copd exacerbation. in addition, we use the chinese guideline to evaluate the degree of copd exacerbation, which relies on the subjective assessment of symptoms of the enrolled participants by the physician. hopefully, an objective method will be proposed to assess the copd exacerbation. our trial has enrolled volunteers in shanghai from august up to today. we have modified protocol according to the practice and standard protocol items [ , ] . in the meantime, we proposed amendment to ethics commitment and chinese clinical trial registry in july of . the protocol version is ky - , , , july, . the new protocol was reported to all subcenter pis in a group meeting. due to covid- , we expect to complete the recruitment process around october and report the results as soon as possible. supplementary information accompanies this paper at https://doi.org/ . /s - - - . additional file . burden of copd prevalence and risk factors of chronic obstructive pulmonary disease in china (the china pulmonary health cph study): a national cross-sectional study prevention and management of copd in china: successes and major challenges chronic obstructive pulmonary disease in china: a nationwide prevalence study mortality, morbidity, and risk factors in china and its provinces, - : a systematic analysis for the global burden of disease study international variation in the prevalence of copd (the bold study): a population-based prevalence study copd exacerbations: defining their cause and prevention symptom variability in patients with severe copd: a pan-european cross-sectional study acute copd exacerbations expert consensus on acute exacerbation of chronic obstructive pulmonary disease in the people's republic of china treatment with systemic steroids in severe chronic obstructive pulmonary disease exacerbations: use of short regimens in routine clinical practice and their impact on hospital stay copd clinical practice guideline of the korean academy of tuberculosis and respiratory disease: a summary diagnosis, prevention and treatment of stable copd and acute exacerbations of copd: the swiss recommendations use of glucocorticoids in patients with copd exacerbations in china: a retrospective observational study controlled trial of oral prednisone in outpatients with acute copd exacerbation oral corticosteroids in patients admitted to hospital with exacerbations of chronic obstructive pulmonary disease: a prospective randomised controlled trial effect of systemic glucocorticoids on exacerbations of chronic obstructive pulmonary disease oral or iv prednisolone in the treatment of copd exacerbations -a randomized, controlled, double-blind study efficacy of corticosteroid therapy in patients with an acute exacerbation of chronic obstructive pulmonary disease receiving ventilatory support systemic corticosteroids in acute exacerbation of copd: a metaanalysis of controlled studies with emphasis on icu patients prednisone in copd exacerbation requiring ventilatory support: an open-label randomised evaluation global strategy for the diagnosis, management, and prevention of chronic obstructive lung disease: the gold science committee report susceptibility to exacerbation in chronic obstructive pulmonary disease acute exacerbations of chronic obstructive pulmonary disease identification of biologic clusters and their biomarkers blood eosinophils to direct corticosteroid treatment of exacerbations of chronic obstructive pulmonary disease: a randomized placebo-controlled trial effects of smoking intervention and the use of an inhaled anticholinergic bronchodilator on the rate of decline of fev ( ) -the lung health study zheng: a systems biology approach to diagnosis and treatments oral chinese herbal medicine for improvement of quality of life in patients with stable chronic obstructive pulmonary disease: a systematic review efficacy and safety of indacaterol and mu g in chronic obstructive pulmonary disease patients from six asian areas including japan: a -week, placebo-controlled study bu-fei yi-shen granule combined with acupoint sticking therapy in patients with stable chronic obstructive pulmonary disease: a randomized, double-blind, double-dummy, active-controlled, -centre study effects of comprehensive therapy based on traditional chinese medicine patterns in stable chronic obstructive pulmonary disease: a four-centre, open-label, randomized, controlled study effects of yupingfeng granules on acute exacerbations of copd: a randomized, placebo-controlled study effects of two chinese herbal formulae for the treatment of moderate to severe stable chronic obstructive pulmonary disease: a multicentre, double-blind, randomized controlled trial important action of improving adrenocortical function for certain diseases recovery effect and mechanism of several traditional chinese medicine components on inflammatory response of chronic obstructive pulmonary disease caused by exposure to cigarette smoke scutellaria baicalensis attenuates airway remodeling via pi k/akt/nf-kappab pathway in cigarette smoke mediated-copd rats model paeonol attenuates cigarette smoke-induced lung inflammation by inhibiting ros-sensitive inflammatory signaling a real-world evidence study for distribution of traditional chinese medicine syndrome and its elements on respiratory disease short-term vs conventional glucocorticoid therapy in acute exacerbations of chronic obstructive pulmonary disease: the reduce randomized clinical trial spirit statement: defining standard protocol items for clinical trials spirit explanation and elaboration: guidance for protocols of clinical trials publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations we thank dr. yiyuan zeng and dr. waijiao cai from school of public health, boston university, for their linguistic assistance during the preparation of this manuscript. qing kong and shuming mo drafted the manuscript. wenqian wang, zihui tang, baojun liu, yijie du and lingwen kong participated in the design of the study. zihui tang participated in the statistic plan. ying wei, yubao lv and jingcheng dong conceived the study, participated in its design and coordination, and drafted the manuscript. all authors read and approved the final manuscript. all named authors adhere to the authorship guidelines of trials; no professional writers have been involved. the enrolled participants who sign the informed consent forms about clinical data and bio-sample collection are provided with free medication of jwby formulas and prednisone for days as needed. participants' information will not be released outside of the study without the permission of individuals except for monitoring. blood samples, data collection, and administrative forms will be identified with the same code and stored separately in a locked place. all data will be uploaded to the resman original data sharing platform (ipd sharing platform) http://www.medresman.org of the china clinical trial registry, which is available to outside investigators when the trial ends. the result will be published in peer-reviewed journals and shared at conferences. the findings of the trial will be released to the participating physicians and patients. central ethics committee approval has been obtained from ethics committee of huashan hospital affiliated to fudan university in shanghai, china (id: ky- ). the local ethics committee of other ten hospitals has approved the protocol, too. the trial was registered on www.chictr.org.cn (id: chictr ) on may , . the investigator will make safety and progress reports to the ethics committee monthly. protocol amendment will be approved by the ethics committee prior to the implementation of amended protocol at the sub-centers. all investigators are trained to carry out the new protocol. there is no conflict of interests among the subcenters. informed consent will be obtained from all study participants. these are available from the corresponding author upon request. there are no competing interests in this work.author details key: cord- - odu lus authors: chen, daohong; qi, eric yining title: innovative highlights of clinical drug trial design date: - - journal: transl res doi: . /j.trsl. . . sha: doc_id: cord_uid: odu lus clinical trials serve as the gold standard to evaluate the efficacy and safety of tested drugs prior to marketing authorization. nevertheless, there have been a few challenging issues well noted in traditional clinical trials such as tedious processing duration and escalating high costs among others. to improve the efficiency of clinical studies, a spectrum of expedited clinical trial modes has been designed, and selectively implemented in contemporary drug developing landscape. herein this article presents an update on the innovated human trial designs that are corroborated through coming up with approval of notable therapeutic compounds for clinical utilization including delivery of several blockbuster products. it is intended to inspire clinical investigators and pharmaceutical development not only timely communicating with the regulatory agencies, but also insightful translating from cutting-edge scientific discoveries. the randomized clinical trials (rcts) of sequential three phases have been traditionally regarded as an official paradigm during drug development for decades. while phase i study is to define the tolerability, pharmacokinetics and adverse effects, subsequently phase ii and then iii studies are to examine the therapeutic efficacy in exploratory and confirmatory manners respectively [ ， ] . although rcts historically played and are still playing a decisive role in evaluating efficacy and safety of a therapeutic agent prior to the marketing authorization, the implementing practice also came up with several challenges including tedious processing duration, ever-escalating costs and lack of subgroup differentiation for maximizing clinical benefits [ , ] . to circumvent these problems, the significant advancement in disease biology and clinical pharmacology has inspired emergence of a spectrum of innovated clinical trial designs in contemporary drug developing landscape [ , , ] . overlapping with the essential principles of classic rcts, these recently emerged clinical studying models are characterized by an impressive list of additional strengths such as improved time and cost effectiveness among others [ ] . based upon the progresses in bio-assays of clinical pharmacology, the usa food and drug administration (fda) made a policy change known as hatch-waxman act in , which officially determined the human pharmacokinetic bio-equivalence (be) to replace traditional rcts for developing generic medications [ ] . as a result, the be policy was also implemented for chemical generics by european union in [ ] . consistently in , chinese state council initiated a campaign requiring the quality of generic drugs be reevaluated through running clinical be studies, compared to that of original reference products [ ] . in the field of innovative medicine, dramatic breakthroughs from life science have revolutionized our understanding in a number of aspects of disease biology, including therapeutic targets and diagnostic bio-markers [ ， ], thus inspiring some flexible modification of traditional rcts in order to deliver novel medicine to the patients in need more efficiently [ ] . accordingly in late , the st century cure act was passed into law by usa congress, and instructed fda to update the adaptive design guidance for investigational drugs and biological therapies [ ] . to date, certain modes of these updated trial protocols have exceptionally contributed to timely translating contemporary scientific discoveries into innovative drugs that addresses unmet clinical needs [ ， ] . in this light, the article herein highlights an array of outstanding developments in the perspective of drug trial design, being corroborated by notable successes in the clinical settings. it has been well recognized that pharmacological effects of drugs are dependent on their targeting tissue concentration which is usually proportional to their distribution in circulation system, known as bio-availability. of note, the latter can be affected by various pharmacokinetic-associated factors including active compound, formulation, manufacture, drug-drug interaction, among others [ , ] . in this sense, major pharmacokinetic parameters, such as area under curve (auc) and maximal concentration (cmax), are utilized as the surrogate parameters to compared efficacy and safety of a generic agent with those of the reference product in clinical be studies [ ] . most drugs are thereby accepted to be therapeutically equivalent when their auc and cmax fall in the range between of %- % regarding limits of % confidence interval [ ] . while having been significantly contributing to development of most chemical generics, be is recently noted as an efficient approach that can not be completely replaced by in vitro methodology even in some high solubility and high permeability products [ ] . in parallel, be study is frequently applied for comparing certain innovative agents in terms of formulation changes or fixed dose combinations; for example just being approved in , consensi consists of a combination of amlodipine and celecoxib to treat concomitant hypertension and osteoarthritis [ ] . realistically for evaluating a generic version of the polysaccharide drug heparin, the activities of anti-factors xa and iia instead of pharmacokinetic parameters are used to be the official standard for human be trials [ ] . in consistent with the policy of fda, european medicines agency (ema) has recently accepted the be investigation in healthy human subjects for generic development of low molecular weight heparin (lmwh) without requiring rcts in thromboembolism prone patients anymore [ ] . besides, utility of be study is being expanded into the field of evaluating biosimilar products such as antibodies and fusion proteins which represent much larger and more complex molecules [ ] . interestingly based upon a human be investigation in healthy volunteers and non-inferiority rcts in the patients, the biosimilar version of etanercept, a medication of fusion protein neutralizing inflammatory cytokine tumor necrosis factor- (tnf), has just be approved for managing rheumatoid arthritis (ra), psoriatic arthritis, and ankylosing spondylitis [ ] . contemporary pharmaceutic innovation is facing a serious realty of declined successful rates from research and development, in contrast to much more new compounds/new targets for examination in clinical investigation. in this regard to reduce a wast of the resources, fda issued a guideline of an alternative approach for first-in-human trials in , which was termed exploratory investigational new drug application (eind, or phase ) allowing a flexible amount of data needed for ind application based upon the specific circumstances of each proposed human trial [ ] . tending to bridge the gap between preclinical studies and traditional clinical development, phase investigation differs from phase i trial in fewer human subjects, short processing time and no tolerability test. as a result, phase study can accelerate the "goor-no go" decision making prior to a formal rct, namely improving the efficiency of identifying drug-like candidates and terminating non-promising compounds [ , ] . the strengths of phase study have been validated in various aspects of drug development including intra-target micro-dosing, clinical pharmacology, vulnerable populations, among others [ ] . interestingly, micro-dosing of insulin through limb artery injection was capable of achieving targeted control of local glucose level with minimal systemic exposure of the drug [ ] ; it offered insights into local application of medications, in particular for those with narrow therapeutic windows, to deliver clinical efficacy precisely while remarkably diminishing the systemic toxic effects [ ] . impressively, an unique role of phase trial has been recently highlighted in the drug development against brain tumor. as a cell cycle-associated kinase inhibitor, azd was revealed to have a strong anti-glioblastoma efficacy and poor blood-brain barrier (bbb) penetration in preclinical studies. it was through a phase investigation demonstrating that the compound substantially crossed bbb and induced the expected biomarker alterations clinically, which thus made a moving-forward decision for this project [ ] . besides, microdosing study appears particularly beneficial for vulnerable populations in which new drug trials are restricted, including children, liver/kidney function impaired, mutipharmacy, among others [ ] . in contrast to a classical clinical trial of sequential three-phases with interval times in between for analysis to determine the next design, seamless development integrates the three phase-processing into a comprehensive clinical study without gaps [ , ] . while reserving the core strength of traditional clinical trials in terms of defining efficacy and safety regarding tested compounds, seamless clinical study significantly improves time and cost efficiency to translate the scientific breakthroughs into innovative medicine [ ] . a seamless phase i/ii design is to examine the toxicity and efficacy in one trial, during which a reasonable dosing set need to be screened out at the first stage, and then forwarded to evaluate the efficacy at the second stage [ , ] . in a similar sense, a seamless phase ii/iii trial is performed with multiple dosing levels of the investigated drug at the exploratory stage to define the most efficacious dose based upon some surrogate end-points such as objective response rate (orr), and this dose is then continued into the confirmatory stage for further testing without a timing gap, to obtain more definitive end-points such as overall survival (os) [ , ] . given the flexibility for modification upon interim analysis, seamless development allows to study accumulating data from the ongoing clinical trial for early assessment of toxicity, efficacy or futility, and thus to decide the terminating or continuing/expanding treatment arms accordingly for further investigation [ ] . it was through a well-designed seamless clinical trial that the programmed death (pd- )-blocking antibody keytruda achieved accelerated approval by fda [ , ] . in this scenario, a dosing-escalation study was carried out at the first stage of the trial, to evaluate safety, tolerability and possible efficacy (orr) in a spectrum of patient cohorts with advanced cancer of various types [ , ] . based upon the interim analysis, the cohorts of melanoma and non-small cell lung cancer (nsclc) were determined to be expanded for the continuing clinical trial on treatment with keytruda at the selected dosages of mg/kg or mg/kg [ , ] . as a result, keytruda was demonstrated to confer a therapeutic benefit for the patients with melanoma (orr: %~ %) and nsclc (orr: . %); moreover, with the bio-marker stratification of pd-l + above %, an even better orr ( . %) was achieved in the subset of patients with nsclc [ ] . additionally, beyond successful application for oncological drug development, seamless clinical study has substantially contributed to therapeutic innovation in other medical fields [ ] . of note, the efficiency of clinical developing inhaled indacaterol, a long-acting  -agonist for treatment of chronic obstructive pulmonary disease (copd), was clearly improved through a seamless clinical trial [ , ] . historically human disease is categorized and managed according to anatomic location of body organs and histological types of pathology， which forms a solid basis for therapeutic indications designed in conventional clinical drug development [ ] . however, since the rise of targeted medicine in recent two decades, the landscape of pharmaceutical innovation has been transformed to modulate an aberrant biological pathway that can contribute to pathogenesis across a number of diseases [ ] , such as a gene mutation occurred in various neoplastic disorders [ ] . intriguingly, a mutational target is typically present only in a portion of the patients with each tumor type due to inter-personal heterogeneity, thus inspiring emergence of basket trials in the clinic [ , ] . in this light, a targeted compound is simultaneously examined across numerous disease baskets, for example different cancer types, in order to determine not only whether the drug is efficacious but also what tumor types and more precisely which patient subsets are sensitive to this therapy [ , ] . the first stage of a basket trial is to select the therapeutically sensitive disease types and patient sub-populations, with a bio-marker-based companion diagnosis if possible [ , ] . following interim analysis, the futile tumor type baskets are terminated, and the efficacious subjects are enriched from the responsive baskets, to be thus forwarded to further clinical investigation at the second stage [ , ] . as a selective tropomyosin receptor kinases (trk) inhibitor, larotrectinib represents the newly approved medicine targeting a wide spectrum of tumor types, reflecting a dramatic success of basket clinical study [ , ] . aberrant trk activation drives oncogenic pathogenesis, and is expressed in more than distinct tumor types. nevertheless, except a few types of rare neoplasms such as congenital fibrosarcomas, trk alterations occur in very low frequencies in common cancers of various tissue/cell lineage origins, for instance below % in lung adenocarcinoma [ ] . based upon a basket trial design to investigate the clinical efficacy of larotrectinib, the subjects were selectively enrolled from patients with the aberrant trk activity in tumors across histological types. impressively, larotrectinib was demonstrated to be well tolerated, conferring an orr of % and a median progression-free survival (pfs) of . months [ , ] . as such, this novel targeted agent stands out to address the unmet clinical need of combating the rare neoplasms and numerous common tumor types with the rare genetic mutations [ , ] . besides the validation in field of anti-cancer drug development, basket clinical has been proposed to go beyond oncology into cardiology, such as therapeutic innovation for preserved ejection fraction (hfpef) across diverse etiologies, pathobiologies, and clinical presentations [ ] . to identify the best therapeutic benefit(s) of a testing drug, the flexibility of adaptive clinical trials can be implicated in indication changes based upon accumulating data or/and cutting-edge scientific discoveries [ ] . depending on bio-medical contexts, such dynamic updating of clinical applications may be pre-planned or inspired by certain serendipitous observations, through shifting from a therapeutic hypothesis to an alternative one or multiple ones [ , ] . in this sense, a study design allows to examine beyond the intend-to-treat disease, thus being re-purposed to an alternative indication according to the initial assignment upon lack of efficacy or safety issues [ ] ; otherwise, a clinical trial can also be optimized to cover more therapeutic directions with the emerged evidence of certain additional efficacy [ , ] . in the era of precision medicine, there has been a consensus that therapeutic outcomes can vary significantly among subgroups of patients with differential genetic profiles [ , ] . accordingly taking the advantage of interim analysis based on novel biomarker approach for detecting the pathogenesis-specific molecular alteration(s), an adaptive clinical study can select the drug-sensitive sub-population from patients with initially targeted disease or an alternative indication, to continue the investigation for an optimized therapeutic efficacy [ ] . as a selective -phosphodiesterase inhibitor, sildenafil was designed to manage angina pectoris in the initial clinical trial. disappointingly this compound appeared non-efficacious in relieving anginal pain, with a side-effect of inducing penile erection [ ] . inspired by this serendipity, the clinical development was successfully re-directed to come up with a major innovative product of anti-erectile dysfunction [ , ] . in the field of oncology, crizotinib was initially identified as a potent inhibitor of mesenchymal-epithelial transition factor (met), thus tending to treat the relevant neoplasms such as nsclc and gastrointestinal tumors [ ] . whereas this compound was then revealed to suppress anaplastic lymphoma kinase (alk) as well, and impressively to exert a clear efficacy of tumor shrinkage in the nsclc with alk rearrangement during a phase i clinical study. in this regard, the therapeutic indication of crizotinib was re-focused and successfully developed through following-up clinical trials, to be a precisely targeted medication for managing a subset of lung cancer patients with alk aberrations [ , ] . interestingly in recent years, besides serving as an exceptional type of medicine for controlling tape diabetes, the clinical trials of sodium glucose cotransporter (sglt ) inhibitors are being expanded or re-positioned to manage heart failure and protect renal function through blood glucose loweringdependent or/and -independent mechanisms [ , ] . orphan drugs are developed to treat a particular spectrum of medical conditions with low morbidity, termed rare disease, resulting from genetically linked pathogenesis in most cases. of note, definition of rare disease varies among different countries depending on epidemic incidence of each illness geographically [ , ] . to address this largely unmet clinical need and meanwhile to deal with the challenging issue of limited market size for profits, fda announced an incentive policy known as the orphan drug act (oda) in , offering research & development funding, market exclusivity, among other attractive benefits [ ] . since then there were similar policies coming forth from other countries, encouraging an increased interest in this regard for pharmaceuticals; orphan drugs have been representing more than % of innovative medications marketed through last three decades [ ] . interestingly in recent years, certain orphan drugs are revealed to be overlapped with blockbuster products, which appear more notable in the field of oncology [ , ] . in this scenario, a novel medicine can be initially approved for a rare disease, and subsequently go beyond to treat additional types of disease; vice versa a conventional drug may occasionally obtain orphan status upon expanding its therapeutic indications toward a rare medical condition [ , ] . clinical development of orphan drugs often involves expedited trial designs described above, and may even be further simplified or alternatively designed under particular circumstances [ , ] . for instance, the approval of orphan drug xuriden was based upon a minimal seamless trial for weeks with only four patients, to treat hereditary orotic aciduria representing a rare disease of only cases worldwide [ ] . in addition, a viral inhibitor tecovirimat has recently been authorized to treat smallpox upon the waive of clinical efficacy study due to lack of patients with the naturally occurred disease. setting the first record in regulatory history, this innovative medication was exceptionally approved based on positive results from the preclinical efficacy studies in animal models of rabbits and monkeys, along with a phase i trial of clinical pharmacology and safety in healthy human subjects [ ] . impressively, certain orphan drug-based clinical trials can also come up with a broad spectrum of efficacy covering multiple indications. as a hallmark success of targeted medicine with orphan status, to combat philadelphia chromosomepositive chronic myelogenous leukemia (cml) imatinib was launched onto the market through the accelerated approval following a phase ii trial consisting of three open-label, single-arm clinical studies [ , ] . moreover, imatinib was subsequently evaluated in a basket clinical trial and demonstrated to be efficacious for an array of extra-therapeutic indications beyond cml, dramatically transforming this orphan drug toward being a blockbuster medication [ ] . besides, an interesting basket clinical study has recently been designed to investigate a target-specific monoclonal antibody for controlling an array of complement-mediated rare disorders, including bullous pemphigoid, antibody-mediated rejection of organ transplants and and warm autoimmune hemolytic anemia [ ] . clinical trial plays an indispensable role in evaluating efficacy and safety of therapeutic agents prior to marketing for human use, and has been constantly co-evolving along with the dynamic interactions between cutting-edge scientific discoveries and regulatory policy updating [ , ] . in this light, a number of agile clinical developing modes were designed through past decades, and impressively some of them have been further corroborated upon the contemporary achievements evidenced by successfully delivering innovative medicine to the patients in a more efficient manner [ ] . while human bioequivalence study is increasingly contributing to evaluation of emerging formulation and bio-similar agents besides chemical generics [ ] , several adaptive trial designs have been capable of translating the scientific breakthroughs into novel therapeutic benefits with shorter processing time and lower financial costs, to address the unmet clinical needs [ , ] . moreover with the assistance of bio-marker-based companion diagnosis, certain innovative trial designs have been streamlined to precisely confer selective therapeutic efficacy to the responding subgroups of patients with an array of serious diseases such as cancer and beyond [ , ] . of note, to preserve the strength of clear defining efficacy and safety of tested drugs, the innovative designs of clinical study are substantially overlapped with classic trial protocols of three phases which still serve as the mainstream approach of clinical investigation [ , ] . realistically, the medications through expedited processing appeared having a higher rate of post-approval black-box warnings than that of regularly approved drugs in terms of safety issues [ ] . it has accordingly been proposed that single-armed phased ii studies to be accepted only for accelerated approval (aa) applications regarding refractory diseases. otherwise, interim analysis of ongoing phase iii trials may be able to support aa processes, with the follow-up studies required to provide further evidence for the drug's effects on human bodies [ ] . whereas precision medicine-inspired trials represent an unique highlight in contemporary clinical studies to optimize the therapeutic efficacy for preferable subsets of patients with certain diseases upon enrichment strategies [ , ] , the validated biomarkers are limited and even much less than established drug-target molecules [ , ] . moreover, there still is a lack of the bio-markers to predict drugtriggered adverse events such as heparin-induced thrombocytopenia or immune checkpoint inhibitor-resulted hyper-progressive disease [ , ] . hence, it takes a dialectic perspective to appreciate the high efficiency of these innovated designs, with mindful efforts on circumventing their imperfectness. replaces rcts biosimilar enoxaparin [ ] biosimilar etanercept [ ] phase trial exploratory trial azd [ ] with microdosing midazolam ddi [ ] seamless trial integrated trial keytruda [ , ] without phase gaps indacaterol [ ] basket trial one trial for numerous larotrectinib [ ] types of disease tafamadis [ ] therapeutic shifting therapeutic sildenafil [ ] re-purpose indication during a trial crizotinib [ , ] empagliflozin [ ] orphan drug various expedited xuriden [ ] trial designs tecovirimat [ ] imatinib [ , ] assessing the gold standard--lessons from the history of rcts cancer clinical trials: the rear-view mirror and the crystal ball adaptive designs for clinical trials chemisimilars and biosimilars: is clinical testing fit for purpose? make up a missed lesson-new policy to ensure the interchangeability of generic drugs in china targeted therapy : resistance and re-sensitization adaptive design clinical trials: a review of the literature and clinical trials.gov adaptive clinical trials: advantages and disadvantages of various adaptive design elements the new european medicines agency guideline on the investigation of bioequivalence investigating the discriminatory power of bcs-biowaiver in vitro methodology to detect bioavailability differences between immediate release products containing a class i drug consensi) for hypertension and osteoarthritis bioequivalence of a biosimilar enoxaparin sodium to clexane ® after single mg subcutaneous dose: results of a randomized, double-blind, crossover study in healthy volunteers the introduction of biosimilars of low molecular weight heparins in europe: a critical review and reappraisal endorsed by the italian society for haemostasis and thrombosis (siset) and the italian society for angiology and vascular medicine (siapav) focus on biosimilar etanercept -bioequivalence and interchangeability microdosing and other phase clinical trials: facilitating translation in drug development phase trail of azd in first-recurrence glioblastoma patients intra-target microdosing-a novel drug development approach: proof of concept, safety, and feasibility study in humans cellular immunity augmentation in mainstream oncologic therapy adaptive clinical trial designs in oncology using model-based "learn and confirm" to reveal the pharmacokinetics-pharmacodynamics relationship of pembrolizumab in the keynote- trial pembrolizumab keynote- : an adaptive study leading to accelerated approval for two indications and a companion diagnostic intergrating indacaterol dose selection in a clinical study in copd using an adaptive seamless design an efficient basket trial design dual targeting autoimmunity and cancer: from biology to medicine efficacy of larotrectinib in trk fusion-positive cancers in adults and children basket trial of trk inhibitors demonstrates efficacy in trk fusion-positive cancers a next-generation trk kinase inhibitor overcomes acquired resistance to prior trk kinase inhibition in patients with trk fusion-positive solid tumors innovative clinical trial designs for precision medicine in heart failure with preserved ejection fraction the role of serendipity in drug discovery translating recent results from the cardiovascular outcomes trials into clinical practice: recommendations from the central and eastern european diabetes expert group (ceedeg) the serendipitous story of sildenafil: an unexpected oral therapy for erectile dysfunction development of crizotinib, a rationally designed tyrosine kinase inhibitor for non-small cell lung cancer sodium glucose cotransporter- inhibition in heart failure: potential mechanisms, clinical applications, and summary of clinical trials orphan drugs and their impact on pharmaceutical development challenges in orphan drug development and regulatory policy in china clinical research for rare disease: opportunities, challenges, and solutions fda approves ultra-orphan drug on a -patient trial oral tecovirimat for the treatment of smallpox gleevec for the treatment of chronic myelogenous leukemia: us. food and drug administration regulatory mechanisms, accelerated approval, and orphan drug status phase ii, open-label study evaluating the activity of imatinib in treating life-threatening malignancies known to be associated with imatinib-sensitive tyrosine kinases combined integrated protocol/basket trial design for a first-in-human trial safety related label changes for new drugs after approval in the us through expedited regulatory pathways: retrospective cohort study accelerated approval of cancer drugs: improved access to therapeutic breakthroughs or early release of unsafe and ineffective drugs? antxr , a stem cellenriched functional biomarker, connects collagen signaling to cancer stem-like cells and metastasis in breast cancer heparin-induced thrombocytopenia hyperprogressive disease in patients with advanced non-small cell lung cancer treated with pd- /pd-l inhibitors or with single-agent chemotherapy we thank our colleagues for encouragement. the authors declare no conflict of interest. key: cord- - e muqnu authors: dal-re, rafael; mahillo-fernández, ignacio title: waste in covid- clinical trials conducted in western europe date: - - journal: eur j intern med doi: . /j.ejim. . . sha: doc_id: cord_uid: e muqnu nan both the scientific and ethical perspectives [ ] . the quality of a trial relates to the design, conduct, analysis and reporting. the design is the only aspect that can be appraised in ongoing trials from the information reported in trial registries. although the design can only provide a partial view of the trial's quality, it is relevant to assess the main design characteristics since illdesign trials will provide very limited useful information. a cross-sectional analysis was carried out based on a series of searches conducted on may - , looking for non-industry-sponsored, ongoing trials, aiming to assess medicines or convalescent plasma for the treatment of covid- patients, in the largest european countries and regions (benelux, scandinavia). the searches were conducted on five registries (clinicaltrials.gov, dkrs, eu-ctr, isrctn and ntr) with the terms 'coronavirus', 'covid' and 'covid- '. so, three searches were conducted per registry and for each country. 'country': belgium, denmark, finland, france, germany, italy, luxembourg, netherlands, norway, spain, sweden, and the uk. on the searches on clinicaltrials.gov, the following descriptors were also used: study type: 'interventional studies (clinical trials)'; status: recruitments: 'not yet recruiting', 'recruiting' and 'active: not recruiting'; funder type: 'nih', 'other us federal agency' and 'all others (individuals, universities, organizations)'. trials on special populations (e.g. cancer, diabetes patients, pregnancy, healthy volunteers, healthcare workers), or with special objectives (e.g. pharmacokinetics) or assessing prophylactic medicines (both pre-and post-exposure) or both prophylaxis and treatment, or assessing vaccines, or dietary supplements (eg, vitamins), or herbal preparations (eg aesculus hippocastanum) were excluded. all phase or phase / trials were excluded. convalescent plasma and ozone trials were included. the following information was extracted from each trial: single-arm or randomized controlled trial (rct), blinding, and planned sample size; whether the trial was multicenter, had a control (standard of care) arm and a data monitoring committee. all these features are associated, in principle, with robust designs. randomization and blinding prevent selection, performance and detection bias. large sample sizes help to eliminate known and unknown confounders. singlecenter trials tend to provide larger intervention effects than multicenter rcts. being covid- a life-threatening disease, dmc should be present in multicenter rcts [ ] . since there are no approved drugs, all trials should have a standard of care (soc) group. fisher's exact test was used for comparisons. a p< . was considered significant (two-sided test). there were different trials, ( %) rcts and single-arm trials. the findings (table ) showed that italy was conducting % of all single-arm trials run in europe and accounted for % of all trials conducted in italy. france and spain carried out between and times the number of rcts run in the other countries/regions. the majority of rcts conducted in germany ( %) were masked, but only some % of those conducted in benelux, france, italy, spain and the uk. eighty-six percent of rcts run in scandinavia were multicenter, but only % of those carried out in spain. no standard of care arm was only present in rcts conducted in france, italy and spain (p= . ). data monitoring committee was differently present across multicenter rcts in all countries/regions (p= . ). there were few large trials ( table ) . of special interest were solidarity, the world health organization-sponsored rct, that was being conducted in five of the countries included in this study, and recovery, the largest trial conducted in only one country (the uk) aiming to recruit , participants. forty-two percent of all trials ─single-arm trials, rcts recruiting ≤ participants/arm and those with no standard of care arm─ were poorly designed. this study shows that low-quality clinical trials were being conducted in all analyzed european countries/regions, which are among the territories of the world with the best clinical research. mediterranean countries (france, italy and spain) had higher number of single-arm and significantly more rcts without standard of care arm than that of the rest of countries/regions, which resulted in a seemingly trend to higher percentage of low-quality trials than in central/northern europe. single-arm trials accounted for a similar percentage in this analysis ( %) and in a study that included trials registered in any continent ( %) [ ] . limited useful information should be expected from single-arm trials, rcts recruiting ≤ participants/arm and those with no standard of care arm. some of these, however, could provide hypothesis-generating findings ─rather than hypothesis-supporting results─, that should be confirm in future adequate rcts. since it should be expected that the absolute reduction on mortality should be limited [ ] , many trials recruiting even hundreds of participants will end up with neutral/negative results. yet, some trials could show positive results, but this will likely happen in non-critical (but important) endpoints ─as occurred in an rct with > participants/arm that found a shorter time to recovery with remdesivir compared with placebo, although non-significant difference was observed on mortality [ ] . when reaching larger number of participants (> /arm), the benefit on mortality could be appropriately assessed as has been shown in the recovery trial: dexamethasone has proven to be beneficial [ ] whereas hydroxychloroquine did not [ ] . the main limitation of this study is that all the information has been captured from trial registries that are not always updated. so, for example, after recruiting participants the actt trial was early terminated [ ] , soon before the searches on trial registries of this analysis were conducted, but it appeared on the registries as 'recruiting' (clinicaltrials.gov) and 'ongoing' (eu-ctr), with a planned sample size of participants. also, the fact that eu-ctr is the only registry reporting on the presence of the rct's data monitoring committee limits its usefulness as a factor of robust design, since it was not available from trials that were only registered in other registries. however, it should be highlighted that it is rather surprising why some trials carried out in all the countries/regions included in this study seemed not to be registered on the eudract database (from which the eu-ctr obtains all the information), something that should be expected from all trials conducted with medicines in europe. since convalescent plasma is not a medicinal product and ozone could also be considered as such, trials assessing only these interventions could reasonably not be on eu-ctr; this, however, cannot be applied to other trials conducted with medicines that were being assessed on the treatment of covid- patients (see supplementary material). so as not to waste the generosity from trial participants and the effort from investigators of all these trials, de-identified individual participant clinical trial data should be forwarded to two teams that are conducting living systematic reviews with network meta-analysis [ , ], that could report their results in a short period of time. this will be of invaluable benefit to clinicians and future patients. waste in covid- research against pandemic research exceptionalism characteristics of registered clinical trials assessing treatments for covid- : a cross-sectional analysis randomized clinical trials and covid- . managing expectations remdesivir for the treatment of covid- -preliminary report different dosages n: planned sample size trial conducted in other countries in addition to the ones considered in this analysis; (b) this trial is also run in finland and norway but with different number of arms (n= ), and thus with different id key: cord- -bn g gi authors: wake, melissa; hu, yanhong jessika; warren, hayley; danchin, margie; fahey, michael; orsini, francesca; pacilli, maurizio; perrett, kirsten p.; saffery, richard; davidson, andrew title: integrating trials into a whole-population cohort of children and parents: statement of intent (trials) for the generation victoria (genv) cohort date: - - journal: bmc med res methodol doi: . /s - - -x sha: doc_id: cord_uid: bn g gi background: very large cohorts that span an entire population raise new prospects for the conduct of multiple trials that speed up advances in prevention or treatment while reducing participant, financial and regulatory burden. however, a review of literature reveals no blueprint to guide this systematically in practice. this statement of intent proposes how diverse trials may be integrated within or alongside generation victoria (genv), a whole-of-state australian birth cohort in planning, and delineates potential processes and opportunities. methods: parents of all newborns (estimated , ) in the state of victoria, australia, will be approached for two full years from . the cohort design comprises four elements: ( ) consent soon after birth to follow the child and parent/s until study end or withdrawal; retrospective and prospective ( ) linkage to clinical and administrative datasets and ( ) banking of universal and clinical biosamples; and ( ) genv-collected biosamples and data. genv-collected data will focus on overarching outcome and phenotypic measures using low-burden, universal-capable electronic interfaces, with funding-dependent face-to-face assessments tailored to universal settings during the early childhood, school and/or adult years. results: for population or registry-type trials within genv, genv will provide all outcomes data and consent via traditional, waiver, or trials within cohorts models. trials alongside genv consent their own participants born within the genv window; genv may help identify potential participants via opt-in or opt-out expression of interest. data sharing enriches trials with outcomes, prior data, and/or access to linked data contingent on custodian’s agreements, and supports modeling of causal effects to the population and between-trials comparisons of costs, benefits and utility. data access will operate under the findability, accessibility, interoperability, and reusability (fair) and care and five safes principles. we consider governance, ethical and shared trial oversight, and expectations that trials will adhere to the best practice of the day. conclusions: children and younger adults can access fewer trials than older adults. integrating trials into mega-cohorts should improve health and well-being by generating faster, larger-scale evidence on a longer and/or broader horizon than previously possible. genv will explore the limits and details of this approach over the coming years. conclusions: children and younger adults can access fewer trials than older adults. integrating trials into megacohorts should improve health and well-being by generating faster, larger-scale evidence on a longer and/or broader horizon than previously possible. genv will explore the limits and details of this approach over the coming years. keywords: research methodology, randomization, registry trials, multiple baseline randomized trials, trials within cohorts, population studies, generation victoria (genv), clinical trial as topic, children, intervention background randomized controlled trials (rct) provide high-quality evidence with regards to the effectiveness of therapies and prevention and are critical to guide translation and optimal resource allocation. the traditional parallelarms trial design is a stand-alone initiative for which each trial identifies a specific question and sample, obtains funding, consents and randomizes subjects to two or more different treatments or interventions, follows the groups in parallel and collects the outcome data. there are many variationsfor example, cluster vs. individual randomization, and stepped-wedge, adaptive and cross-over designed. stand-alone randomized trials are challenging, slow and costly [ ] . more than two-thirds of multi-center, publicly funded uk trials do not recruit their target number of patients within the specified timeframe [ ] . consequently, trials are often underpowered, require an extension with additional costs, and encounter delayed translation into clinical or preventive practice, or are never completed. furthermore, most trials interrogate a small number of hypotheses in restricted groups over a short time frame (when a long-term benefit is often the real goal), often with considerable heterogeneity between trials in samples, methods and outcomes. collectively, this results in financial and scientific inefficiencies and a lack of generalizability and translatability [ ] . this situation is particularly problematic for children [ ] , whose evidence base (and therefore care) lags due to a paucity of trials [ , ] . one efficient and generalizable solution is to embed trials in existing data collection structures such as registries, electronic health records, and administrative databases [ , ] . there are many examples of this approach. high-quality registries focus on full, unbiased condition ascertainment with standardized outcomes embedded into clinical care. these can support registry trials, whereby a registry participant meets a trial's eligibility criteria, is consented, randomized and accrues trial outcomes that are usually fully embedded in the registry. point-of-care trials embed trial processes (like randomization, ascertainment of outcomes) into the clinical care process, increasingly by effectively using the electronic medical record (emr). large, simple trials may compare interventions already in standard care, but for which evidence of superiority or equivalence is not available [ ] . as individual risk is low, these may include opt-out (with inclusion the default) or waiver of consent, point-of-care randomization (see above), and/or short information statements. registry, point-of-care and large, simple trial elements may co-occur in a single trial. a recent development to leverage even greater health gain from single registries is to coordinate simultaneous registry trials using pre-specified master protocols. these may test the impact of targeted therapy on multiple diseases (basket trials), of multiple therapies on a single disease (umbrella trials), or of several interventions against a common control (platform trials, also known as multiarm multi-stage (mams) trials) [ ] . park's 'landscape' analysis reported rapid growth in master protocols over the last years. however, very few of these trials target children, and most are in highly specialized fields. thus, of the master protocols identified ( basket, umbrella, platform; in the us), most were in adults ( / , %), exploratory, and designed to examine experimental drugs ( / , %) in the field of oncology ( / , %) [ ] . challenges include stakeholder coordination, infrastructure and governance requirements, and the integration and complexities of the pre-specified trial and analysis design [ ] . multiple trials can also be conducted within longitudinal epidemiological cohorts. the traditional role of a cohort study is to observe incidence, prevalence, trajectories, natural history and exposure-outcome associations. however, their sampling design and longer outcome horizons may also be appealing to trials, for which the cohorts can act essentially as populationbased registries. one advantage is that the trial sample can be compared to a broader population in terms of baseline characteristics and natural history of a condition of interest and its short and long term outcomes. trials may be conducted in parallel with the cohort and across a wide range of patient and participant groups, as in western australia's origins project [ ] ; with appropriate consent, a cohort may subsequently provide pre-randomization (e.g., genetic) and some or all outcomes data for trials. a second model has been variously labelled zelen trials [ ] , cohort multiple randomized controlled trials (cmrcts) [ , ] , and trials within cohorts (twics), as in england's born in bradford better start cohort [ ] . in twics, cohort participants consent to contribute control data to future unspecified trials at recruitment into the cohort itself, with only participants randomly allocated to the intervention arm then asked for informed consent into any given twics trial. despite the potential for allocation bias, they can be efficient and achieve valid [ ] and meaningful outcomes [ , ] , and can evaluate the impact of 'stacked' interventions [ ] approximating how parents and children naturalistically navigate needed services. a third potential model is that of master protocols as above that predefine from the outset and coordinate a set of trials that may occur. we are aware of two potential examples for healthy cohorts: the developing global alpha collaboration aims to embed large platform trials in routine care to improve maternal and perinatal health, while helti (healthy life trajectories initiative) comprises harmonized interventional periconception cohorts in china, south africa, india and canada testing preplanned, stacked interventions over a -year period aiming to prevent obesity and other non-communicable diseases in over , children. if a cohort were to involve a sufficiently large and complete population, then it would be possible to enact a range of these ideas simultaneously, encompassing a wide range of unmet trial needs from rare diseases through to population and health services research. the forthcoming generation victoria (genv) provides an opportunity to explore and operationalize these ideas. genv is a state-wide cohort that will approach for recruitment, parents of all newborns (estimated , ) in the state of victoria (population . million [ ] ), australia, over two full years from . its goal is to generate translatable evidence (prediction, prevention, treatments, services and policy) to improve the future wellbeing of all children and adults and to reduce future disease burden. however, there is no existing road map in the international literature to achieve and encourage the flexible trials-genv integration that could contribute to this goal. here, we report on the preliminary processes and guidance we have developed so that trials can prospectively integrate within a substantial cohort (genv) to maximize these opportunities. as a statement of intent, this paper differs from a master protocol in that it does not prespecify any one trial or trial design. this statement of intent outlines the proposed principles and processes for the integration of future trials into genv. its development has been guided by the spirit statement [ ] and the anticipated consort extension for rcts using cohorts and routinely collected health data. the genv cohort is currently in advanced design (see below). an aud . million grant from the paul ramsay foundation supports its infrastructure development, while a $ million grant from the victorian government supports its design, cohort planning and implementation, stakeholder engagement and knowledge translation activities. its sponsor-investigator is professor melissa wake, who is also genv's scientific director. the core executive comprises melissa wake, richard saffery (deputy director, biosciences), sharon goldfeld (deputy director, equity and knowledge translation) and kathryn north (director, murdoch children's research institute (mcri)). the directors regularly report to genv's operational advisory committee and thence to the board of the mcri. several advisory committees inform genv, including a broad range of senior victorian researchers comprising the investigator committee, and several working groups of which the ethics & governance and trials working groups are relevant here. this relatively simple administrative structure may mature after successful cohort implementation. genv has been endorsed by the royal children's hospital human research ethics committee (hrec . ), including in-principle approval as a mechanism to support trials. genv proposes to work with trials that fulfil the administrative information requirements laid out in the spirit checklist (see additional file ) or equivalent at time of application. these cover title, trial registration, protocol with version control, funding sources and types, and roles and responsibilities of protocol contributors, sponsor/s and funder/s, and other individuals or groups overseeing the trial such as the coordinating center. these documents will contain the unique features of each trial's design, participants, timelines and outcomes, as well as data sharing and other relationships with genv. at the time of writing, the genv cohort is in advanced planning and, therefore, still evolving. a range of genv summary documents are available for review on figshare (https://mcri.figshare.com/projects/generation_ victoria/ ) [ ] . here, we outline genv with only enough contextual detail to inform the context of the trials. genv's primary objective is to create very large, parallel whole-of-state birth and parent cohorts for discovery and interventional research. genv data and biosamples can only be used for research that benefits human health. genv's setting is the entire state of victoria (population . million in ), australia [ ] . because it may be relevant to trials that could be undertaken in a cohort of this magnitude, we provide some victorian descriptive information here. in the census, the median age of victorian people was years; . % of its population were - -year-old children, . % were aged years and over, and . % were male [ ] . around % of victorian residents were born in australia. the most common ancestries were english ( . %), australian ( . %), irish ( . %), scottish ( . %) and chinese ( . %), but its multi-ethnicity is reflected in more than languages. around % of the population identify as aboriginal and/ or torres strait islander [ ] . like other australians, victorians are relatively affluent, with a median weekly pre-tax personal income for people aged years and over of aud in [ ] . however, a wide range of advantage-disadvantage exists, with % of victoria's population, and % of its children, living below the poverty line based on the census data [ ] . over % of australian parents report their child has at least one ongoing health or developmental problem at every age from age , rising to around % from age to years [ ] . genv is a population-based cohort study that blends study-collected, study-enhanced and linked data. the cohort design comprises four elements: ( ) consent soon after birth to follow the child and parent/s indefinitely until the study closes (no end date set at this point) or withdrawal, ( ) retrospective and prospective linkage to clinical and administrative datasets, ( ) banking of retrospective and prospective universal and clinical biosamples, and ( ) genv-collected biosamples and data. genv proposes to recruit for two full years from in all of victoria's birthing hospitals (n = at time of writing) [ ] , in which collectively around , babies are born each year. mcri-employed recruiters aim to personally approach parents of all newborns for consent, with interpreting and translation support as needed. children will have the opportunity to decide on their continued participation as they reach the age for legal consent. genv's consent includes parent permission for approved researchers to access genv's data, for data sharing between genv and external trials, and for recontact to offer additional research opportunities. all children born in victoria during the recruitment period whose parents/guardians have decisional capacity, and their parents, are eligible to take part. participants who leave victoria may continue to take part via linked and contributed data, and families may join genv, who move into victoria later and have children born within the recruitment period. however, in both of these instances, data may be incomplete. principles for genv and for trials within and alongside genv figure is an infographic that shows the concept of how trials might integrate with genv across the life course. all genv activities (including those that relate to trials) are informed by the genv principles, as outlined in fig. (a): collaboration, inclusivity, sustainability, enhancement, systemized processes and value. therefore, it is implicit that all trials working prospectively within and alongside genv would also be a good fit with these principles. we do not anticipate that this would impose an additional burden since funding bodies and international guidelines already stipulate best practice integral to trials, such as highquality evidence of need (e.g. a prospero-registered systematic or rapid review) and checklists (e.g. con-sort, spirit). all trials need their own ethical and other approvals before they can start. for maximal mutual benefit, we envisage that partnerships between genv and trials will generally be prospective, i.e., worked out and agreed before the trial begins. we are currently developing processes to operationalize the trial-specific principles outlined in fig. (b) in minimally burdensome ways. other trials will continue as they have always done, independently and unrelated to genv. genv may support trials within (model ) and trials alongside (model ) genv, as outlined in fig. ; these models do not dictate the design of the trial (e.g., whether individually or cluster randomized). trials within genv (model ) may be conducted as standard trials with opt-in, opt-out, or waiver of consent preceding randomization. alternatively, twics/cohort multiple rcts may sometimes be considered whereby participants are randomized and then only those randomized to the intervention provide additional opt-in or opt-out consent. interventions for trials alongside genv (model ) may be delivered either by genv or by trials themselves, with the latter well suited to trials arising from emrs and external registries for participants born in the genv window. for model , the trial collects the participant's consent for two-way data sharing with genv, including a minimum dataset of items common to multiple trials. governance will be agreed upon before the trial implementation. for trials alongside genv, all or some of the trial sample is also in genv. in the latter situation, the genv principles and the data sharing and enhancing benefits of genv would only apply to that subsample. we anticipate that most trials that integrate with genv will be proposed by researchers outside genv's implementation team, who would, therefore, also define the content, ideally to have a good 'fit' with genv. some trialists are already actively approaching us with ideas for trials that would either be impossible without genv or would benefit from otherwise inaccessible outcomes data. genv hopes to elicit other possibilities via activities (to be developed) such as publicized annual open faceto-face and web-based fora to brainstorm and prioritize trial ideas, ideally involving a range of stakeholders including services, communities and families. good communication, transparency and agreement are vital and will underpin a working together agreement between genv and each trial (example shown in additional file ) developed following genv's rapid evidence review of large research-led partnerships [ ] . trialists and those responsible for genv may at times have differing opinions on where the balance of benefit vs burden lies, and this will need to be considered openly. the benefit can be demonstrated by a rapid or systematic review supporting the need for the trial, ideally within the context of a 'living' review that can be readily updated over time, including with the results of the trial [ ] [ ] [ ] . burden relates not only to participants (consent, intervention content, follow-up and thence potential attrition) but also to impacts on genv itself and its guiding principles. genv is not funded to conduct trials, which will require their own funding. genv can currently provide limited support (including supporting trials to apply for funding) and is seeking dedicated funding to be able to provide additional support. in the meantime, activities such as determining eligible participants, consent, randomization and (potentially) intervention delivery as per fig. , model may need to be undertaken on a cost-recovery basis. for support in the form of expertise, genv -rather than reinventing the wheel -proposes to connect proposed trials to local expert bodies such as the melbourne children's trials centre, monash university trials hub, nhmrc clinical trials centre, australian clinical trials alliance (acta), and the interdisciplinary maternal perinatal australasian collaborative trials (impact) network of maternal and perinatal trials. we anticipate an initial trial-genv discussion that articulates the rationale of the intervention, trial design and research questions using the picot framework. if feasibility (potentially demonstrated through pilot studies) and mutual alignment appear likely [ ] , the trial would proceed to a partnering agreement that defines at least the following items: ) which genv trial model is being followed; ) design and high-level (or draft) protocol; ) timelines; ) data sharing and governance plans; ) status of ethical approval; ) communication with participants, including information statement and consent; ) trial oversight and ) capacity assessment, including trial quality, human resource and funding. we envisage that this discussion will be enabled by a yet-tobe-established genv trials oversight committee with cross-disciplinary expertise. inclusion of consumers, including genv participants, will be important to minimize participant burden and streamline data collection and trial conduct. the trial sponsor will usually be a representative of a university, research institute or similar organisation, but genv does not preclude collaboration with commercial sponsors provided all its principles are met. it is assumed that, throughout the trial, genv will collect agreed data and maintain high retention while the trial will maintain independent quality, ethical and governance protocols in line with international standards. it is also assumed that genv staff will collaborate with the trial management staff in order to understand, prevent and solve any day-to-day issues at the genv end that may impact on the trial or genv. integration of trials with genv to greatest effect will occur with appropriate consent wording in both the trial and genv. at recruitment into genv, parents provide consent for genv to follow themselves and their child. as per the consort extension [ ] for rcts using cohorts and routinely collected health data, this includes consent to use their data for research purposes, with ongoing mechanisms to enable change in consent status (such as partial or full withdrawal or re-entry) any time after that. genv's full parent information & consent statement (picf) is available for review [ ] ; its explicit trial-relevant wording is shown in table (a). all trials require ethically-approved consent models and wording. as noted in fig. , it is plausible that trials could be undertaken via waiver, opt-out or opt-in consent models, and that opt-in/opt-out consent could be undertaken in full (all arms, ideally before table picf wording for (a) genv to work with and (b) trials to work with genv (a) wording in the genv picf that is specific to supporting trials: • "you may be offered the chance to take part in future ethically approved studies working with genv …. you can always choose whether to take part." • "genv's data can only be used for ethically approved research to improve health, development, or wellbeing for children and adults. over time, researchers will use lots of different methods to answer new and important questions. therefore, the value of your information will keep growing for many years." • "some genv participants may join research trials testing new approaches. all trials need ethical approval. who is offered the new approach is randomly picked, like tossing a coin. in some trials, only people offered the new approach are contacted about taking part. genv data can be used to compare the outcomes of people who do and do not receive the new approach." • "trials … may ask your consent to share data with genv, with ethics approval. we support this." (b) suggested wording for trials to include in their picf, as appropriate to its degree of integration with genv ( fig. • [model c onlychoose relevant wording] "you can be in this trial and not in genv, but we will be missing some information about you/your child or you can only take part in this trial if you are also in genv." randomization) or twics models. for trials wholly within genv (fig. , model ) , genv already includes consent for data sharing with approved users. for trials alongside genv (fig. , model ), trials will likely benefit most if data can flow from trial to genv and not merely from genv to trial (for which consent is already in place) as outlined in section (data) below. therefore, genv recommends that trials include wording along the lines of table b to support maximal data utility and value. design of randomization is determined by each trial as is most appropriate to the intervention, questions and sampling, and as per best sample selection, randomization and blinding of follow-up practice at the time. random allocation can take place before (in the case of twics) or after informed/waived consent, and by genv or by the trial, as indicated in fig. . randomization procedures will be reported in each trial's consort statement. genv may inform trials in a variety of ways. it may provide primary or secondary outcomes, measures on which to select or stratify participants, and moderator and mediator variables. figure illustrates the range and timing of measures being explored by genv at the time of writing, spanning linked, biosample-derived and genvcollected data. it is expected that ultimately many, but not all, will prove feasible for genv to include via data linkage, data collection or biosamples. whereas birth cohorts have traditionally been purely observational in design, and focused on the discovery of longitudinal associations, genv's focus is on solutions to improve health and reduce the burden of disease. testing such solutions may occur not only via trials but also natural experiments, simulation and causal modeling. all require robust outcome measures with sufficient sensitivity to demonstrate meaningful effects and an ability to quantify potential health gain when putative causal factors are targeted. the commonality of outcomes would enable comparisons of benefits and costs of different interventions for different target groups within a single dataset. a further benefit to trials is that genv intends to collect such measures over many years, enabling trials to access longer-term data than might be possible for a single trial. genv is planned without knowledge of what future trials may be proposed. however, genv has developed a framework (j wang, yj hu, s clifford, s goldfeld, m wake: selecting lifecourse frameworks to guide and communicate large new cohort studies: generation victoria (genv) case study, submitted) and outcomes hierarchy (fig. ) to guide its measures selection and prioritization (additional file ). this framework considers genv's whole-of-state remit and principles of inclusivity, sustainability and systematized processes, which require low measurement burden with high ease of administration. therefore, genv will not collect prioritized measures that are already reliably collected and accessible via linkage sources, unless required for the day-to-day running of genv. at the highest level, genv will repeatedly capture overarching health and wellbeing with generic measures that have international as well as local salience: health-related quality of life (quality-adjusted life years, qalys [ ] ), disease/disability burden (disability-adjusted life years, dalys [ , ] ), requiring information on conditions, illnesses and problems that parents and children experience (international classification of diseases th revision, icd- [ ] ), and functional status (international classification of functioning, disability and health, icf [ ] ). when coupled with service-related data, for example, regarding encounters, costs and medications, these measures would also support economic analyses. although comprehensive, these highest-level measures do not capture the individual traits/phenotypes that are critical to many interventions. therefore, to support the greatest number of trials while retaining parsimony, genv proposes to prioritize collecting outcomes included across multiple core outcome sets (cos) [ ] and thus already demonstrated to be of broad importance to patients, families, clinicians and policymakers as well as researchers. these span physical phenotypes (e.g., growth, body composition, dysmorphology, motor skills and senses), and mental, social, cognitive, learning and positive health. given that genv is targeting over , children and their parents with data from hundreds of participants daily, the only feasible way of collecting such phenotypes is remotely and digitally. to enable capture of multiple outcomes and phenotypes, therefore, genv is exploring developing an 'ephenome', a high-throughput digital platform will let genv measure and evaluate diverse outcomes cheaply and at very large scale, while maintaining genv's principles of value (including to participants) and inclusivity. we envisage that each ephenome measurement encounter would select from a suite of ultra-short, universalcapable digital survey items, measures, images and videos. these will either be pushed universally from genv for participants to complete remotely on any device or (for measures that meet the genv enhancement principle) could be collected by services within existing universal contacts. future funds permitting, we envisage that face-to-face school-based assessment will capture measures that require physical equipment, technical skill in administration and/or wearable devices. such assessments may be shaped by the needs of individual trials ongoing in the years before each wave. as well as its direct digital platform, genv proposes to draw data wherever possible from administrative datasets. this information includes, for example, data from health, education and other providers; electronic medical records (emr); geographic datasets; and from trials and registries. all available data will be integrated with the genv data systems, using direct deposition, enduring data linkage and/or ephemeral safehaven linkage processes. genv's 'victorian child's lifecourse journey in data' [ ] lists many of the datasets that victorian children and parents currently accrue, many of which genv may link to in the future. genv's website [ ] will publicly record all datasets accessed each calendar year. biological samples at any stage are frequently out of reach of trials due to burden or cost, especially samples that predate trial commencement. genv is working towards the consented storage and research use of existing and new universal biosamples to the highest possible standards of conservation (including transfer to genv's − °c autostore). figure outlines the range of samples currently being explored. it is hoped that the biosamples will span multiple tissues (e.g., blood, saliva, stool, breast milk), all participants, and multiple time points including all trimesters of pregnancy, the neonatal period and school entry. there may be potential for collaborating trials to help shape future whole-of-genv biosample collection. due to the depletable nature and likely small available volume of biosamples, it is highly unlikely that genv will approve individual assays for trials, but rather will make available comprehensive biological data of the broadest value possible such as metabolomics or polygenic risk data. genv will encourage trials to collect a generic minimum dataset relevant specifically to trials, following precedents set by initiatives such as the dutch older persons and informal caregivers survey minimum dataset (topics-mds) to which over projects have now prospectively contributed [ ] . this small minimum dataset is to be developed collaboratively in - . potential benefits include the ability to compare trials on common outcome metrics, evaluate effects of stacked interventions for those in more than one trial simultaneously or over time, and pooling of data for individual participant meta-analyses. many trials will also require outcomes specific to their research questions. once trials and genv are agreed, the trial investigators will most likely collect samples themselves outside of genv in dedicated visits. the data genv hold may prove very valuable to trials because they would otherwise be unavailable due to population coverage, timeframe, jurisdiction, logistic, funding or other constraints. however, constraints could likewise apply to genv. genv commits to making data available on completion of a given 'sweep' or 'wave,' i.e., once all participants have provided a particular set of data. like other major cohorts, it will generally handle data processing for its vast numbers of participants en masse, with benefits including efficiency and cost reduction, uniform access to technology advances (e.g., new automated scoring or assays), consistency (e.g., avoidance of batch effects/drift and of conflicting or overturned results) and completeness (unfinished data waves that do not meet the principle of inclusivity, whereby all data are available for all participants). each wave of genv data collection will likely take years from first to last participant to collect measures that are predicated on age milestones; thus, trials data would be available much sooner for trial interventions conducted later than earlier in a data collection wave. while some data items (e.g., straightforward proms (participant-reported outcome measures)) need no or minimal processing, others (e.g., image extraction) take additional time even when processing begins before the wave is complete. therefore, it will usually be important that trials work closely with genv during the design and ethics approval phases to consider the timing of likely outcomes data and its implications. for example, desired genv data may not be available sufficiently close to real-time to contribute to data safety & monitoring committees or to adaptive trial designs whereby the intervention is modulated according to therapeutic effect. note, however, that for trials within genv, process monitoring data (e.g., consent rates, interaction durations, data response rates) will be available promptly to optimize the trial's compliance to its protocol. for trials conducted wholly within genv, all data will be within its data repository. for trials conducted alongside genv, data will need to be shared between the trial and genv. figure illustrates the benefits of two-way data sharing. by genv transferring data to trials, trials can access additional outcomes over more extended time frames, and potentially examine variation in response by moderators (such as pre-existing prospectively-collected biological or psychosocial traits) or mediators. by trials transferring their data into the genv repository, they can access data that genv cannot on-transfer (e.g., linked administrative datasets according to custodian agreements), model causal effects to the whole population using actual whole-population data, and combine and compare costs and benefits/utility across trials. all of these should enhance trial prominence, impact, and translation of significant findings. data standardization, quality control, safety and privacy applied throughout the genv data repository will also apply to trial data that enter the genv dataset. genv's data, legal, linkage and cohort personnel will provide technical support and guidance for these issues. genv is designed to be accessed by a wide variety of analysts, including researchers, service providers and policy-makers while maintaining confidentiality. its data are intended to be an equal access resource, via the fair [ ] and five safes (safe people, projects, settings, data, outputs) [ ] principles, to facilitate uptake and translation. from the point at which a complete useable dataset is available to them, we propose that trialists would have exclusive access to trial data placed within genv for months (in line with non-trial studies such as the uk biobank and the longitudinal study of australian children), with intervention/control status masked for a full months. each trial will undertake its own sample size calculations, statistical analysis, and reporting according to its design and best international standards at that time. for example, it is assumed that each trial will involve a biostatistician experienced in trials, and that the trial will be analyzed and reported according to standards such as consort (including the forthcoming consort extension for trials using cohorts and routinely collected health data), spirit and template for intervention description and replication (tidier) [ , , ] . trials may also access advice and support from genv's biostatisticians, subject to genv funding. genv's trials capabilities will work within the solutions hub, the arm of genv that is concerned with epidemiology, science, knowledge translation and researcher engagement. the authors of this statement of intent comprise the current expert genv trials working group, whose working together agreement is shown in additional file . during - this group will support the governance and planning work needed to move from this statement of intent to a position where genv is fully enabled to support trials as envisaged. although impacted by the covid- pandemic, consumer and stakeholder engagement has commenced and will continue. in late , genv conducted an open web-based focus area survey whose analysis is nearing completion. genv has engaged and will continue to engage widely with research and service bodies spanning health (universal, primary, secondary and tertiary), education and other sectors, who are represented on many of its working groups. consumer consultation will proceed through engagement led by genv's solutions hub, including how people of aboriginal and torres strait island descent may choose to be involved. we propose a formal consultation process on an annual basis, with formats yet to be determined, and focusing mainly on the major issues and opportunities for the cohort's age and stage - years hence (see fig. ). this review allows enough time to plan and fund trials, put partnerships in place, and complete preparatory work such as rapid or systematic reviews supporting the need for the trial. genv does not propose to formally limit the number of trials that a participant could enter, but rather take into consideration the needs of individual trials and apply a 'reasonableness' approach. if more trials are proposed or funded, than the genv sample can accommodate, then a collaborative prioritization process will be needed to determine which can be supported. to our knowledge, genv is the first mega-cohort internationally aiming to maximize its experimental as well as observational evidence via an integrated and purposive program of trials. this statement of intent lays out broad principles and processes ahead of genv's planned commencement in , streamlining the integration of trials within or alongside genv from its earliest days. commencement of genv immediately after covid- will enable a unique and powerful platform for ongoing surveillance and response; its population reach, digital infrastructure and ability to remotely support decentralized trials place it uniquely to evaluate experimental strategies to manage the pandemic's health, economic and social aftermath on children, families and communities during potentially lengthy partial quarantine periods and recovery. the major strength is the design of genv itself. as a whole-population study aiming to recruit all babies born and their parents over full years in the sizable and stable state of victoria, it reaches into every metropolitan, regional, rural and remote community and every level of advantage. its data linkage and ephenome capacity lower the burden for both trialists and participants. the existing genv data systems can support the activities outlined in this statement of intent without architectural changes. this trial-ready scaffold may empower communities that have typically lacked the necessary infrastructure to lead or join trials, especially relating to health services research and behavioral interventions. multiple trials could be embedded, evaluating multiple interventions and identifying multiple participant groups all with a true population denominator. genv's outcomes hierarchy and time horizons should, for the first time, enable comparison on the same metrics of the lasting benefits and costs of multiple, widely-differing approaches to improve health and wellbeing. this statement of intent should expedite trial planning, documentation and implementation. regarding limitations, this statement of intent does not take into account the unknown success of genv's recruitment. this is potentially an issue if groups that could most benefit from a boost in trials-based evidence are under-represented (e.g. disadvantage, ability, minority) while noting that trials alongside genv offer a route to redress this via later recruitment into genv of those who initially declined or were missed. we also do not yet know whether or how much the inclusion of potential future trials in our parent information statement at the time of consent will impact on genv's uptake rates. while smaller studies (such as origins and born in bradford better start, see below) have been generous in sharing their learnings and thus shaping our plans, we have not identified existing very large-scale studies that could highlight possible unintended consequences. we hope in due course that genv can provide such empirical learning. despite the collaborative thinking underlying this statement, detailed capabilities remain to be designed and constructed (such as processes to identify and to randomize eligible participants), some of which may only be solved once trials are in planning or underway. some of these are discussed in practical or operational issues, below. a further limitation is that genv is not at this time funded to support trials or their administration. such support (over and above the funds required by the trial itself) may be vital to help collaborators navigate the requirements for starting and conducting trials, especially in regional or rural hospitals and communities that do not have a robust research infrastructure. obtaining such internal genv 'support' funding will be an ongoing focus. for those who may wonder if genv might stifle other research, we affirm that genv has no capacity or desire to impose collaboration with clinical or other trials involving children born in the genv birth window. we do hope for mutual awareness and communication. others are recognizing the promise of longitudinal intervention cohorts to 'stop describing and start fixing' children's problems [ ] . the bibbs (born in bradford better start) experimental birth cohort [ ] aims to recruit pregnant mothers by in inner-city bradford, north england, and to test over interventions for children's social and emotional development using a range of designs including trials within cohorts (twics) and quasi-experimental designs. origins [ ] aims to recruit , mothers in the joondalup region (a community in northern perth, western australia) between and . within this, active participants are invited to participate in sub-projects if they meet the eligibility criteria, with participation in some projects restricted if the outcomes overlap; at time of writing, twelve nested randomized trials are currently under way (personal communication, j davis). experience from both indicates a healthy appetite from researchers, policymakers and trial funders, but also that regular transparent two-way communication is vital, as are burden minimisation of and realism about timelines for trialsrelated data management and release unless the cohort itself is adequately funded to handle this. the helti (healthy life trajectories initiative) consortium has attracted large-scale funding from national funding bodies in its member countries canada, india, south africa and china in collaboration with the world health organization, and is well along the path of establishment [ ] . over time, genv participants may encounter more than one trial, and therefore "stacked" interventions across childhood that respond to the issues they are experiencing. this approach is less planned than helti but may mimic how children and adults accrue services naturalistically. at least one observational study has demonstrated a cumulative beneficial effect of participation in more services across childhood [ ] and another that stacking multiple intervention components is costeffective [ ] . while examples are accruing of trials integrated with cohort studies under different names, such as cohort embedded rct [ ] , cohort multiple rct [ ] , cohort nested rct [ ] and trials within cohorts (twics) [ ] , genv's freedom of design -spanning trials both within and alongside genv -appears unusual. here, we mention some of the many details that remain to be decided. many will require resourcing both in genv and the trials themselves. genv will need vigilance in limiting red tape while at the same time being in a position to help prioritize, plan, standardize (e.g. measures, processes), execute and monitor trials in ways that help trials while upholding genv principles. we have yet to develop genv's internal administrative structures to achieve this (ahead of recruitment commencing in ). we are currently developing a framework to enable rapid, followed by increasingly deep conversations and filtering with potential collaborating trials to enable mutual understanding of likely success and benefit that does not waste time. we are also developing our mechanisms to prioritize consumer engagement and involvement, to integrate genv-generated evidence into living evidence reviews, and to progress a brief minimum trials dataset. despite its recognized value, data linkage remains challenging in almost all jurisdictions in australia and worldwide, and goalposts will no doubt continue to shift. trial ethical approval and participant consent for data sharing, sometimes years in the future, will be critical. genv is conducting a privacy impact assessment and data security audit, even while knowing that practice and legislation in both continue to evolve. victoria's health and educational systems vary by individual practice, hospital and school, and across private/public and regional/district lines, so genv's statewide remit may bring challenges in terms of developing standardized interventions. most trials require piloting; we are unsure as to whether pilots would be conducted within or outside the genv environment and the impact of any pilots themselves. we are uncertain as to the extent to which a trial's research team may share genv's infrastructure (it systems, data management practices) to perform trials alongside genv, which could have practical benefits to both but would require resourcing. there may be external constraints on exactly how genv can provide data back to trials, and the extent to which trials need to use the genv data analysis and visualisation environments because of constraints of data custodians. at this time, there seems to be no evidence for an upper limit of trials for a single cohort or a single participant, but their possibly interacting effects may be challenging to tease out. we do not at this time propose any limit other than participant willingness to consent, a 'reasonableness' lens and commitment to the genv principles. while the inclusion of low-burden universal outcome measures makes participation in multiple trials possible both from the human and costs perspectives, genv will need a way to monitor and prevent participant fatigue and not to overburden/compromise either genv or the participating trials. there may be occasions where participation in one trial precludes participation in another, which will need to be considered on a caseby-case basis. research ethics committee decision-making in relation to an efficient neonatal trial what influences recruitment to randomised controlled trials? a review of trials funded by two uk funding agencies rethinking pragmatic randomised controlled trials: introducing the "cohort multiple randomised controlled trial" design the lancet editorial. paediatric research should take centre stage a descriptive analysis of child-relevant systematic reviews in the cochrane database of systematic reviews children are not just small adults: the urgent need for high-quality trial evidence in children born in bradford's better start: an experimental birth cohort study to evaluate the impact of early life interventions the imperative of overcoming barriers to the conduct of large neonatal brain injuries in england: population-based incidence derived from routinely recorded clinical data held in the national neonatal research database master protocols to study multiple therapies, multiple diseases, or both systematic review of basket trials, umbrella trials, and platform trials: a landscape analysis of master protocols pre-emptive medicine: public health aspects of developmental origins of health and disease a new design for randomized clinical trials development and internal validation of a clinical risk score to predict pain response after palliative radiation therapy in patients with bone metastases benefits and challenges of using the cohort multiple randomised controlled trial design for testing an intervention for depression cohort randomised controlled trial of a multifaceted podiatry intervention for the prevention of falls in older people (the reform trial) spirit statement: defining standard protocol items for clinical trials generation victoria figshare project victoria records highest population rise of all states and territories accessed census quickstats accessed parent-reported prevalence and persistence of common child health conditions generation victoria (genv) cohort s synopsis genv rapid evidence assessment report: large research-led partnerships. in: generation victoria figshare project figshare living systematic review: . introduction-the why, what, when, and how living systematic reviews: . combining human and machine effort living systematic reviews: . living guideline recommendations the evidence based-medicine working group protocol for the development of a consort extension for rcts using cohorts and routinely collected health data utilities and quality-adjusted life years understanding dalys measuring the burden of disease: disability adjusted life year (daly) toward icd- : improving the clinical utility of who's international classification of mental disorders staff who: international classification of functioning, disability and health: icf: world health organization methodology in core outcome sets. color dis the victorian child's lifecourse journey in data accessed generation victoria: generation victoria official website accessed the development of the older persons and informal caregivers survey minimum dataset (topics-mds): a large-scale data sharing initiative the fair guiding principles for scientific data management and stewardship five safes: designing data access for research statement: updated guidelines for reporting parallel group randomised trials better reporting of interventions: template for intervention description and replication (tidier) checklist and guide editorial perspective: stop describing and start fixing -the promise of longitudinal interventioncohorts healthy life trajectories initiative (helti) potential of 'stacking' early childhood interventions to reduce inequities in learning outcomes is stacking intervention components cost-effective? an analysis of the incredible years program the warwick hip trauma evaluation -an abridged protocol for the white study: a multiple embedded randomised controlled trial cohort study review of an innovative approach to practical trials: the 'cohort multiple rct' design core outcomes in ventilation trials (covent): protocol for a core outcome set using a delphi survey with a nested randomised trial and observational cohort study publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations we thank all members of genv and its working groups who have reviewed the paper and contributed to developing processes. as genv's scientific director, mw is the sponsor-investigator responsible for its overall design, including enabling it for trials. mw and ad conceived this paper. mw, yjh and hw drafted the first manuscript. md, mf, fo, mp, kpp, rs and ad each provided critical review and contributed to its contents. all authors reviewed the manuscript and had final approval of the submitted and published version of this paper. the authors comprise the current genv trials working group. this statement of intent outlines how the large genv cohort can serve as a platform to increase the number, speed, range and duration of trials for parents and children. much remains to be worked out. however, its innovative design could guide best practice for these groups which currently lack robust generalizable evidence, and whose good health and wellbeing are so vital to the functioning of populations going forward. supplementary information accompanies this paper at https://doi.org/ . /s - - -x. availability of data and materials no data are yet available. it is intended in the future that genv data analyses will be supported for all researchers meeting governance requirements such as the five safes principles. a range of materials are available at genv's figshare project [https://mcri.figshare.com/projects/generation_victoria/ ].ethics approval and consent to participate genv is endorsed by the royal children's hospital human research ethics committee (hrec . ). parents will provide written informed consent. all trials conducted within/alongside genv will have appropriate ethical approval. not applicable (no participants enrolled). the authors declare that they have no competing interests. key: cord- -vfl i p authors: largent, emily a; lynch, holly fernandez title: paying participants in covid- trials date: - - journal: j infect dis doi: . /infdis/jiaa sha: doc_id: cord_uid: vfl i p trials are in development and underway to examine potential interventions for treatment and prophylaxis of coronavirus disease (covid- ). how should we think about offering payment to participants in these trials? payment for research participation is ethically contentious even under ideal circumstances. here, we review functions of research payment—reimbursement, compensation, and incentive—and identify heightened and novel ethical concerns in the context of a global pandemic. we argue that covid- trial participants should usually be offered reimbursement for research-related expenses, and compensation for their time and effort, as for other types of research under usual circumstances. given increased risk of undue influence against pandemic background conditions, incentive payment should be avoided unless essential to recruitment and retention in important trials whose social value outweighs this risk. where essential, however, incentives can be ethically permissible, so long as reasonable efforts are made to minimize the possibility of undue influence. with much of the globe rushing to respond to coronavirus disease (covid- ) , clinical trials to evaluate safe and effective options for treatment and prevention are critical. the trials are diverse, examining new and repurposed drugs and vaccines at various stages of development, and involving a variety of designs with a range of opportunity for direct benefit. a wide variety and number of research participants are needed to enroll in these trials, reflecting a spectrum of experience with covid- -including healthy individuals, individuals exposed to the virus, individuals experiencing different levels of disease severity, and recovered individuals-as well a diversity of age, sex, race and ethnicity, socioeconomic status, medical comorbidities, and more. we have already seen that many individuals are eager to try nearly anything that has exhibited some promise against this disease, which may facilitate trial recruitment [ ] . but what role will payment play in encouraging trial participation-and what role should it play? paying research participants is ethically contentious under ideal circumstances and pandemic circumstances are far from ideal. concerns about offering payment are therefore likely to be heightened in this context [ ] . existing frameworks for evaluating the ethics of paying research participants offer a strong foundation for assessing the acceptability of payment in covid- trials. nonetheless, consideration must be given to the unique medical, financial, and institutional circumstances wrought by the pandemic. we argue that participants in covid- trials should be offered reimbursement for research-related expenses and compensated for their time and effort, just as they should be for any other type of trial. given the pandemic's devastating economic effects, as well as the fact that risks may be higher or more uncertain in covid- trials than in nonpandemic research, there is an increased likelihood of undue influence stemming from incentive payments. still, it may be permissible to offer incentives when necessary to recruit and retain participants for important trials offering the possibility of social value sufficient to outweigh these concerns, so long as steps are taken to minimize the risk that participants will be unduly influenced. paying research participants serves several discrete functions [ ] . first, participants may be reimbursed for out-of-pocket expenses incurred as a result of research participation, such as copayments or travel costs. second, payment can compensate participants for the fair value of their time and effort expended in research participation. third, incentive payments go beyond what is necessary to either reimburse or compensate, with the intention of improving recruitment or retention. offers of payment that fall into any of these categories can be ethically acceptable; however, each function raises distinct ethical considerations influenced by the pandemic. offering reimbursement for reasonable, research-related expenses to restore participants to their financial baseline should be viewed as the ethical default in covid- trials, as in research more broadly. this treats participants fairly by preventing them from having to pay to contribute to socially valuable research that may not, or is not expected to, benefit them directly. where direct benefit is possible, covering expenses can also promote distributive justice by making those potential benefits more widely accessible without regard to participants' financial need or wealth [ ] . considering that a high percentage of the population is likely to be infected with severe acute respiratory syndrome coronavirus (sars-cov- ), the current absence of curative therapies or a vaccine, and the widespread financial hardship and exacerbation of economic disparities caused by the pandemic, covid- trial participation must not be limited only to those who can afford it. making participation widely accessible is especially important in light of the disproportionate impact this virus is having on minority communities [ , ] . while there are of course many barriers to trial participation, financial barriers are among the most easily modified [ ] . although reimbursement should be the rule, there will be exceptional circumstances when it can be ethically acceptable to proceed without it. in the context of covid- , for example, investigators and institutions may be initiating trials without traditional sources of public research funding or with only limited support, such as the provision of product, by commercial companies. in these instances, there may not be adequate funding to reimburse participants. lack of funding should not preclude important trials from proceeding, so long as participants are made aware of the financial implications of enrollment and reasonable efforts are made to minimize financial burden. if a limited budget is available for reimbursement, it may be acceptable to reimburse only select participants-such as those with the greatest financial need-or to cover expenses up to a prespecified limit without necessarily reimbursing them all [ ] . compensation also should be viewed as the ethical default because it helps to minimize the chance participants will be exploited by receiving benefits that are disproportionately low compared to the burdens they undertake and the value they contribute to research. while the prospect of direct benefit is relevant to avoiding exploitation, research benefits are not always present and are never certain. thus, it often makes sense to compensate participants for their work via a wage-payment model, using a fair local wage for similarly burdensome nonresearch endeavors as a benchmark [ , ] . this is not intended to make participants better off as compared to their financial baseline or even to fully compensate for participants' opportunity costs but rather to acknowledge the value of their time and effort. compensation can also help distinguish research activities, with their distinct goals and risks, from clinical care, signaling that participants are contributing to science and that individual benefit may not result from their research participation [ ] . this is likely to be particularly important for covid- patients, because the dearth of compelling treatment options means that clinical care will often incorporate experimental methods and research. nevertheless, it may be ethically acceptable to proceed without compensation when reasonable budget constraints preclude it, as above. when compensation is offered, ethical concerns can arise, which sometimes point in opposite directions. first, if the amount is unfairly low, it will not adequately address the possibility of exploitation; this concern can be addressed by offering more compensation. yet, relatively higher compensation may be linked to a second concern, which arises when circumstances extrinsic to a trial transform payments intended as compensation into de facto incentives. this is especially likely in the context of covid- . compensation is intended to render research participation as financially attractive as other, nonresearch opportunities that demand a similar amount of time, burden, and skill. however, the availability and type of those nonresearch opportunities can be impacted by a variety of factors. even for those who have remained healthy, the covid- pandemic is causing a human tragedy with significant economic impact, including widespread layoffs, small business closures, hiring freezes, and dramatic market fluctuations [ ] . these challenges have exacerbated longstanding economic inequalities and laid bare the fragility and inadequacy of the social safety net in many countries, including the united states. these economic and social challenges are only likely to increase as the pandemic continues. against this backdrop, many prospective covid- research participants will no longer have meaningful alternatives for paid work, which may contribute to a perception-and perhaps reality-that paid research participation is their best opportunity to make money. this perception may be heightened if compensation is increased to the extent necessary to avoid exploitation, even if the total amount of money remains relatively modest. these difficult background circumstances do not make offers of payment to covid- research participants impermissible. it would be perverse to withhold fair compensation simply because participants are facing economic hardship [ ] . rather, in light of pandemic circumstances-similar features of which may be replicated in other contexts, including research conducted in low-and middle-income countries or with participants whose nonresearch options are limited even in the absence of a pandemic-offers of compensation may raise ethical concerns akin to incentives [ ] . thus, institutional review boards (irbs) should consider whether it is sometimes appropriate to treat them as such. rather than aiming to satisfy obligations of fairness to participants or to set research on a par with other payment-generating activities, incentives aim to push trial enrollment higher on an individual's choice set-or incidentally have that effect based on available alternatives. as such, incentives are not ethically obligatory. however, to the extent incentives can contribute to recruitment and retention, and thus to efficient trial completion with adequate statistical power, they can be ethically important [ ] . incentives may also advance distributive justice by increasing willingness to enroll, thereby spreading the burdens of research participation over a larger swath of the population and avoiding concentrating those burdens exclusively on individuals who are the least well off financially by making research more broadly attractive. yet, incentives can be ethically fraught. some people worry that incentives may coerce research participants. the prevailing view is that coercion entails a threat to violate an individual's rights or not fulfill an obligation to her unless she complies with some request, in a circumstance in which she has no reasonable alternative but to comply [ ] . by this definition, genuine offers of payment are not coercive because they are not threats [ ] . importantly, feeling that there is no reasonable alternative to make a similar amount of money-a feeling likely to arise for at least some prospective covid- trial participants-is not the same as being coerced. a more salient ethical concern is that incentives may unduly influence participants, although this is more complex than just making a decision motivated by a desire for financial benefit. undue influence occurs when an excessive reward leads the recipient to make a choice that is unreasonably against her selfdefined values, interests, or responsibilities [ , ] . a decision would be objectively unreasonable if it reflected a level of risk an irb would not or should not approve for participants in the target population. for example, it would be objectively unreasonable for persons with a known hypersensitivity to hydroxychloroquine to participate in a trial assessing that drug's effect on the progression of covid- [ ] . approval of a trial protocol by a well-functioning irb should generally eliminate concerns that incentives will lead to objectively unreasonable decisions. in contrast, research participation may be subjectively unreasonable if it is discordant with an individual's particular values and interests or if the risks (should they materialize) would be particularly burdensome for the individual to bear [ ] . because irbs are tasked with making population-level judgments, they cannot be expected to assess the subjective reasonableness of trial participation for every individual. thus, irb review cannot eliminate the possibility that that incentives will lead to subjectively unreasonable decisions-and, therefore, to undue influence. for instance, a jehovah's witness considering participation in a trial of intravenous immunoglobulin-a blood plasma product-for covid- might have consciencebased concerns. similarly, someone who is highly risk-averse may prefer to avoid the uncertainties associated with research participation. either of these individuals could be encouraged by offers of incentive payment to set aside those concerns. while this may seem worrisome, in practice, it is quite hard to distinguish between cases in which an offer of payment has unproblematically tipped the balance in favor of an otherwise undesirable but not unreasonable choice and those in which it has problematically tipped the balance in favor of an unreasonable or irrational choice. in many cases, a decision to enroll in research will reflect a participant's reasoned judgment that, in these circumstances, participation is aligned with her overall interests, even if certain considerations might have weighed in the other direction. decisions are often multifactorial, with various pros and cons; the fact that all cons have not been resolved does not necessarily render a decision subjectively unreasonable. acknowledging this challenge, the best irbs can do is to minimize the possibility of undue influence for trial participants on the whole by making it unlikely for research participation to constitute an objectively unreasonable choice for members of the target study population. they should also make sure the consent process alerts participants to factors that might make participation subjectively unreasonable [ ] . although careful irb review generally constitutes a critical bulwark against undue influence, there are additional considerations when evaluating the potential for undue influence in covid- trials. first, given that there is still much to be learned about sars-cov- and so little is known about the various interventions under study in this context, we may justifiably be concerned that it will be difficult for irbs to make risk-benefit determinations confidently. therefore, participation even in irb-approved studies may not always be truly objectively reasonable for the target study population, or there may be disagreement about what is objectively reasonable. moreover, the challenge of evaluating this research may be exacerbated by the heightened burdens currently facing irbs. the sheer volume of covid- research proposals being put forward [ ] and the dire need for clinical advancement mean that members are being asked to review a tremendous number of protocols as quickly as possible, often while meeting remotely to promote social distancing, all of which may influence the quality and nature of review. second, the global scale of the pandemic and associated morbidity and mortality may render even quite risky research objectively reasonable on the basis of high social value. for example, we have already seen vaccine trials proceed without the usual animal trials, and there is increasing discussion of using challenge trial design to speed vaccine development, a design far riskier for sars-cov- than for viruses used in other recent challenge trials given the lack of a proven cure [ ] [ ] [ ] [ ] . it is widely accepted that trials can be justified on the basis of benefits to society; in the absence of direct benefits, it may be desirable to offer larger incentive payments similar to hazard pay offered to emergency workers or others performing dangerous work-as noted above, this can help make participation attractive across a wider socioeconomic swath [ ] . yet, larger incentives might also increase the likelihood of undue influence by making it more likely that people will make subjectively unreasonable decisions [ ] . a third concern is that incentives, because they go beyond amounts available through participating in other activities, may cause individuals "to lie, deceive, or conceal information that, if known, would disqualify them as participants in a research project" [ ] . such obfuscation can have effects [ , ] . first, it may expose individuals to research-related risks that exclusion criteria were designed to shield them from. second, it may jeopardize the scientific integrity of the research. this can be of particular concern when it is not possible to rely on objective, independently verifiable measures to confirm trial eligibility [ ] . in response to these challenges, one might consider the most conservative course of blocking incentives for covid- trials. however, this approach could inhibit recruitment, retention, or both, impeding the conduct of critically important research, in turn creating a greater possibility that participants will be exposed to research risks without realizing the social benefits that initially justified them. there are potential errors here: ( ) inhibiting ethically acceptable trials out of an abundance of caution, or ( ) risking undue influence and deception by incentivizing participation in covid- trials [ ] . given the importance of research in response to the pandemic, as well as limited, albeit encouraging, empirical data suggesting that higher offers of payment increase participants' perceived risk as well as time spent reviewing research-related risks [ , [ ] [ ] [ ] , our view is that incentive payments can be ethically permissible, despite the residual risk of undue influence and deception. rather than blocking incentives for covid- trials due to the concerns raised above, the better approach is to differentiate those circumstances in which incentive payments are truly essential to boost recruitment and retention for important research. if they are not, as may be the case for covid- research that offers participants other benefits or that can rely on altruism born of social solidarity, it is best to avoid incentive payments. this is also the more efficient approach; why spend resources on incentives that are unneeded? however, incentives sometimes will be critical. for example, covid- trials that offer no or low potential for direct benefit and impose substantial burdens and risks, such as early phase trials focused on dosing and safety, may otherwise fail to adequately enroll participants. of course, the fact that participants may have qualms about participating that need to be overcome by offering incentives might suggest that concerns about undue influence are highest in these circumstances. yet these are also the covid- trials for which irb attention is likely to be most intense and focused, potentially reducing concerns about objectively unreasonable decisions to enroll. risks of subjectively unreasonable decisions remain, but should be viewed in a similar context to other research risks, meaning that they can be justified when both minimized and reasonable in relation a study's potential for benefit. thus, incentives should be limited to covid- trials with adequate time and research personnel to facilitate robust informed consent processes that can help prospective participants carefully consider the risks, burdens, and discomforts of participation, as well as those that can adopt objective measures of eligibility and adherence. they should also be limited to those trials with sufficient importance to the battle against covid- that their potential benefits can overcome residual concerns about undue influence. relatedly, care should be taken to avoid incentive payments being used to draw participants into lower priority trials [ ] . irbs are tasked with minimizing-not eliminating-the possibility of undue influence; we should accept that minimization may look different for covid- trials compared to other research. offering payment for trial participation intended to combat a pandemic that is coupled with economic distress raises unique considerations. we argue that reimbursement and compensation should be offered in covid- trials as a matter of fairness, as is true for other types of clinical research. yet the economic stressors of the pandemic may cause compensation to be experienced as an incentive, raising concerns about undue influence, while the usual protections against undue influence may also be weakened by pandemic circumstances. rather than abandoning the utility of incentives, we recommend that they be limited to those covid- trials that truly need them, that will permit undue influence to be minimized, and whose social value and importance can outweigh residual risks of undue influence. this suggests that financial incentives will be most appropriate for covid- trials without the prospect of direct benefit, although in the face of such a massive global threat, altruism and a call to duty may render incentives even less critical. hospital orders for old malaria drugs have spiked amid coronavirus pandemic conventional wisdom versus actual outcomes: challenges in the conduct of an ebola vaccine trial in liberia during the international public health emergency fernandez lynch h. a framework for ethical payment to research participants what makes clinical research ethical covid- and african americans evidence mounts on the disproportionate effect of covid- on ethnic minorities addressing financial barriers to enrollment in clinical trials differential payment to research participants in the same study: an ethical analysis what's the price of a research subject? approaches to payment for research participation human research subjects as human research workers informative inducement: study payment as a signal of risk kff health tracking poll -early economic vulnerability and payment for research participation precarity, clinical labour and graduation from ebola clinical research in west africa the continuing unethical conduct of underpowered clinical trials paying research participants: regulatory uncertainty, conceptual confusion, and a path forward payment for research participation: a coercive offer undue inducement: nonsense on stilts? hydroxychloroquine in outpatient adults with covid- paying research participants: the outsized influence of "undue influence a real-time dashboard of clinical trials for covid- [published online ahead of print researchers rush to test coronavirus vaccine in people without knowing how well it works in animals human challenge studies to accelerate coronavirus vaccine licensure ethical considerations for zika virus human challenge trials ethics of controlled human infection to address covid- compensation for cures: why we should pay a premium for participation in 'challenge studies on paying money to research subjects: 'due' and 'undue' inducements association between financial incentives and participant deception about study eligibility exploring ethical concerns about human challenge studies: a qualitative study of controlled human malaria infection study participants' motivations and attitudes making the case for completion bonuses in clinical trials empirical assessment of whether moderate payments are undue or unjust inducements for participation in clinical trials perceptions of financial payment for research participation among african-american drug users in hiv studies the influence of risk and monetary payment on the research participation decision making process when clinical trials compete: prioritising study recruitment key: cord- - rbcwb authors: serpa neto, ary; hodgson, carol title: will evidence-based medicine survive the covid- pandemic? date: - - journal: ann am thorac soc doi: . /annalsats. - ed sha: doc_id: cord_uid: rbcwb nan while mankind is facing the worst global healthcare crisis of this century, our use of evidence-based medicine has suffered major setbacks. as recently discussed ( ) , social network, television shows, and other media platforms have been flooded by "experts," who have voiced strong opinions on the treatment of patients with coronavirus disease . in this setting, hydroxychloroquine has been portrayed as a potential lifesaving drug in the current pandemic, mainly based on opinion or results of small clinical studies and uncontrolled experiments ( , ) . one common argument for the use of an untested intervention during a pandemic is that "we must do something." this may be dangerous and abandons the principles of evidence-based medicine. indeed, physicians using an untested intervention at the bedside may suffer from attribution bias, which is a selective observation of favorable effects attributed to the intervention that lead to undue confidence in its effectiveness. the other available possibilities, such as harm resulting from the direct use of the intervention or a good recovery independent from it, can only be drawn with well-conducted clinical trials. at the time of writing this editorial, the use of hydroxychloroquine for covid- is a clear example of practice changing despite limited evidence to support its use. hydroxychloroquine has failed to prove beneficial in trials when used as a potential treatment for previous viral diseases ( ) ( ) ( ) . specifically, in covid- , the available evidence points toward a neutral effect of the drug ( , ) , with some studies suggesting harm ( ) . nevertheless, to date, there are no well-powered randomized clinical trials testing hydroxychloroquine in this group of patients to inform safety and effectiveness. in the face of the covid- pandemic, it is understandable that physicians and patients are scared, overwhelmed, and want quick answers. there are concerns about the time to complete randomized clinical trials, and this is used as a justification to accept anecdotal and low levels of evidence. indeed, the traditional approach to validate a new treatment is to complete lengthy phase i, ii, and iii studies. one major challenge during the pandemic is to design clinical trials that can mitigate these concerns and quickly identify effective or harmful interventions to improve patient outcomes. careful consideration is required to optimize trial design to achieve this, and several novel trial designs are available and could be considered. these include adaptive sequential designs, responseadaptive randomization, historical and dynamic borrowing, multistage multiarm trials, shared controls, and strategies that aim to "pick the winner" to identify early which treatments are effective. a good example of trial design that allows this flexibility is bayesian design. bayesian trials allow evidence about treatment to be continually updated with new information as it becomes available, maintaining trial integrity ( ) ( ) ( ) . another benefit of the bayesian approach is to estimate the probability that a treatment is effective rather than focus on whether it is effective or not according to p values. frequentist classical trials rely on previous knowledge to calculate sample size and define features of the study according to known assumptions about the treatment, something not widely available during a pandemic. also, classical designs are less flexible, and if the assumptions are not met, the study will end without providing useful evidence. in a pandemic with more than , new cases per day, the use of a bayesian adaptive trial design can quickly incorporate existing evidence, drop interventions that have a higher probability of futility, redirect patients to be randomized to the most promising ones, and constantly include new and potential candidate interventions. in this issue of annalsats, casey and colleagues (pp. - ) ( ) describe the study protocol of a randomized, doubleblind, placebo-controlled clinical trial (orchid trial) assessing the impact of hydroxychloroquine in hospitalized adults with confirmed severe acute respiratory syndrome coronavirus (sars-cov- ) and symptoms of acute respiratory infection. the primary outcome of the study is the patient's clinical status days after randomization, assessed by a seven-category ordinal outcome scale. an important feature is that data can be collected from the electronic health record, decreasing the person-to-person contact, conserving personal protective equipment, and reducing the risk of infection. a major strength of the study is to employ a bayesian framework, allowing multiple interim analyses, the possibility of incorporating new external evidence, and the early stopping of the trial according to predefined probabilities of benefit or futility. in the last years, bayesian adaptive clinical trials have been increasingly used to hasten the overall trial process ( , ) . in general, these studies share common features, especially algorithms to greatly reduce the sample size needed to assess the intervention without lowering the statistical power of the study. as a result, interventions can progress more quickly through all the processes, which is urgently needed and of utmost importance during a pandemic. a quick approval of a potential intervention is expected by policymakers and consumers, as early use of an effective treatment may improve patient outcomes and prevent harm. however, the risk of approving an ineffective or unsafe intervention is not often considered. indeed, withdrawing an approved therapy can be challenging, disruptive, and sometimes impractical. the orchid trial investigators expect to finish the study with a sample size of patients, which is realistic and acceptable as the united states of america currently has z , new cases a day. in a recent simulation study ( ) , the optimal sample size for clinical trials decrease with the infectivity of the epidemic. this suggests that a bayesian adaptive design allows the flexibility needed to adapt the study to specific parameters and stages of the epidemic. one potential limitation of orchid trial, and several other trials testing therapies in covid- , is evaluation of the primary outcome in a short time frame. though understandable because of the urgent need for answers, we have learned that these patients usually have a longer convalescent period. fifteen days is relatively short and may not accurately capture the majority of patients who have died or recovered. a longer period of observation is somewhat problematic when there is very rapid randomization, such as during the pandemic, as it delays the trial results. this may interfere with the interim analyses, as many randomized patients will not have a measure of the outcome of interest, making it difficult for the data safety and monitoring committee to recommend stopping the trial if required. in this time of uncertainty, when clinicians are desperately seeking effective therapies to fight covid- , the orchid trial provides several unique solutions to the constraints of traditional clinical trials. these include processes for rapid administrative and regulatory approvals, production of visually identical placebo pills, the use of interactive platforms for informed consent, and flexible bayesian trial design. studies like the orchid trial, and others in the field, are a whisper of rationality during irrational times and rekindle the hope that evidence-based medicine will survive the covid- pandemic. n in defense of evidence-based medicine for the treatment of covid- acute respiratory distress syndrome in vitro inhibition of severe acute respiratory syndrome coronavirus by chloroquine multicenter collaboration group of department of science and technology of guangdong province and health commission of guangdong province for chloroquine in the treatment of novel coronavirus pneumonia. expert consensus on chloroquine phosphate for the treatment of novel coronavirus pneumonia chloroquine for influenza prevention: a randomised, double-blind, placebo controlled trial a randomized controlled trial of chloroquine for the treatment of dengue in vietnamese adults hydroxychloroquine trial team. effects of hydroxychloroquine on immune activation and disease progression among hiv-infected patients not receiving antiretroviral therapy: a randomized controlled trial association of treatment with hydroxychloroquine or azithromycin with in-hospital mortality in patients with covid- in new york state observational study of hydroxychloroquine in hospitalized patients with covid- effect of high vs low doses of chloroquine diphosphate as adjunctive therapy for patients hospitalized with severe acute respiratory syndrome coronavirus (sars-cov- ) infection: a randomized clinical trial arguing for adaptive clinical trials in sepsis contemporary strategies to improve clinical trial design for critical care research: insights from the first critical care clinical trialists workshop using bayesian adaptive designs to improve phase iii trials: a respiratory care example orchid protocol committee and the national heart, lung, and blood institute prevention and early treatment of acute lung injury (petal) network investigators. rationale and design of orchid: a randomized placebo-controlled clinical trial of hydroxychloroquine for adults hospitalized with covid- bayesian adaptive clinical trials for anti-infective therapeutics during epidemic outbreaks author disclosures are available with the text of this article at www.atsjournals.org. key: cord- -zj ro j authors: dipaola, joshua d.; yindee, marnoch; plotnik, joshua m. title: investigating the use of sensory information to detect and track prey by the sunda pangolin (manis javanica) with conservation in mind date: - - journal: sci rep doi: . /s - - -x sha: doc_id: cord_uid: zj ro j pangolins are of conservation concern as one of the most heavily poached, yet least understood mammals. the sunda pangolin (manis javanica) in particular is a critically endangered species. here, we investigate the behaviour of these pangolins, for the first time, using a battery of cognitive tasks based on a manipulation of available sensory information. in an object-choice task in which only one of two containers was baited with food, the pangolins were able to find the food with olfactory information alone (n = ), but not with visual or acoustic information alone (n = ). the single subject tested on all three domains was further tested on how he used smell to find food by providing him with an opportunity to find it from a controlled distance or by using scent trails as a guide. the results suggest that our subject may have the capacity to exploit scent trails left by prey which can be tracked to a final source, though we found no evidence to suggest that he had the ability to initiate hunts based on distant prey odors. despite the small sample size, this is the first controlled experiment to investigate pangolin foraging behaviour and cognition, which may have implications for the future protection of pangolin habitat based on the location of prey species. pangolins are of conservation concern as one of the most heavily poached, yet least understood mammals. the sunda pangolin (manis javanica) in particular is a critically endangered species. here, we investigate the behaviour of these pangolins, for the first time, using a battery of cognitive tasks based on a manipulation of available sensory information. in an object-choice task in which only one of two containers was baited with food, the pangolins were able to find the food with olfactory information alone (n = ), but not with visual or acoustic information alone (n = ). the single subject tested on all three domains was further tested on how he used smell to find food by providing him with an opportunity to find it from a controlled distance or by using scent trails as a guide. the results suggest that our subject may have the capacity to exploit scent trails left by prey which can be tracked to a final source, though we found no evidence to suggest that he had the ability to initiate hunts based on distant prey odors. despite the small sample size, this is the first controlled experiment to investigate pangolin foraging behaviour and cognition, which may have implications for the future protection of pangolin habitat based on the location of prey species. one of the most important confounds in the design of comparative cognition research centers on the question of ecological validity. drawing cognitive comparisons across evolutionarily distant taxa requires a basic understanding of an animal's umwelt, with particular attention to how behaviours such as foraging and problem-solving are affected by sensation. for instance, while developing puzzles that require visual access and manipulation with tools makes ecological sense for animals like chimpanzees - and corvids , , apparatuses that provide olfactory information and can be manipulated using an animal's own body parts are more relevant for elephants [ ] [ ] [ ] and dogs , . understanding the sensory perspective of a particular species is even more critical when designing behavioural or cognitive tests for animals that have rarely, if ever been studied previously. investigations of the proximate, cognitive mechanisms that underlie an animal's decision-making processes can have important implications both from a theoretical (i.e., for understanding how cognition evolves across species to help an animal forage, interact with conspecifics, and avoid predation, for example) and an applied conservation perspective. for the latter, there is a growing need for conservationists developing endangered species strategies to collaborate with ecologists, biologists and psychologists in order to gain a greater understanding of animal behaviour and cognition. to this end, the experimental object-choice task can be used to better understand the importance of particular sensory modalities in an animal's foraging behaviour. this task typically involves the presentation of two or more options where only one is baited with food. the experimenter can control how much or how little sensory information is available to the animal when it investigates and subsequently makes a choice between the provided options. this task has been used to investigate different capacities, including cue-following in the visual (e.g. [ ] [ ] [ ] ), subjects. this study was conducted between january-march and june-july, with two wild-born, sexually mature sunda pangolins (m = , f = ) housed at the livestock and wildlife hospital of the faculty of veterinary science, mahidol university, kanchanaburi, thailand. permission to conduct this research was granted by the director of the hospital (co-author, m.y.), in whose care these pangolins were under. our initial sample included three subjects (m = , f = ), however, one pangolin showed no interest in the testing apparatus while being habituated to the research materials and thus, no data were collected on this subject. the two subjects that participated in the study were tested directly within their enclosures to minimize stress associated with handling. the pangolins' housing consisted of semi-enclosed rooms in an outdoor barn at the veterinary hospital, and the animals had access to water, shelter and climbing structures. the pangolins had strict diets developed by the veterinary staff to maintain their health. they were each given a predetermined quantity of dead weaver ants (oecophylla smaragdina) once per day (in the evening), with occasional supplements of live ants. due to the predetermined, singular feeding period as well as the relatively fragile nature of the pangolins' health in captivity, we did not withhold or reduce their diet to increase their motivation levels for testing. the hunter college institutional animal care and use committee (protocol # jp-pangolin- / ) reviewed and approved this research, and the methods were carried out in accordance with the relevant guidelines and regulations. procedure. these animals were first tested in phase i of this study on their ability to locate food, a sample of ants, across foraging tasks within three sensory domains: visual, acoustic, and olfactory (see plotnik et al. for similar procedures on which the current research is based). because sunda pangolins are nocturnal , , , , , all testing was conducted between - hrs. for initial sensory testing, the pangolins were presented with a basic object-choice task in which two hard plastic cm × cm × cm containers, one containing food the other not, were presented inside a wooden testing chamber. each container rested flat on top of its own sliding wooden platform, which had a cm vertical wooden handle attached so that the experimenter could manipulate the position of the containers throughout a trial without physically entering the testing chamber (fig. ) . because the pangolins were wild-caught and had limited experience with novel objects prior to testing, we first habituated the animals to the paradigm by allowing them to retrieve food from a single baited container without a lid. we then gave them an opportunity to search two open containers, with only one baited, and then repeated this inside the testing chamber. once they voluntarily entered the chamber and searched the two containers consistently, testing www.nature.com/scientificreports www.nature.com/scientificreports/ began on the two-object choice task. for each trial of testing in phase i, the pangolin could first approach two locked containers (with lids) placed adjacent to each other on their respective wooden platforms. during this time, the pangolin could directly touch and investigate both containers. after s, the experimenter used the vertical handles to pull the two containers to the back of the chamber where a short, cm partition separated the two containers. the purpose of this partition was to isolate each container to make the decision-making process as clear as possible to both the subject and experimenter. the pangolin was then given s to make a choice. the criterion for 'choice' required that the pangolin pass the partition and touch the container with its snout or claws. this criterion for 'choice' was used for all conditions except phase ii 'olfactory distance' testing (see below for details on this condition). the experimenter then manually unlocked the lid of the chosen container and withdrew the alternative by lifting it out of the apparatus using the vertical handle. if a pangolin failed to make a choice . the view is from behind the testing apparatus. once the pangolin enters the chamber (from the top of the image), the metal mesh screen is removed and the pangolin can investigate the two containers. the containers are then pulled to the back of the chamber, at which point the pangolin can make a choice and the lid of the chosen container is removed. (c) experimental setup for 'olfactory distance' testing in phase ii. similar to phase i, the pangolin entered the chamber (from the top of the image), the screen was removed, and the pangolin could investigate the two containers, which were positioned on top of pvc pipes above the pangolin's head. this photograph represents a -cm training trial. (d) experimental setup for 'scent trail' testing in phase ii. the pangolin could again enter the chamber but there was no screen to remove; the pangolin could follow the scent on either of two wooden planks that led to a baited ('food' plank) or unbaited ('water' or 'mint' plank) container. this image does not include the cloth tape applied to the planks prior to testing. all photos by j.d.d. in the time allotted, the trial was reset. all experimental testing was voluntary, with the pangolins being allowed to enter and exit the testing area at will as soon as the testing apparatus was rebaited after each trial. rebaiting of containers happened out of view of the pangolins between trials, and metal mesh screens were used to temporarily block off the apparatus while it was rebaited. because we did not want to handle the pangolins or restrict their movement within their enclosures, there was no specific inter-trial interval (iti) established for testing. in phase i of testing, the pangolins were presented with three conditions that each provided sensory information about the presented food from a single modality (see supplementary video s ). the containers were manipulated so that they were either: a) opaque, held g of dead ants and were locked with perforated lids (so that the pangolins could smell but not hear or see the ants -the 'olfactory' condition), b) transparent with solid lids and g of dead ants (so that the pangolin could see but not hear or smell the ants -the 'visual' condition), or c) opaque with solid lids and live ants (so the pangolin could hear but not see or smell the ants -the 'acoustic' condition). containers and lids were made opaque using black duct tape applied on the inside. in this last condition, the average weight of the live ant nests (the ants plus the leafy substrate from their environment) in the closed baited container was g. in addition, even though the pangolins are nocturnal, the 'visual' condition was conducted either in the presence or absence of an artificial light to investigate whether visual light made any difference in the pangolin's ability to locate the food. control trials utilized opaque containers, solid lids and g of dead ants (so that pangolins had no access to sensory information about the ants), and allowed us to investigate whether the pangolins were using cues independent of the food present -e.g., inadvertent experimenter cuing -to make a choice. the -g amount of dead ants for each trial within a non-acoustic condition was determined based on the average weight of a live ants' nest minus the substrate, as well as a relatively equal distribution of the pangolin's daily diet across an evening's test trials as determined by the hospital's veterinary staff. subjects were given the opportunity to participate in one testing session per day across both phase i and phase ii of this study. testing was conducted on consecutive days unless the subject failed to voluntarily participate, in which case testing was delayed until the next day. phase i conditions were presented in the following order: olfactory, visual no-light, visual light, acoustic untreated, and acoustic treated. the 'olfactory' condition consisted of four sessions of trials (eight test and four control). the 'visual' condition consisted of two parts: four sessions of trials (eight test and four control) presented in the absence of an artificial light source (no-light), followed by an equal number of trials in the presence of light. in the first three sessions of the 'acoustic' condition, live weaver ants were collected from the environment and immediately presented to the pangolin in the same containers for testing ('acoustic untreated' -three sessions of trials (eight test and four control)). it was very difficult to move the ants between containers once they had been collected due to their high level of aggression and fast movement. because we realized later that the ants may have left olfactory cues on the exteriors of the containers during collection, we ran an additional three sessions of trials in which we meticulously cleaned the outside of the containers between ant collection and presentation to the pangolin using a mild soap and water solution ('acoustic treated'). in all types of condition, containers were cleaned between sessions. in the 'acoustic' condition, the 'baited' container remained unchanged throughout a session (due to the impracticality of moving a live ant colony easily between trials), however, the 'baited' container was placed on each side of the apparatus an equal number of times. in this 'acoustic' condition, the pangolin was rewarded with a separate quantity of dead ants if it correctly chose the container with live ants, to avoid having to open and rebait the live ant container. because the 'olfactory' and 'visual' conditions utilized dead ants throughout, both containers were 'baited' or 'unbaited' an equal number of times within each session to avoid any confound of residual odor (e.g., from previously baited food or the pangolin's own scent). all trials across the 'olfactory' , 'visual' , and 'acoustic' conditions of phase i were pseudo-randomised within a session so that each session consisted of an equal number of trials of left and right correct choices (based on the side of the chamber in which a container was placed), no one side was baited with food more than three consecutive times in a row, and the first three trials for each session across all conditions were always tests rather than controls to avoid a frustration effect. the female pangolin, 'betty, ' was only tested on a portion of her first condition, the initial 'olfactory' condition (table ) , because we noticed a significant decrease in both her nightly food intake and her interest in entering the experimental apparatus as testing progressed. because we did not want to negatively impact her welfare state, she was not subjected to any further testing. the male pangolin, 'pluto' , was tested on all of the conditions as he remained highly motivated throughout testing. in phase ii of the study, the male pangolin was tested on two different olfactory conditions. the 'olfactory distance' condition specifically aimed to investigate whether pangolins may detect their prey or initiate a hunt based on distant olfactory cues (see supplementary video s ). this condition was similar to the phase i olfactory condition with the following exceptions. first, in 'olfactory distance' trials, the pangolin could not physically touch either container while he first investigated them. instead, he could only smell, but not touch the containers from a distance of , or cm. second, the pangolin was given s to investigate both containers in 'olfactory distance' testing compared to the s used for the olfactory condition in phase i. the investigation time was increased for 'olfactory distance' trials to ensure that the pangolin had sufficient time to navigate the additional spatial components of this condition (details below). third, the containers were no longer presented to the pangolin for investigation on the floor of the apparatus in 'olfactory distance' . they were now positioned overhead on a vertical plane to best mimic what the pangolin might spatially encounter in an arboreal hunt; he would now need to smell upwards in order to gain access to the food's olfactory information. the containers were placed porous lid-side down on top of a solid, pvc tube cut to the predetermined length/distance and set on top of a wooden plank overhanging the testing chamber (fig. ) . holes were cut into this plank so that the pangolin could smell into the tubes (and thus the distant containers) above it. in any given training or test trial of the 'olfactory distance' condition, after the pangolin entered the testing chamber, he had s to investigate the two sides. the containers were then placed on the floor of the chamber so the pangolin could choose one. the pangolin then had s to pass the -cm partition separating the two scientific reports | ( ) : | https://doi.org/ . /s - - -x www.nature.com/scientificreports www.nature.com/scientificreports/ containers, at which point the experimenter unlocked and removed the chosen container's lid, while the other container was removed (see supplementary video s ). this early designation of the pangolin's choice (i.e., before he made contact with a container) was used to minimize his opportunity to make a decision using olfactory information from within the containers once they were placed on the floor. the food reward used in this condition consisted of g of dead ants. since this was the pangolin's first experience with testing on a vertical plane, he was first trained with the containers placed directly overhead on the wooden plank without the added distance created by the pvc tubes. the pangolin reached the criterion of % correct in a single session of eight trials (i.e., seven out of eight trials correct) within his first training session. subsequently, he was trained on a distance of cm to ensure he understood the task with all of the apparatus components in place, reaching the same criterion in his third session. following completion of training, the pangolin participated in the test condition consisting of ten total sessions. each testing session included six test trials (two trials of each distance, randomised) and two control trials, for a total of eight trials per session. control trials across all 'olfactory distance' testing were conducted at the shortest distance ( cm) using the same control containers from phase i. as in phase i, in the 'olfactory distance' condition, once the containers were placed on the pvc by the experimenter, the metal mesh screen was removed so the pangolin could enter the apparatus. again, no inter-trial interval (iti) was set to avoid having to handle the pangolin or restrict his movements, however, he never entered the apparatus sooner than s after the containers were placed on the pvc (it usually took considerably longer), and he always had s to investigate them before they were removed and placed on the ground. thus, at least s passed between setting a trial and the pangolin having to make a choice. the 'scent trail' condition aimed to investigate whether pangolins could use scent trails to track their prey (see supplementary video s ). due to the pangolin's natural tendency to smell as it moved through the wooden chamber, no training was needed for this condition. test trials made use of two wooden planks on each of which a strip of cloth tape was applied to prevent the smeared liquid from running. a different scent was smeared on each tape strip using a sponge (approximately ml of scent was applied per strip, per trial). one of two stimulus containers (one baited and one unbaited) was placed at the end of each plank, on its side and with the locked lid facing the pangolin. once the animal voluntarily walked into the testing chamber, he had s in which to make a choice. as soon as he reached one of the two containers, the lid was unlocked and removed. for all test trials, the food (f) scented trail always lead to the baited container. this scent was extracted by taking dead weaver ants and pressing them within the barrel of a syringe, producing a concentrated liquid extract. in one type of test trial, the scent of ants (f) was applied to one trail while plain tap water (w) was applied to the non-food bearing trail (f vs. w). the w trail acted as a control for a scent 'smear' by providing comparable characteristics of the ant 'smear' without the odor of prey. for the other type of test trial, the scent of ants (f) was applied to one trail while the non-prey odor of mint (m) was applied to the alternative trail (f vs. m). the mint odor was produced by gathering wild mint leaves, dicing them and soaking them in warm water min prior to testing. the resulting extract (devoid of the leaves) was smeared on its own plank. this condition investigated whether the smell of a non-prey, naturally occurring odor was distracting or potentially more interesting than prey-derived odors. phase i control containers (i.e., opaque with solid lids) were used as the stimulus containers in the test trials for this condition as we wanted the subject to make olfactory decisions based on the trail scent without being influenced by the odor of the food inside the container. 'scent trail' control trials followed the same procedure as all prior testing in this condition except that the trail strips used were devoid of any smear. these trials again ensured the pangolins were not using any cues besides those which we intentionally provided for them to find the food. this condition consisted of ten sessions, with each session consisting of two control trials and six test trials (three f vs. m and three f vs. w trials). prior to each trial, scents were reapplied to their respective planks. phase ii pseudo-randomisation procedures were identical to that of phase i except that only the first two trials in visual -light visual -no light acoustic -untreated acoustic -treated betty ♀ / * / table . raw data for both subjects ('pluto' , the male, and 'betty' , the female) across all test and control conditions in phase i and ii. the ratios provided represent the number of trials in which the pangolin selected the correct, baited container (numerator) over the total number of trials for each respective condition (denominator). the top of each column provides an abbreviation of the condition tested with the type of trial listed below. control trials are represented by "c". test trials are represented by "t". in phase i, the olfactory condition was comprised of only one type of test trial, while visual and acoustic conditions included two separate types each. in phase ii, the 'olfactory distance' condition was comprised of three types of test trials based on the distance at which the stimuli were presented, and are delineated in the table by this distance in centimeters (cm). there were two types of test trials in the 'scent trail' condition. 'scent trail' test trials which provided a food vs. mint scent pair are represented by "f vs m". test trials which provided a food vs. water scent pair are represented by "f vs w". ratios that are bolded and marked with an asterisk indicate that the pangolin performed significantly better than chance (p < . ) (*binomial test, two-tailed). see the methods section for details on each condition across the two phases. a phase ii session were always test trials; the remaining trials within a session contained a randomised order of test and control trials. all experimental conditions were recorded using the built-in infrared nightshot capabilities on a sony fdr-ax (see supplementary videos s -s ). trials were coded for the pangolin's container choice in real-time by j.d.d. both subjects selected the baited container significantly more often than chance in the olfactory condition of phase i (p < . for both pangolins, two-tailed binomial test). the olfaction results for the female, however, reflect only partial completion for this condition. she did not participate in any other testing. the male did not perform significantly better than chance in any other condition in phase i or in the 'olfactory distance' condition in phase ii (table ). in the 'scent trail' condition, the male chose the food-scented trail significantly more often than chance across all trials ( / trials, p = . , two-tailed binomial test). although he chose the food-scented trail more often in both types of 'scent trail' test trial (the food vs. mint and food vs. water conditions) - . % in both -the results were not statistically significant when analysed separately ( / trials each, see table ). in this study, two pangolins (m = , f = ) demonstrated a capacity for finding food using olfactory information alone. one pangolin's performance (m = ) on the additional olfactory foraging tasks suggests he may have been using scent trails to track prey rather than olfactory information at a distance directly from the source. in addition, this pangolin was unable to use visual or acoustic information to find food when this was the only sensory information about the food's location available to him. the two pangolins' poor performance on control trials confirmed that they did not use any other inadvertent cuing from the experimenter to find the food. in order to discuss and interpret these results, we recognise the small sample size is a significant limitation. in particular, most of our results are based on the performance of a single pangolin. further testing with a larger sample size that focuses on different types of visual and acoustic stimuli in a foraging context would be illuminating, and would allow for extrapolation of results to a larger population within and potentially across pangolin species. without access to a larger sample size and the opportunity to test multiple subjects with a randomised order of conditions, we also cannot exclude the possibility that the data from our one subject were influenced by a possible order effect. though we did not find any evidence to suggest this, it is also possible that our subject had sensory impairments which could have impacted our results. however, due to the extreme difficulty of testing pangolins in controlled settings and the sunda pangolin's status as a critically endangered mammal on the verge of extinction, we present these results as a case study. we feel it is important to hypothesise about how a single pangolin's performance may help inform our understanding of pangolin behavioural ecology and cognition. the pangolin's performance on the visual condition, for example, makes sense considering the pangolin's morphology and known ecology. the sunda pangolin is a predator with relatively small eyes that forages on small insects at night , , , and thus vision may not be as relevant to its foraging strategy as olfaction. it is also possible that our use of dead ants as the rewarding stimulus was not ecologically salient enough for the pangolin to locate the food visually. in addition, the pangolin in this study did not demonstrate an ability to locate its food based on the acoustic cues provided by live ants. although we included leaves as a substrate in order to increase the sound produced by the ants inside the container, the plastic may have muted the acoustics enough that the pangolin could not differentiate between the baited and unbaited containers. however, from an ecological perspective, if pangolins do use their hearing to hunt insects in the wild, they would have to detect their acoustic signatures from varying distances and amid ambient noises within their natural habitats. they would also need to be able to hear their prey through tunnels in the ground, dense dirt mounds, or high within tree nests. thus, at close proximity, if the pangolin in our study used acoustic information to locate prey, he should have been able to detect it even behind the artificial plastic barrier. our own experience with the three pangolins suggests they do react to loud noises, and may adjust their tail posture based on the substrate on which they are walking. this behaviour may indicate that pangolins adjust their locomotion and their tail's position to minimize the noise they make in the wild. thus, although the pangolins may not use acoustic information when locating prey, it is likely they use it in predator detection or avoidance. plotnik et al. , for instance, found that asian elephants did not use auditory cues to find food in a similar object-choice task, even though elephants are well-known for their complex acoustic repertoires (e.g. [ ] [ ] [ ] . like for elephants, the "sound food makes" may not be important for the pangolins to find it. considering the coevolution that can take place between predatory and prey (e.g. [ ] [ ] [ ] , it might make more sense that the predatory habits of the pangolin would have evolved to exploit the more obvious chemosensory signatures generated by their prey. ants and termites rely heavily on the transmission of odors and pheromones to communicate amongst themselves [ ] [ ] [ ] [ ] . thus, a keen sense of smell would allow the pangolin to efficiently exploit the abundant availability of these olfactory signatures. from a foraging perspective, it seems likely that pangolins would prioritize olfactory information when searching for prey. while it is unknown if it is pheromones or other prey chemicals (such as odor from larvae) that influence the pangolin's navigation in a hunt, we tried to understand how olfaction was functionally employed by pangolins across different ecological scenarios. since the sunda pangolin has been noted for its semi-arboreal foraging behaviour , , 'olfactory distance' testing was conducted on a vertical plane to best replicate a foraging situation that a wild pangolin might encounter as it walks through a forest in search of arboreal prey. in the absence of nearby environmental cues, we found that our subject was either not sufficiently motivated by the distant prey odor offered, or was unable to locate it based on the olfactory information available. another possible explanation for our results in 'olfactory distance' testing was our inability to use a specific, longer inter-trial interval (iti) in this condition to ensure that the odor molecules had sufficient time to make their way from the container at the ( ) : | https://doi.org/ . /s - - -x www.nature.com/scientificreports www.nature.com/scientificreports/ top of the pvc down to the ground, thus creating an odor concentration gradient sufficient enough to be detected by the pangolin. due to the danger of the experimenter negatively impacting the pangolin's health by excessive handling or spending too much time inside his enclosure, we could not enact a longer iti. thus, the minimum time between trials plus the time the pangolin had to investigate was at least seconds. the food amount used in each trial, g of ants, was determined based on the average weight of an ant nest minus substrate, and thus this should also have enhanced the pangolin's capacity for finding them in a short period when the smell was confined within a tube and that smell was of a sufficient, ecologically relevant concentration. another potential confound in all olfactory conditions was the fact that we could not completely remove residual odor between trials, and thus the pangolin's choices may have been confused by olfactory information from prior trials. however, the pangolins' success on phase i olfaction trials and phase ii 'scent trail' trials suggests residual odor was likely not enough to overcome the pangolins' interest in odors emanating from a present food source, and thus the one pangolin's poor performance on 'olfactory distance' was likely not due to residual odor. in addition, the pseudo-randomisation of distance trials and the placement of different pvc tube lengths also meant that residual odor was regularly dispersed when tubes were removed and replaced, making it an unreliable source of information about food location. finally, if residual odor was a confound, we would expect the pangolin's performance on control trials to be influenced by the location of food in the test trials immediately preceding them. in a post-hoc review of these trials in 'olfactory distance' testing, we found that of the control trials that immediately followed a test trial, pluto the pangolin chose the same location in the control trial that had the baited container in the immediately preceding test trial only six of times (two-tailed binomial, p = . ). this suggests he was not using residual odor to find his food. in studies of elephants, another highly olfactory animal, they too did not seem to use residual odor to guide their choices about food , . despite the pangolin's poor performance on the 'olfactory distance' condition, it seems unlikely that a predator with a developed olfactory system , , , , , would be unable to detect its prey from the distances we investigated. scent trails would be most useful once they are found, but detection of distant odors may offer valuable information about directionality when there are no immediate environmental cues available. overall, the performance of our subject in the 'scent trail' condition suggests that the proximate environmental cues offered by scent trails may offer a more efficient method to localize live, mobile prey. our results were only significant in this condition when viewed in aggregate (i.e., across all 'scent trail' trials but not within food vs. mint or food vs. water trials). however, the similar percentage correct on each of these trial types ( . %) suggests that the pangolin likely could differentiate between the food and the non-food odor trails and followed the former. in fact, the anatomy of the pangolin may be conducive to following scent trails, as it has a low-hanging head and elongated nose that slopes towards the ground during terrestrial locomotion. the reliance on terrestrial scent trails, however, is somewhat confounding considering the semi-arboreal ecology of sunda pangolins , , , and we believe further testing is needed to understand whether the value of foraging cues changes as pangolins move from the forest floor into the trees. for instance, perhaps pangolins prioritize different sensory information as they pursue prey across different spatial planes, or they may follow olfactory information continuously as they track their food's movements. it is not always clear how the study of animal cognition fits into improving welfare and conservation in practice , . the goal of this research was to identify the pangolin's primary sensory modality in foraging behaviour, and then how that modality (olfaction) may be used to track prey. although this is only one study with a small sample size, research on the behaviour and cognition of animals like pangolins may have important applications for ex-situ captive management , , , . considering the high mortality rates of pangolins in captivity due to artificial diets and a lack of knowledge about their ecology , - , a greater understanding of the pangolin's natural behaviour (including foraging) is crucial for improving the welfare of these animals when reliant on human care. investigating pangolin behavioural ecology and cognition is also relevant to their conservation in the wild , , , . it is possible, for instance, that pangolins not only detect and value the scent of their prey, but also associated odors linked to it, such as the aroma of specific flora typically inhabited by their prey. in addition, although our study did not investigate ecologically relevant non-food odors in detail, it is possible that pangolins may prefer areas of their habitat which are devoid of predatory odors, while the presence of conspecific odors may influence their movement patterns based on whether the olfactory information comes from competitors or mates. our study, while limited, could complement future research on pangolin foraging behaviour to help create more efficient conservation protocols that focus on protecting pangolin food resources and habitat rather than on locating elusive, individual animals. a broader perspective on how pangolins use sensory information to forage and navigate through their environment could thus highlight key conservation points to protect not only pangolins, but the more discrete resources which they value as well. finally, the covid- pandemic highlights a crucial need to look beyond the charisma of endangered species when deciding which animals deserve the field of conservation's attention. it is vital that we work to better understand wildlife behaviour and ecology through research across scientific disciplines, even when focal species are difficult to study or lack popular or political attention. although it is still unknown whether the pangolin acted as a vector for the coronavirus between wildlife and humans , , what is clear is that the illegal wildlife trade and the increasing physical contact between wildlife and humans pose an existential threat not only to biodiversity in general, but also to our own existence as a species. all of the raw data that support the findings of this study are included in table . differences in the cognitive skills of bonobos and chimpanzees conformity to cultural norms of tool use in chimpanzees problem solving in great apes (pan paniscus, pan troglodytes, gorilla gorilla, and pongo abelii): the effect of visual feedback the mentality of crows: convergent evolution of intelligence in corvids and apes thinking with their trunks: elephants use smell but not sound to locate food and exclude nonrewarding alternatives elephants have a nose for quantity african elephants use plant odours to make foraging decisions across multiple spatial scales smelling more or less: investigating the olfactory experience of the domestic dog the world from a dog's point of view: a review and synthesis of dog cognition research a comparative analysis of animals' understanding of the human pointing gesture domestic dogs (canis familiaris) use human and conspecific social cues to locate hidden food cues that chimpanzees do and do not use to find hidden objects inferences about the location of food in the great apes (pan paniscus, pan troglodytes, gorilla gorilla, and pongo pygmaeus) domestic pigs' (sus scrofa domestica) use of direct and indirect visual and auditory cues in an object choice task asian pangolins: how behavioural research can contribute to their conservation in proceedings of the workshop on trade and conservation of pangolins native to south and southeast asia the global trafficking of pangolins: a comprehensive summary of seizures and trafficking routes from - in traffic report: southeast asia regional office overview of pangolin trade in southeast asia in proceedings of the workshop on trade and conservation of pangolins native to south and southeast asia molecular tracing of confiscated pangolin scales for conservation and illegal trade monitoring in southeast asia an overview of pangolin trade in the role of pangolins in ecosystems in pangolins: science, society and conservation scaling up pangolin conservation in iucn ssc pangolin specialist group conservation action plan evaluating methods for detecting and monitoring pangolin (pholidata: manidae) populations. global ecology and conservation , e sunda pangolin manis javanica (desmarest, ) in pangolins: science, society and conservation meat and medicine: historic contemporary use in asia in pangolins: science, society and conservation bushmeat and beyond: historic and contemporary use in africa in pangolins: science, society and conservation home range, activity cycle and natal den usage of a female sunda pangolin manis javanica (mammalia: pholidota) in singapore pangolins in peril: using local hunters' knowledge to conserve elusive species in vietnam a note on captive breeding and reproductive parameters of the chinese pangolin, manis pentadactyla linnaeus, a long way from home: the health status of asian pangolins confiscated from the illegal wildlife trade in viet nam in proceedings of the workshop on trade and conservation of pangolins native to south and southeast asia time-budgets and activity patterns of captive sunda pangolins (manis javanica) biology, husbandry, and veterinary care of captive chinese pangolins (manis pentadactyla) captive breeding of pangolins: current status, problems and future prospects behavioural descriptions of indian pangolins (manis crassicaudata) in captivity keeping and breeding the rescued sunda pangolins (manis javanica) in captivity feeding asian pangolins: an assessment of current diets fed in institutions worldwide captive husbandry of pangolins: lessons and challenges in pangolins: science, society and conservation formosan pangolin rescue, rehabilitation, and conservation in proceedings of the workshop on trade and conservation of pangolins native to south and southeast asia veterinary health of pangolins in pangolins: science, society and conservation the rescue, rehabilitation and release of pangolins in pangolins: science, society and conservation the tongue and associated structures in two species of african pangolins, manis gigantea and manis tricuspis evolution and morphology in pangolins: science, society and conservation the brain of the tree pangolin (manis tricuspis). i. general appearance of the central nervous system the brain of the tree pangolin (manis tricuspis). ii. the olfactory system the brain of the tree pangolin (manis tricuspis). vi. the brainstem and cerebellum management guidelines for sunda pangolin (manis javanica), first ed. carnivore and pangolin conservation program a survival blueprint for the chinese pangolin, manis pentadactyla, national trust for nature conservation lalitpur morphometrics and retinal organization in the arboreal african pangolin supports nocturnality ecological research and conservation of sunda pangolin manis javanica in singapore in proceedings of the workshop on trade and conservation of pangolins native to south and southeast asia behavioral contexts of elephant acoustic communication in the amboseli elephants: a long-term perspective on a long-lived mammal acoustic communication in the asian elephant, elephas maximus maximus long-distance communication of acoustic cues to social identity in african elephants coevolution can reverse predator-prey cycles sedentary snakes and gullible geckos: predator-prey coevolution in nocturnal rock-dwelling reptiles the evolutionary response of predators to dangerous prey: hotspots and coldspots in the geographic mosaic of coevolution between garter snakes and newts the chemical basis of insect sociality in chemicals controlling insect behaviour arthropod defensive secretions in pheromone communication in social insects: sources and secretions in pheromone communication in social insects: ants wasps, bees, and termites organization of the olfactory pathway and odor processing in the antennal lobe of the ant camponotus floridanus taking the elephants' perspective: remembering elephant behavior, cognition and ecology in humanelephant conflict mitigation comparative cognition for conservationists current status and future directions of applied animal behavioral research for animal welfare and conservation identifying sars-cov- related coronaviruses in malayan pangolins probable pangolin origin of sars-cov- associated with the covid- outbreak we thank the faculty and staff of the livestock and wildlife hospital of mahidol university in kanchanaburi, thailand for their support of this research and their care of the pangolins. we also thank parntep ratanakorn for his continued support of collaborative research on endangered species behaviour in thailand. we greatly appreciate andrew michalski's help with apparatus construction and design, and thank martin chodorow for his analysis advice. we also thank melissa schmitt, peter moller and reviewers for comments on an earlier version of this manuscript, and matthew rudolph for assistance with the figure. this paper is dedicated to the memory of pluto, our primary research subject, who passed away from natural causes in january, . his voluntary participation in this research was crucial to this study's success, and his curiosity and good nature made it a pleasure to work with him. this work formed the basis of the first author's master's degree thesis in the animal the authors declare no competing interests. supplementary information is available for this paper at https://doi.org/ . /s - - -x.correspondence and requests for materials should be addressed to j.m.p.reprints and permissions information is available at www.nature.com/reprints.publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.open access this article is licensed under a creative commons attribution . international license, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the creative commons license, and indicate if changes were made. the images or other third party material in this article are included in the article's creative commons license, unless indicated otherwise in a credit line to the material. if material is not included in the article's creative commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. to view a copy of this license, visit http://creativecommons.org/licenses/by/ . /. key: cord- - fuy tlp authors: patson, noel; mukaka, mavuto; otwombe, kennedy n.; kazembe, lawrence; mathanga, don p.; mwapasa, victor; kabaghe, alinune n.; eijkemans, marinus j. c.; laufer, miriam k.; chirwa, tobias title: systematic review of statistical methods for safety data in malaria chemoprevention in pregnancy trials date: - - journal: malar j doi: . /s - - -z sha: doc_id: cord_uid: fuy tlp background: drug safety assessments in clinical trials present unique analytical challenges. some of these include adjusting for individual follow-up time, repeated measurements of multiple outcomes and missing data among others. furthermore, pre-specifying appropriate analysis becomes difficult as some safety endpoints are unexpected. although existing guidelines such as consort encourage thorough reporting of adverse events (aes) in clinical trials, they provide limited details for safety data analysis. the limited guidelines may influence suboptimal analysis by failing to account for some analysis challenges above. a typical example where such challenges exist are trials of anti-malarial drugs for malaria prevention during pregnancy. lack of proper standardized evaluation of the safety of antimalarial drugs has limited the ability to draw conclusions about safety. therefore, a systematic review was conducted to establish the current practice in statistical analysis for preventive antimalarial drug safety in pregnancy. methods: the search included five databases (pubmed, embase, scopus, malaria in pregnancy library and cochrane central register of controlled trials) to identify original english articles reporting phase iii randomized controlled trials (rcts) on anti-malarial drugs for malaria prevention in pregnancy published from january to july . results: eighteen trials were included in this review that collected multiple longitudinal safety outcomes including aes. statistical analysis and reporting of the safety outcomes in all the trials used descriptive statistics; proportions/counts (n = , %) and mean/median (n = , . %). results presentation included tabular (n = , . %) and text description (n = , . %). univariate inferential methods were reported in most trials (n = , . %); including chi square/fisher’s exact test (n = , . %), t test (n = , . %) and mann–whitney/wilcoxon test (n = , . %). multivariable methods, including poisson and negative binomial were reported in few trials (n = , . %). assessment of a potential link between missing efficacy data and safety outcomes was not reported in any of the trials that reported efficacy missing data (n = , . %). conclusion: the review demonstrated that statistical analysis of safety data in anti-malarial drugs for malarial chemoprevention in pregnancy rcts is inadequate. the analyses insufficiently account for multiple safety outcomes potential dependence, follow-up time and informative missing data which can compromise anti-malarial drug safety evidence development, based on the available data. guidance on clinical trial reporting of safety outcomes through adherence to the consolidated standards of reporting trials (consort) guidelines [ , ] . however, there is scant literature on standardized ways to statistically analyse the safety outcomes in clinical trials. although there exist some general regulatory guidelines on safety data analysis, such as international conference on harmonization which recommend descriptive statistical methods supplemented by confidence intervals [ , ] , the proposed statistical methods rarely account for the complexity of the collected safety data, e.g., recurrent adverse events (aes). effective solutions to statistical analysis of safety data in clinical trials may need to be tailored to specific indications (set of diseases with similar characteristics) since safety data collected are also influenced by the medical condition under study. absence of standardized guidelines for safety data analysis in specific settings may limit the ability to draw rich conclusions about the safety of the investigational product, based on collected data. standardized guidelines can simplify integration of safety information from multiple outcomes across rcts [ ] and would ensure optimal use of data in developing the safety profile of the investigational product. statistical analysis of safety data in clinical trials is characterized by a challenge of multiple and related endpoints measured over time. the safety endpoints may include clinical and laboratory defined aes. laboratorybased aes are defined based on standard cut-off points for measures such as vital signs (e.g., body temperature), hepato-toxicity measures (e.g., bilirubin level), cardiotoxicity measures (e.g., electrocardiograms), and other tests relevant to the medical indication being studied [ ] . the safety endpoints may be correlated within patients and over time such that failure to account for this in an analysis may yield biased estimates and false inference. furthermore, time to occurrence of the safety endpoint may be very informative in profiling the drug safety. such data present statistical analysis and interpretation challenges due to the complexity in structure [ ] . for instance, in the case of multiple, repeatedly measured, safety outcomes, false positives may arise from multiple statistical testing if appropriate longitudinal or time to event methods and/or multiplicity adjustments are not considered. in clinical trials, aes may impact compliance and study participation which may further affect treatment efficacy estimates [ , ] . occurrence of (even mild) aes due to a drug would lead to non-adherence, leading to informative censoring. the dropping of the patients from the study generates missing data that may lead to biased results if poorly accounted for. therefore, safety data analysis accounting for missing data is useful to facilitate identification and characterization of the safety profile of the drug as early as possible. other analysis challenges include lack of adequate ascertainment and classification of aes, and limited generalizability of results [ ] since some aes cannot be pre-specified at study design stage. there are many populations where drug safety assessment is complex. one of the special settings in safety data assessment is the use of drugs to prevent adverse outcomes in pregnancy, currently referred to as intermittent preventive treatment of malaria in pregnancy (iptp). for example, the world health organization recommends that pregnant women receive routine treatment with anti-malarial drugs to clear any malaria infection that is present and also to prevent infection in the weeks after administration [ ] . recent review indicates that methodological issues in studying antimalarial drugs in pregnancy have prevented firm conclusions on the safety of new anti-malarial drugs in pregnancy [ ] . previous efforts have attempted to standardize safety assessment methodology for antimalarial drug trials in pregnancy, including study designs and data collection [ , ] . however, literature remains limited in describing the standard practice in the statistical analysis of safety data that are collected on anti-malarial drugs during pregnancy trials. the current review focusses on safety assessment in anti-malarial drugs for chemoprevention in pregnancy trials. since anti-malarial drug for malaria chemoprevention is given repeatedly to healthy pregnant women, it is critical to improve safety assessment in this vulnerable population. specifically, appropriate statistical analysis of safety outcomes can improve development of anti-malarial drug safety profile. this can be achieved through sufficient use of the data generated during the rct which provides a comprehensive drug safety insight. this review, therefore, aims at identifying applied statistical methods and their appropriateness in the analysis of safety data in anti-malarial drugs for malaria prevention during pregnancy clinical trials. the systematic review was conducted according to preferred reporting items for systematic review and meta-analyses (prisma) statement [ ] which outlines minimum standards for reporting systematic reviews and meta-analysis (additional file : table s ). the protocol for this review was registered and published with prospero (crd ). the study population is pregnant women on any anti-malarial drug for malaria chemoprevention. primary original articles published in english from phase iii rcts were considered for inclusion. the articles were from rcts assessing the efficacy and safety of malaria chemoprevention in pregnancy. this review focused on phase iii rcts, because they have the largest sample size among pre-marketing trials and accommodates multidisciplinary support in safety evaluation. further, the data are systematically collected and have the benefit of being randomized, which aids a fair comparison of treatment groups. observational studies, case reports, letters to the editor, narrative reviews, systematic reviews and trials in phase i or phase ii or phase iv were excluded from this review. this review did not include clinical trials on malaria prevention in pregnancy using intermittent screening and treatment (istp) as an alternative to iptp. istp refers to intermittent rapid diagnostic testing (rdt) for malaria in pregnant women followed by treatment of rdt-positive cases with an effective artemisinin-based combination therapy, and iptp is given to pregnant women regardless of their malaria status. hence, istp and iptp consider different populations which may confound the practice in safety assessment methods (i.e., istp considers symptomatic population and iptp considers both symptomatic and asymptomatic population). non-english publications were excluded. studies published between january and july, were searched from five databases (pubmed, embase, scopus, malaria in pregnancy libray (mipl) and cochrane central register of controlled trials (cen-tral). the mipl is an excellent scholarly source of articles on malaria in pregnancy that enabled the review to capture both indexed and non-indexed articles beyond the searched databases. additional searches included reference lists of the identified trials and relevant reviews to identify trials potentially missed in the database search. the year was selected since it is when consort guideline updated and emphasized on appropriate statistical analysis and reporting of clinical trials [ ] . conference proceedings were not included because they usually contain abstracts that do not give detailed analysis of the presented results and they are not rigorously peer-reviewed. the review focussed on published studies only so no experts or abstract publication authors were contacted for unpublished data. the key search items included: malaria, anti-malarial drug, pregnancy, efficacy safety or tolerability. the detailed search strategy is presented in additional file : table s . the search was customized per database. based on prisma procedure, after removing duplicates, two reviewers (np and ank) independently screened titles and abstracts initially before arriving at a final list of eligible articles. based on the eligible studies list, full text articles were retrieved. the references were managed using endnote x . (thomson reuters). if there were disagreements, the two reviewers discussed the paper to reach a consensus and reasons for exclusion were provided for ineligible publications/studies the data extraction file created in microsoft excel was used to record all key variables from the selected articles. some of the collected variables such as mode of safety data collection, participant withdrawal due to ae and handling of continuous measures were based on consort guideline. the following key variables were extracted from the papers: main author, publication date, study design, study location, main efficacy outcome, sample size, list of safety parameters collected (including laboratory data), nature of safety data collection (i.e., passive or active), list of statistical methods used for respective safety outcomes, how the results were presented, retention rate at the end of the follow up and how missing safety or efficacy data were handled. the primary hypothesis type (as superiority, non-inferiority or equivalence) was defined based on what was reported in the actual manuscript or inferred by the lead author (np), based on how the study framed the primary hypothesis. superiority hypotheses aim to show whether treatment is better than control, non-inferiority hypotheses intend to show that one treatment is not worse than the other and equivalence hypotheses intend to show that a given treatment is similar to another for defined characteristics [ ] . the statistical methods were classified as descriptive or inferential and univariate or multivariate depending on the purpose and nature of the statistical methods based on previous similar reviews [ , ] , reviewing statistical methods. the extracted quantitative data were reported as percentages in tables. the commonly reported safety parameters, suitability of the used statistical methods and other findings were also summarized narratively. the search identified articles. after removing duplicates, unique articles were identified and considered for possible inclusion in the review. the duplicates (i.e. repeated citations) were the same articles but identified in multiple search databases. figure presents details of the selection process. during screening, a total of articles were excluded based on relevance of their titles and abstracts. the remaining full text articles were assessed for possible inclusion and articles satisfied the inclusion criteria, and were included in this review as shown in table . reasons for exclusion are shown in fig. . the trials included in this review were conducted in oceania ( trials, . %) and sub-saharan africa ( trials, . %) regions. the rcts reviewed recruited , pregnant women with a median sample size of (interquartile range (iqr): , ) women per treatment arm in a trial. thirteen trials ( . %) recruited more than patients per arm. as expected, all trials ( trials, %) computed sample size based on the efficacy outcome(s). the majority of the trials ( trials, . %) had two treatment arms and the rest had three treatment arms. all trials had an active comparator and iptp-sp was studied as a standard malaria chemoprevention in the majority of trials ( trials, . %). although the review focussed on published trials from to , the trials were conducted between and . based on the primary hypothesis tested, superiority design rcts were the most common ( trials, . %) and the other trials had a noninferiority hypothesis. over half of the trials ( over half of the trials ( trials, . %) reported that they collected safety data using a combination of scheduled and non-scheduled visits (table ) , while a third of the trials ( trials, . %) did not specify the safety data collection approach used. the median retention rate (based on the defined efficacy outcome reported for respective trials) was . % (iqr: . %, . %) and trials ( . %) had a retention rate below %. all the reviewed trials indicated that they had collected multiple longitudinal safety endpoints. as expected, almost all the trials ( trials, . %) reported obstetric safety outcomes such as foetal loss. table s and s in additional file provide a detailed list of safety outcomes and respective statistical method reported in each reviewed trial. despite the reported occurrence of multiple aes, none of trials seemingly reported recurrence of aes during pregnancy. in total, trials ( . %) reported adverse events with different severity levels, e.g., mild, moderate and severe. all trials reported occurrence of aes by treatment arm. almost all trials ( trials, . % %) reported laboratory data in their safety assessment of the drug and trials of these ( . ) dichotomized at least a single continuous safety outcome (e.g., haemoglobin) based on standard cut-off points, to define an ae. the safety analysis approach (based on treatment allocation and adherence) was specified and reported in trials ( . %). per protocol (pp) and intention to treat (itt) analysis approaches were used in trials ( . %) and trials ( . %), respectively. two trials indicated that they used both pp and itt to analyse the safety data. although all the reviewed trials had at least one patient lost to follow-up, only trials ( . %) reported missing efficacy data and of the trials indicated that the missingness was ignorable after exploring data missingness patterns ( table ) . none of the reviewed trials conducted an advanced sensitivity analysis on the relationship between missing data and drug safety. for example, none of the studies assessed the safety outcomes (e.g., aes) in relation to missing efficacy outcomes. this review found that most trials ( trials, . %) had at least one participant who experienced an ae leading to discontinuation from the trial although the studies did not formally investigate/quantify the relationship between the aes and trial completion. all the trials included reviewed used descriptive statistics as one of the methods to summarize aes (table ) . proportions or counts were the descriptive statistics used in all of the studies to report safety data. definition safety data was dependent on respective trials as shown in the additional file : table s and s ). incidence rates were reported in trials ( . %). most trials ( trials, . %) reported univariate inferential statistical methods; these included chi square or fisher`s exact test ( trials, . %), t-test (n = , . %). only trials reported multivariate statistical methods. the multivariable methods were poisson regression (n = , . %), and negative binomial regression (n = , . %). usage of at least two inferential statistical methods to compare safety outcomes was reported in trials ( . %). although all studies reported multiple safety outcomes, none reported adjustment for multiplicity during analysis. the review showed that at least a single optimal statistical methods was reported in trials ( . %) that considered multivariable modelling. even though univariable analysis comparing arms in an rct were appropriately used, further inferential statistical methods reported in the rest of the trials were suboptimal for the type of data being collected. for further details, additional file : tables s provide a detailed list of reported statistical methods with their respective safety outcome(s). in terms of presentation of results, none of the trials presented aes in a graph. only trials ( . %) narratively presented the safety results; the other trials ( . %) presented the results in tabular format. a total of trials reported p-values after comparing treatments and there were only trials ( . %) that reported point estimates with their respective confidence intervals. this review sought to provide a detailed overview of the actual practice of the statistical analysis of safety data in the unique setting of drug trials for the preventions of malaria in pregnancy as reflected published literature. the results demonstrate that there is limited reporting of statistical analyses of safety data, at the end of the trial, in these published reports. the findings are useful to advance the development of standardized guidelines for safety data statistical analysis in analysis in anti-malarial drugs in pregnancy trials and related fields. such guidelines will not replace but rather complement the con-sort guidelines that are general (i.e., not providing specific statistical methods in analysing harms in rcts). based on the authors' knowledge of the available literature, this is the first paper to review statistical methods for safety data in anti-malarial drugs in pregnancy. descriptive methods were commonly used to summarize safety data. this review found that each clinical trial used at least one descriptive method to summarize safety data. univariate statistical methods such as chi square or fishers exact tests were used in two-thirds of the articles reviewed. such descriptive statistics and univariate statistical inference ignored useful information such as variability in follow-up time, missing data and correlation (for those trials which had their multiple safety outcomes repeatedly measured). hence there was inefficient data use during analysis that may lead to a loss of useful information for improved and informative conclusions. although a third of the reviewed trials attempted to use crude incidence, the analyses failed to adequately account for individual patient follow-up-time and potential confounders. all trials dichotomized at least a single continuous clinical laboratory safety outcome (i.e., where ae was defined based on standard cut-off points for adult toxicity). although this aids in providing time-specific drug safety status and easy interpretation, the dichotomized outcome may miss some information on the magnitude of the temporal changes, overtime during the trial. the information loss may lead to reduction in statistical power to detect safety signal if it exists. valid longitudinal methods (used without restriction on cut-off points) can address the information loss by exploiting potential within-subject correlations for the repeated clinical laboratory measurements [ ] [ ] [ ] . furthermore, the longitudinal methods can provide the basis for developing improved cut-off points tailored to pregnant women in malaria-specific settings. to ensure improved uptake of such methods, future work needs to strive towards making the results from the longitudinal methods feasibly interpretable to the medical practitioners. only three studies appropriately used multivariable statistical methods. adjusting for known prognostic covariates is useful to control for confounding that can be introduced due to imbalance when assessing if treatment is independently associated with safety outcome(s). of secondary interest, covariate adjustment also preserves type i error [ ] . such adjustment for potential confounders (e.g., age) in safety data analysis are suitable in clinical trials with at least moderate sample size, unlike small sample sizes that lead to unstable estimates. of specific interest in this review, the poisson model was more suitable in the context of rare aes which usually have low event rates [ , ] . since poisson regression assumes a constant rate of occurrence of a rare event, it is not ideal for other multiple transient aes that were common or recurred and would vary in occurrence overtime [ ] . alternatively, mixed effects models could be considered to characterize the safety events over time since they capture patient-specific effects [ , ] . whenever time to ae occurrence information is available, survival analysis models may also be preferred to characterize the time to ae occurrence. for recurrent safety events, that may induce dependence, methods that extend the cox regression model may be preferred; such models include survival mixed effects models (e.g., frailty models) [ ] . almost half of the reviewed trials did not explicitly define the population on which the safety analysis was based. if per protocol analysis is used to address non-adherence there is potential selection bias since it destroys the balance due to randomization. although consort recommends itt, as an alternative for analysis of safety endpoints, non-adherence cannot be explicitly addressed with itt approach since it ignores dropouts, withdrawal or loss-to-follow up for various reasons including safety concerns; itt-based inference ignores causal effect of the actual treatment received [ ] . patient withdrawal or dropout due to aes can induce informative censoring useful in quantifying anti-malarial drug safety. for example, if a patient withdraws due to vomiting after taking an anti-malarial drug, their obstetric efficacy outcomes such as birth weight may appear as missing data. in the context of anti-malarial drug for malaria prevention, even mild ae can lead to drug nonadherence. since the patient has no disease symptoms, they would judge it less costly for them to discontinue the drug than continue experiencing aes. hence, inclusion of information on treatment/trial completion status in relation to anti-malarial safety would enrich development of the safety profile of anti-malarial drug in pregnancy. although study completion status, anti-malarial drug safety and missing data may be interlinked, missing data received limited attention such that the few trials that considered efficacy missing data did not explicitly explore the potential link. studying such complex associations requires statistical methods that can appropriately estimate the pathway from the anti-malarial chemoprevention to study completion. advantageously, methods based on causal inference framework, such as mediation analysis [ ] [ ] [ ] [ ] could be adapted/extended to assess the influence of the aes on non-adherence in rcts. despite about three-quarters of the trials reporting p-values after comparing safety outcomes by treatment arms, only about half of the reviewed trials adhered to international harmonisation conference guideline e in reporting of confidence intervals in quantifying the safety effect size [ , ] . use of confidence interval aids in interpretation of results by providing a measure of precision. furthermore, graphical displaying of safety data to aid in summarizing of safety data was inadequate. graphs on safety data have a greater ability to convey insight about patterns, trends, or anomalies that may signal potential safety issues compared to presentation of such data in tabular form only [ ] . for example, the graphs could help to visualize frequency and changes in aes over time by treatment arm. the graphs could further help in assessing assumptions for some statistical methods. over three-quarters of the reviewed trials were designed as superiority trials based on efficacy outcomes. although the statistical approach for safety assessment was mainly on superiority hypotheses (for both the superiority and non-superiority trials), clinical and statistical justification of assessing safety based on superiority hypotheses may be invalid. superiority hypotheses concentrate on the absence of difference in drug safety effect/risk between or across the treatment arms which may be challenging [ ] . for example, when comparing high ae incidences, non-significant difference (when using a superiority hypothesis) would not necessarily translate to a conclusion that a drug is safe and welltolerated since sometimes all compared treatment arms may have high ae incidence. perhaps, drug safety evaluation should strive to prove that there is no risk beyond a protocol-defined/hypothesized priori clinical safety margin (i.e., no excessive safety risk). based on the findings in this work, researchers are encouraged to consider defining safety margins in safety assessment of anti-malarial drugs. since safety is mostly a secondary outcome, it is not straightforward on how to define a non-inferiority margin and the appropriate analysis population. currently, it is still unclear and debatable how to implement this, such that further research is needed [ ] . interestingly, over half of the trials were openlabel which may influence physician clinical safety assessment on a patient and patient reporting of aes based on their expectations since they know the treatment assigned. appropriate reporting of the aes would be guided by data safety and monitoring boards (dsmb) from early stages of the trial. however, availability of dsmbs in over three-quarters of the trials did not translate to improved reporting and analysis. therefore, dsmb members should advocate for improved analysis approaches for aes. tabel summarizes recommendations to consider on best practices for safety analyses. this provides a general framework for statistical analysis of safety in malaria chemoprevention in pregnancy trials. as highlighted above, the recommendations assume a context where sample size is moderate or large. for rare events, bayesian approaches can be considered since they do not depend on asymptotic properties when handling rare events and can incorporate prior/external information [ ] . future research work can further consider adapting/extending recently developed statistical methods for rare disease or small population clinical trials towards analysis of rare safety outcomes in iptp trials [ ] [ ] [ ] . this review agrees with other similar publications focusing on drug safety assessment in clinical trials that have noted the need for further improvement in the statistical analysis of the safety data [ , ] . this review concurs with a recent review that has noted that inappropriate handling of multiple test is prevalent, although their review focussed on four high impact journals, ae in general and a short time of review period [ ] . issues raised in this review include time-dependence of aes, informative censoring due to discontinuation of treatment because of aes, safety graphs, and repeated occurrence of aes and multivariate longitudinal structure of laboratory data that yields complex correlation. this is an ongoing work whereby further analysis will be explored to address the identified statistical issues above. the application of the systematic review protocol in describing the current practice is highly reliable and objective since it exhaustively identified the published anti-malarial drug clinical trials in pregnancy for studied period. however, this review covered only the last decade of publications and may have missed studies published in other languages or that did not appear in during the literature search. because the trials reported in the publications spanned for a decade, it was difficult to assess temporal trends. this review represents the most comprehensive review of safety data analysis practice for this important indication. although useful safety data are collected in malaria chemoprevention in pregnancy clinical trials, the analysis remains sub-optimal and this hinders definitive conclusions about drug safety in this setting. descriptive statistical methods and dichotomization of continuous outcomes are predominant which may lead to loss of useful information. the definition of analysis population and informative presentation of results are not standardized. overall, the results suggest that choice of a statistical method(s) to use should be jointly driven by the scientific question of interest, epidemiological/clinical plausibility of the method and structure of the raw data. further work in addressing the highlighted gaps can enhance drug safety decisions and conclusions. supplementary information accompanies this paper at https ://doi. org/ . /s - - -z. a systematic review of safety data reporting in clinical trials of vaccines against malaria, tuberculosis, and human immunodeficiency virus explanation and elaboration: updated guidelines for reporting parallel group randomised trials statistical principles for clinical trials (ich e ): an introductory note on an international guideline ich harmonised tripartite guideline. statistical principles for clinical trials. international conference on harmonisation e expert working group sources of safety data and statistical strategies for design and analysis: clinical trials a question-based approach to the analysis of safety data adherence to therapy and adverse drug reactions: is there a link? reporting of lost to follow-up and treatment discontinuation in pharmacotherapy and device trials in chronic heart failure drug safety assessment in clinical trials: methodological challenges and opportunities updated who policy recommendation: intermittent preventive treatment of malaria in pregnancy using sulfadoxine-pyrimethamine (iptp-sp). geneva, world health organization treatment of uncomplicated and severe malaria during pregnancy methodology of assessment and reporting of safety in anti-malarial treatment efficacy studies of uncomplicated falciparum malaria in pregnancy: a systematic literature review evaluating harm associated with anti-malarial drugs: a survey of methods used by clinical researchers to elicit, assess and record participant-reported adverse events and related data the prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration statement: updated guidelines for reporting parallel group randomised trials superiority, equivalence, and non-inferiority trials the statistical content of published medical research: some implications for biomedical education use of statistical analysis in the the analysis of multivariate longitudinal data: a review modeling laboratory data from clinical trials statistical methods for evaluating safety in medical product development the risks and rewards of covariate adjustment in randomized trials: an assessment of outcomes from studies a model for the distribution of daily number of births in obstetric clinics based on a descriptive retrospective study some simple robust methods for the analysis of recurrent events an approach to integrated safety analyses from clinical studies analysis of adverse events in drug safety: a multivariate approach using stratified quasi-least squares a framework for the design, conduct and interpretation of randomised controlled trials in the presence of treatment changes correcting for non-compliance in randomized trials using structural nested mean models addressing complications of intention-to-treat analysis in the combined presence of all-or-none treatment-noncompliance and subsequent missing outcomes intention-to-treat meets missing data: implications of alternate strategies for analyzing clinical trials data estimating treatment effects in randomised controlled trials with non-compliance: a simulation study graphical approaches to the analysis of safety data from clinical trials bayesian methods for design and analysis of safety trials recent advances in methodology for clinical trials in small populations: the inspire project lessons learned from ideal- recommendations from the ideal-net about design and analysis of small population clinical trials applicability and added value of novel methods to improve drug development in rare diseases recommendations to improve adverse event reporting in clinical trial publications: a joint pharmaceutical industry/journal editor perspective analysis and reporting of adverse events in randomised controlled trials: a review effect of repeated treatment of pregnant women with sulfadoxine-pyrimethamine and azithromycin on preterm delivery in malawi: a randomized controlled trial intermittent preventive treatment of malaria with sulphadoxinepyrimethamine during pregnancy in burkina faso: effect of adding a third dose to the standard two-dose regimen on low birth weight, anaemia and pregnancy outcomes superiority of over doses of intermittent preventive treatment with sulfadoxine-pyrimethamine for the prevention of malaria during pregnancy in mali: a randomized controlled trial efficacy of malaria prevention during pregnancy in an area of low and unstable transmission: an individually-randomised placebo-controlled trial using intermittent preventive treatment and insecticide-treated nets • fast, convenient online submission • thorough peer review by experienced researchers in your field • rapid publication on acceptance • support for research data, including large and complex data types • gold open access which fosters wider collaboration and increased citations maximum visibility for choose bmc and benefit from: in the kabale highlands, southwestern uganda intermittent preventive treatment with sulfadoxine-pyrimethamine versus weekly chloroquine prophylaxis for malaria in pregnancy in honiara, solomon islands: a randomised trial cotrimoxazole prophylaxis versus mefloquine intermittent preventive treatment to prevent malaria in hiv-infected pregnant women: two randomized controlled trials intermittent preventive treatment of malaria in pregnancy with mefloquine in hiv-negative women: a multicentre randomized controlled trial intermittent preventive treatment of malaria in pregnancy with mefloquine in hiv-infected women receiving cotrimoxazole prophylaxis: a multicenter randomized placebo-controlled trial effectiveness of co-trimoxazole to prevent plasmodium falciparum malaria in hiv-positive pregnant women in sub-saharan africa: an open-label, randomized controlled trial safety of daily co-trimoxazole in pregnancy in an area of changing malaria epidemiology: a phase b randomized controlled clinical trial intermittent screening and treatment or intermittent preventive treatment with dihydroartemisinin-piperaquine versus intermittent preventive treatment with sulfadoxine-pyrimethamine for the control of malaria during pregnancy in western kenya: an open-label, three-group, randomised controlled superiority trial sulphadoxine-pyrimethamine plus azithromycin for the prevention of low birthweight in papua new guinea: a randomised controlled trial dihydroartemisinin-piperaquine for the prevention of malaria in pregnancy efficacy and safety of azithromycin-chloroquine versus sulfadoxine-pyrimethamine for intermittent preventive treatment of plasmodium falciparum malaria infection in pregnant women in africa: an open-label, randomized trial intermittent preventive treatment with dihydroartemisinin-piperaquine for the prevention of malaria among hiv-infected pregnant women chloroquine as weekly chemoprophylaxis or intermittent treatment to prevent malaria in pregnancy in malawi: a randomised controlled trial comparative study of mefloquine and sulphadoxine-pyrimethamine for malaria prevention among pregnant women with hiv in southwest nigeria monthly sulfadoxine-pyrimethamine versus dihydroartemisinin-piperaquine for intermittent preventive treatment of malaria in pregnancy: a double-blind, randomised, controlled, superiority trial publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations utrecht university, utrecht, the netherlands. key: cord- - uucx ss authors: randremanana, rindra vatosoa; raberahona, mihaja; randria, mamy jean de dieu; rajerison, minoarisoa; andrianaivoarimanana, voahangy; legrand, agathe; rasoanaivo, tsinjo fehizoro; randriamparany, ravaka; mayouya-gamana, théodora; mangahasimbola, reziky; bourner, josie; salam, alex; gillesen, annelies; edwards, tansy; schoenhals, matthieu; baril, laurence; horby, peter; olliaro, piero title: an open-label, randomized, non-inferiority trial of the efficacy and safety of ciprofloxacin versus streptomycin + ciprofloxacin in the treatment of bubonic plague (imasoy): study protocol for a randomized control trial date: - - journal: trials doi: . /s - - - sha: doc_id: cord_uid: uucx ss background: bubonic plague is the primary manifestation of infection with yersinia pestis, accounting for % of all plague cases and with % of global cases reported in madagascar. all drugs in use for treating plague are registered based on experimental data and anecdotal evidence, and no regimen currently recommended is supported by a randomized clinical trial. the imasoy trial intends to fill this knowledge gap by comparing two -day regimens included in the national guidelines in madagascar. the primary objective of the trial is to test the hypothesis that ciprofloxacin monotherapy is non-inferior to streptomycin followed by ciprofloxacin for the treatment of bubonic plague, thus avoiding the need for injectable, potentially toxic, aminoglycosides. methods: a two-arm parallel-group randomized control trial will be conducted across peripheral health centres in madagascar in five districts. males and non-pregnant females of all ages with suspected bubonic or pneumonic plague will be recruited over the course of three plague ‘seasons’. the primary endpoint of the trial is to assess the proportion of patients with bubonic plague who have a therapeutic response to treatment (defined as alive, resolution of fever, % reduction in the size of measurable buboes, has not received an alternative treatment and no clinical decision to continue antibiotics) as assessed on day . discussion: if successful, the trial has the potential to inform the standard of care guidelines not just in madagascar but in other countries afflicted by plague. the trial is currently ongoing and expected to complete recruitment in . trial registration: clinicaltrials.gov nct . registered on october plague is caused by the gram-negative bacteria yersinia pestis. there are three main clinical forms in which plague can manifest, sometimes with overlapping clinical syndromes: bubonic plague, septicaemic plague (primary or secondary) and pneumonic plague (primary or secondary) [ ] . bubonic plague is the most common clinical form and is characterized by the rapid onset of fever and tender and painful lymphadenopathy. it accounts for over % of plague cases [ ] . septicaemic plague is characterized by non-specific symptoms, as for septicaemia in general [ ] . pneumonic plague presents with rapid onset of fever and respiratory symptoms, with haemoptysis being a cardinal feature [ ] . plague is primarily present in africa, although the usa and countries in asia and south america are also affected. the incidence of plague continues to decrease (from cases notified to the who in to in ) and geographical spread to shrink ( % of reported cases currently from just madagascar and the democratic republic of congo (drc)) [ ] . however, the reservoir and conditions for transmission are still present in many areas of the world, and there is a risk of outbreaks when plague hits densely populated areas, as in the large urban outbreak of predominantly pneumonic plague in madagascar, with over suspected cases [ ] . in the pre-antibiotic era, the case fatality rate (cfr) of plague was > % [ ] . antibiotics have significantly reduced mortality: for the cases notified to who during - cfr averaged % [ ] . during the - plague outbreak in madagascar, the cfr of confirmed pneumonic plague was % and bubonic plague % [ ] . until july , the st-line treatment regimen for bubonic plague in madagascar was streptomycin (days to ) followed by cotrimoxazole (days to ). following the epidemic, an after action review was conducted and treatment guidelines for bubonic plague were modified to include, for adults, either streptomycin intramuscularly g (except pregnant women) or gentamicin . mg/kg twice daily for days, followed by ciprofloxacin mg twice daily for an additional days or ciprofloxacin alone at mg twice daily for days. intravenous ciprofloxacin at a dose of mg every h can be used if the oral route is not possible. treatment options in children are now either oral ciprofloxacin mg/kg twice daily for days or, if oral administration is not initially possible, streptomycin mg/kg every h intramuscularly for days followed by ciprofloxacin mg/kg twice daily for an additional days. the first-line treatment regimen for pneumonic plague in adults is streptomycin g intramuscularly twice daily for days combined with ciprofloxacin mg orally twice daily for days, and for children, intravenous gentamicin at . mg/kg every h for days (or streptomycin) combined with oral ciprofloxacin at mg/kg every h for days (with the possibility of intravenous ciprofloxacin at mg/kg every h if the oral route is not available). the maximum dose for ciprofloxacin is mg per oral dose and mg per intravenous dose for children. streptomycin alone or in other combination therapies is also the treatment of choice in other countries, including the drc and peru. the evidence base for the current treatment regimens for plague is weak, and none is supported by a randomized clinical trial (rct). two randomized control trials have previously attempted to improve on or contribute to a better evidence base for current plague treatment regimens [ , ] . however, neither trial was successful nor included children under years of age, who constitute a significant proportion of the population affected by plague. treatment recommendations are based on animal models, case reports, case series and a trial of doxycycline versus gentamicin [ ] . all drugs approved for treating plague by the the united states food and drug administration (us fda) (streptomycin, tetracyclines and the fluoroquinolones ciprofloxacin, levofloxacin and moxifloxacin) are registered based on the 'animal efficacy rule', in the absence of clinical data [ ] . political instability and practical difficulties related to highly dispersed cases in rural areas have been the major obstacles to conducting treatment trials of plague. the main issue with current regimens in madagascar is the use of aminoglycosides-with the need for injections, class toxicity and need for monitoring of renal and auditory functions, which is not routinely available in most low-and middle-income countries, and streptomycin cost and stock-outs. ciprofloxacin is cheaper, more readily available in lowand middle-income countries, has greater ease of administration and there is no need for biochemical or drug monitoring. there is also no reason to believe that there is a difference in effectiveness. these reasons would likely result in ciprofloxacin being the first-line choice for the treatment of plague should it be shown to be effective. the primary objective of this trial is to test the hypothesis that ciprofloxacin monotherapy given (orally, or for subjects who cannot take oral medications intravenously or in combination) for days is noninferior to streptomycin (given on days - ) followed by ciprofloxacin (given on days [ ] [ ] [ ] [ ] [ ] [ ] [ ] for the treatment of bubonic plague. the regimens are respectively thirdand first-line treatments as per the national plague treatment guidelines in madagascar. the secondary objective is to collect data on the effectiveness of ciprofloxacin in the treatment of pneumonic plague, although the trial is not able to formally assess the non-inferiority of ciprofloxacin monotherapy compared to streptomycin and ciprofloxacin combination therapy in pneumonic plague, considering the operational and practical complexities of a plague rct, the study also has additional exploratory objectives to optimize investments: to evaluate the level and kinetics of anti-y. pestis antibodies in bubonic and pneumonic plague during treatment and follow-up, to compare different methods for detection of anti-y. pestis antibodies, to measure the performance of qpcr for plague diagnosis and to allow for the evaluation of new rapid tests that may become available. this is an individually randomized two-group parallel arm control trial with a : ratio. patients will be randomized to receive either streptomycin + ciprofloxacin or ciprofloxacin alone. we are using a non-inferiority design since the overall cure rate for bubonic plague without septicaemia with streptomycin is expected to be %. as a result, demonstrating superiority would be unnecessary and impractical given the sample size that would be required. our aim is therefore to demonstrate that ciprofloxacin alone is not more than % inferior to streptomycin followed by ciprofloxacin ( % is the non-inferiority margin in our study, meaning that the lower bound of the confidence interval around the risk difference in the success rates of streptomycin + ciprofloxacin and ciprofloxacin alone must not include %). the trial is powered for bubonic plague, although we will recruit patients with pneumonic plague as well. probable and confirmed cases are defined as: probable case: rapid diagnostic test (rdt) or qpcr or serology (anti-f igg elisa) is positive but without evidence of seroconversion or a fourfold increase in antibody titre. confirmed case: rdt and qpcr are positive, or culture is positive, or there has been a seroconversion or a fourfold increase in antibody titre on two separate serological samples (either between d and d , between d and d or between d and d ). we will recruit patients with clinical suspicion of bubonic plague, but the size of our sample is powered based on an intention to treat infected patient sample size of , where 'infected' is defined as a 'confirmed' or 'probable' case of bubonic plague. as a result, the total number of patients to be enrolled will be higher than . we estimate that we will need to recruit approximately patients with bubonic plague to achieve a sample size of confirmed/probable bubonic plague patients. however, to mitigate risks of being under-powered, we will recruit for three full seasons ( - ) with a minimum target of confirmed/probable cases, patients in each arm. should we achieve the target of confirmed/ probable bubonic plague cases before the end of the third season ( - ), we will nevertheless continue to recruit until the end of the season to retain power in the event the observed treatment success rates differ from those expected, and to allow us to increase precision. whilst we will also recruit and collect data on patients with pneumonic plague, it is highly unlikely that we will have the power to complete a non-inferiority trial for ciprofloxacin monotherapy for pneumonic plague patients. for example, with a case fatality rate of - %, we would need a sample size of approximately patients with probable/confirmed pneumonic plague for a non-inferiority margin of %, which is unrealistic. recruitment is expected to last years and will take place during each 'plague season' (the annual high transmission period of plague, which starts at the beginning of august and runs until the following march). the study will enrol males and non-pregnant females of any age with suspected bubonic or pneumonic plague. study setting { } study setting the trial will take place in up to five pre-identified districts in madagascar. madagascar reports more plague cases per year than any other country with an average number of confirmed/probable cases per year of [ ] . recruitment is planned to take place in health centres (primary health centres (centres de santé de base, csb) and district hospitals) in five districts. recruiting districts and sites have been selected based on their incidence of plague (as reported to ipm) and also for logistical reasons. however, the trial may be carried out in further districts or sites depending on the real-time incidence of suspected cases of plague during the study period. the majority of plague patients are treated at the csb level. in general, csbs do not have a laboratory, so sites perform an rdt to obtain a preliminary diagnosis and then send further samples to ipm for confirmation where a second rdt, qpcr and culture are routinely performed. patients of any age recent onset (< days) of fever (uncorrected axillary temperature ≥ . °c) or history of fever one or more buboes (tender lymph node swelling) residence or travel to a plague endemic or outbreak area in madagascar within days of the onset of symptoms patients identified as clinically suspected of plague by health personnel (doctors or paramedics) patients of any age recent onset (< days) of fever (uncorrected axillary temperature ≥ . °c) or history of fever cough tachypnoea (respiratory rate > in adults and agespecific in children) epidemiological link with a confirmed or probable case of primary or secondary pneumonic plague within days of onset of symptoms a small proportion of patients with bubonic plague will also develop pneumonic plague. these patients will be randomized to receive pneumonic plague therapy. for a patient to be considered to have mixed bubonic and pneumonic plague, they must have developed cough and dyspnoea after the onset of the bubo. known allergy to aminoglycosides or fluoroquinolones tendinitis myasthenia gravis theophylline or warfarin use already treated for bubonic or pneumonic plague in the preceding months women who report being pregnant given that streptomycin is an fda class d drug, women who report being pregnant will not be randomized into the trial but will be treated outside the trial according to the national guidelines. lactating women can safely be treated with streptomycin as streptomycin poses minimal risk to the infant when used during breastfeeding and will therefore be randomized into the trial. all eligible patients will be offered the opportunity to participate in the study. patient information sheets, consent and assent forms will be available in relevant written local languages. written informed consent to participate will be required from all participants or their representatives and will be requested by local study staff, who are qualified, trained and authorized to do so by the principal investigator. patients are fully informed about the use of biological specimens and their data in the patient information sheet, including lay descriptions of the procedures for collecting samples and details about data and sample storage. both treatment regimens tested are provided for by the national guidelines. there are several reasons why streptomycin is a suboptimal therapeutic agent for plague including global stock shortages, the requirement for parenteral administration and severe toxicity. in vitro assays suggest that ciprofloxacin has comparable efficacy to streptomycin and is superior to doxycycline or gentamicin for the killing of intracellular y. pestis [ ] . studies in non-human primates of pneumonic and bubonic plague have also shown high therapeutic efficacy of ciprofloxacin, equivalent to streptomycin [ , ] . ciprofloxacin also offers the advantage of the possibility of a switch from intravenous to oral treatment if clinically appropriate, does not require drug level or renal function monitoring, has fewer and less severe side effects, it is classed by the us fda as pregnancy category c (whereas other treatment alternatives are category d), has greater global availability and is cheaper. unlike other treatment alternatives, ciprofloxacin can be administered to children. dosing schedule in bubonic plague adults: mg orally twice daily (or mg iv twice daily for those who cannot take oral medication) for days. children: mg/kg twice daily (max mg per dose) orally (or mg/kg iv twice daily for those who cannot take oral-maximum dose mg) for days. patients who begin intravenous therapy may switch to oral administration once they are able to swallow or once deemed clinically appropriate by the treating physician. adults: streptomycin g twice daily for days, followed by ciprofloxacin mg orally twice daily (or ciprofloxacin mg twice daily by iv for those who cannot take it orally) for an additional days. children: streptomycin mg/kg twice daily for three days followed by ciprofloxacin mg/kg twice daily (max mg per dose) orally (or mg/kg iv twice daily for those who cannot take oral-maximum dose mg) for additional days. patients who start taking intravenous ciprofloxacin may switch to oral administration once they are able to swallow or once deemed clinically appropriate by the treating physician. adults: mg orally twice daily (or mg iv twice daily for those who cannot take oral medication) for days. children: mg/kg twice daily (max mg per dose) orally (or mg/kg iv twice daily for those who cannot take oral-maximum dose mg) for days. patients who begin intravenous therapy may switch to oral administration once they are able to swallow or once deemed clinically appropriate by the treating physician. adults: streptomycin g twice daily for days with ciprofloxacin mg orally twice daily (or ciprofloxacin mg twice daily iv for those who cannot take it orally) for days. children: streptomycin mg/kg twice daily for days with ciprofloxacin mg/kg twice daily (max mg per dose) orally (or mg/kg iv twice daily for those who cannot take it orally-maximum dose mg) for days. this is the alternative treatment proposed by the ministry of public health since the - plague season. patients who start taking intravenous ciprofloxacin may switch to oral administration once they are able to swallow or once deemed clinically appropriate by the treating physician. streptomycin is administered intramuscularly to all patients in the control group. ciprofloxacin is administered orally, except for patients with the following manifestations: vomiting unable to swallow unconscious systolic blood pressure < mmhg any other conditions or situations for which the physician decides that intravenous treatment is indicated ciprofloxacin: doses will be administered morning and evening. streptomycin: doses will be administered morning and evening. the study drugs will be administered for a total of days. after days, the decision whether to continue therapy will be at the treating physician's discretion. we are using days of therapy in both the ciprofloxacin arm and streptomycin and ciprofloxacin arm as this is the current standard duration of plague treatment in madagascar. to date, the only rct in plague used days of therapy (gentamicin versus doxycycline) in both groups. criteria for discontinuing or modifying allocated interventions { b} if, for any reason, a dose of the study drug is not administered at the scheduled time, it may be administered later, but not more than h after the scheduled time of administration. no dose adjustment is planned. the trial will be co-managed by ipm who will deploy a clinical research associate and two clinical study nurse to each district to assist with the recruitment, trial procedures and data entry. these members of staff will be based in the districts for the duration of each season. before activation, site investigators will receive training on the identification of plague patients and the study protocol. after activation, the site investigators will be responsible for patient management alongside the clinical study nurse. to optimize patient compliance, community health workers (chws) in each district will be trained to identify cases of plague based on community case definition as per the national guideline, refer patients to the local csb i/ii, perform treatment visits to patient homes and complete the treatment record for each home-based visit. at the time of discharge from the facility, patients will also be equipped with a mobile phone, as follow-up visits on d and d , and possibly m will require a return visit to the treating csb. at the beginning of each new plague season, all site staff will be retrained on the trial protocol and reinitiated to the trial. this will reinforce the protocol following the annual recruitment suspension and ensure that, due to high staff turnover at the csbs, any new staff members receive the same comprehensive training as those who were trained at the original initiation visit. a monitoring plan has been developed to ensure regular reviews of trial data and site compliance with the protocol. these reviews will be completed within h of each of the first five patients being recruited at each site, annually at the end of each plague season and on the identification of persistent or serious protocol violations and serious breaches. the trial operations committee (toc) will also meet on a weekly basis to review recruitment and compliance and discuss any other issues affecting the trial. the trial has also appointed a data safety monitoring board (dsmb) and trial steering committee, who will meet regularly to discuss the management of the trial. other treatments, including the administration of fluids and organ support, will be at the discretion of the treating clinician. post-trial care following their exit from the trial, patients will return to their standard care pathway. the university of oxford, as study sponsor, has arranged appropriate insurances to provide for the university's responsibilities to research subjects and to cover the legal liabilities of the university to those engaged by the university in the performance of this research. the proportion of patients with bubonic plague with a therapeutic response (assessed on d ). therapeutic response is defined as follows for subjects with a visible and measurable bubo: resolution of fever (uncorrected axillary temperature < . °c) a ≥ % decrease in bubo size (in the case of multiple buboes, the largest bubo) has not received alternative treatment for plague no clinical decision to continue anti-plague antibiotics beyond day for patients with small buboes that are palpable but not measurable: absence of fever (uncorrected axillary temperature < . °c) bubo has not enlarged has not received alternative treatment for plague no clinical decision to continue anti-plague antibiotics beyond day proportion of patients without fever (uncorrected axillary temperature < . °c) at d proportion of patients with a pain score < at d proportion of patients with a pain score < at d mean % change in bubo size at d mean % change in bubo size at d proportion of patients experiencing a serious adverse event on or before d proportion of patients experiencing a serious adverse event on or before d proportion of patients experiencing a serious adverse event on or before d proportion of patients who are fully adherent to the study treatment schedule proportion of patients with a therapeutic response at d . therapeutic response is defined as follows: alive resolution of fever (uncorrected axillary temperature < . °c) resolution of tachypnoea (rr < in adults, but age-specific in children) proportion of patients without fever (uncorrected axillary temperature < . °c) at d proportion of patients with tachnypnoea resolution (rr < in adults, but age-specific in children) at d proportion of patients experiencing a serious adverse event on or before d proportion of patients experiencing a serious adverse event on or before d proportion of patients experiencing a serious adverse event on or before d proportion of patients who fully adhere to the study treatment schedule the participant timeline is described in the flow chart (fig. ) . the schedules of biological procedures, assessments and samples are summarized in the following tables (tables and ) . the conduct of the study is described in fig. . the maximum volume of blood to be collected during the entire duration of the study ( months) for confirmed cases with a positive serology at d will be ml for children and ml for adults. assuming that % of individuals receiving streptomycin plus ciprofloxacin (control treatment) meet the primary endpoint definition of therapeutic response on day , confirmed/probable bubonic plague patients ( per group) would be required to have % power to demonstrate the non-inferiority of ciprofloxacin treatment compared to streptomycin plus ciprofloxacin, with a % non-inferiority margin and a one-sided confidence interval of . %. this calculation takes into account a % loss to follow-up rate by the end of treatment. in total, approximately suspect patients will have to be recruited to obtain a sample size of confirmed/probable patients. table shows the total sample sizes (both arms) for confirmed/probable bubonic cases to demonstrate noninferiority with % power, . % one-sided test and assuming % loss to follow up, based on treatment success in the streptomycin and ciprofloxacin group and the non-inferiority margin. sites will be geographically spread throughout preidentified districts which report a high incidence of plague. further districts may be included later in the trial in the event that they also report a high incidence of plague. whilst it is the responsibility of site medical staff to assess eligibility and enrol patients, the clinical study nurse, who will be based within the district, will review and verify the eligibility of each patient. community health workers, who will cover a wider geographical area within the districts and reach more remote communities, will also be trained on the protocol and refer suspected cases of plague to the local csbs. outreach teams will disseminate health promotion messages to local health authorities and communities about plague (i.e. signs and symptoms), as well as the background and objectives of the trial. community health workers will also raise awareness about going to health centres in the event of suspected plague. a social science component will address the practical aspects and uptake of the trial that may be affected by the perception of the trial design by both heath care professionals and patients. bubonic or pneumonic plague patients will be randomized using a computer-generated randomization sequence generated from the master list, stratified by health facility and clinical form of the disease (bubonic or pneumonic plague). the allocation sequence was compiled in stata in a way that is unpredictable and unreproducible. a copy of the stata randomization code file used to compile the list can be requested from the trial statistician. the trial statistician will have access and be responsible for the generation and quality assurance of the randomization list. this also includes any changes and/or extensions to the trial statistician has generated the allocation sequence. the doctor in-charge at the csb will confirm the eligibility of patients with the support of the clinical study nurse. to randomize patients, the doctor in-charge will call the clinical research associate who will randomize the patient on the site's behalf, informing the site of the treatment allocation over the phone. contingency plans have been put in place to ensure randomization can take place in the event of internet or mobile signal outage. blinding of patients and the trial team to treatment allocation will not be possible due to the different administration routes of the treatments. not applicable-this is not a blinded trial. data will be collected on a paper crf and then transferred on to a redcap database for assessment. automatic validations on redcap will assess data for completeness and accuracy. a thorough assessment of data accuracy will be performed via regular monitoring activities with a focus on primary endpoint data and safety data, as described in the trial's monitoring plan. to validate the methods used to achieve the primary endpoint (% decrease in bubo size), a pilot study and training exercise were conducted ahead of activation in which multiple bubo (real and artificial) measurements were taken using digital callipers (and ultrasound during the pilot). the data were used to assess the concordance of measurements being taken by independent observers. the percentage difference between the sum of the longest and shortest axes of measurements taken by two different clinical study nurse was calculated with an acceptable margin of %. the results showed there was less than % difference between each measurement. patients will be provided with a mobile phone upon discharge from the treating centre to optimize retention adverse events x x x x x x bubo size and pain score x x x x assessments will be done within h of enrolment, depending on the time of admission m months after inclusion during follow-up. community health workers will also assist with reminding patients about their upcoming follow-up visits. if a patient chooses to withdraw from trial treatment (or the treating clinician believes this is in the best interest of the patient), the patient will continue to receive the standard of care treatment at the discretion of their treating clinician, regardless of the treatment arm to which the patient was randomized. the patient will remain within the trial for the purpose of follow-up, data collection, monitoring and data analysis. healthcare personnel should explain the importance of allowing existing data to be used, as well as the importance of remaining within the trial for follow-up data collection. if it is not possible to collect further data from the patient at the scheduled trial visits, routine data should be collected via the patients' medical records. if a patient withdraws their consent to participate in the trial, data and sample collection will cease and only data up to the point of withdrawal will be used for analysis. all data will be recorded on a paper crf by clinical study nurses at each study site using a combination of data directly from patients, patient records and laboratory results. the clinical study nurses will then transfer the data from the paper crf to a redcap database (version . . ), which will be located centrally at ipm with restricted access. data queries will be automatically generated by redcap and transferred to the operational teams in the districts for data quality monitoring as the study progresses. data monitoring will be conducted on a regular basis as defined in the monitoring plan. the final database will be transferred to oxford university. a statistical analysis plan will be prepared prior to the receipt of any data by the trial statistician. the management, reporting processes and data storage within this trial will comply with the requirements of european directive (reference / , "gdpr directive") on data protection which is translated into english law by the "data protection act" of may , the good clinical practices of the gcp-ich, and cioms. all the systems used for data management comply with the gcp-ich and cioms recommendations as well as the european directive (reference / , "gdpr directive") on the protection of personal data, which is translated into english law by the "data protection act" of may . trial staff will ensure that the participants' anonymity is maintained. all trial documents will be labelled only with a trial identification number. trial samples will be labelled only with a trial id. routine bubo and sputum samples, which will be analyzed as part of the trial, will be transferred to ipm as per the standard transfer process. the routine samples taken from trial patients will be labelled with a collection label which will not contain a patient trial identifier. a separate letter will be sent to ipm containing anonymised collection information and the patient's trial id. this will be used to ensure the results of sample testing from the lab are collected for the trial without compromising the patient's anonymity. for the purpose of ensuring data integrity and to facilitate quality assurance, study records will be linked to clinic files, which include the patient's name and other identifying information. participants' names will be recorded confidentially at site in a protected master list at the time of enrolment to allow for their identification at follow-up visits. the master list will not be shared with any other parties involved or not involved in the trial. all trial data will be stored in a secure database only accessible to trial staff. only members of the trial team who have completed appropriate data protection training will have access to the password-protected computer where trial data is stored. study sponsors and health authorities will be given controlled access for the purpose of audit. after the conclusion of the project, data will be removed from the computers and stored in a secure location. any scientific publications or reports will not identify any patient by name or initials. plans for collection, laboratory evaluation and storage of biological specimens for genetic or molecular analysis in this trial/future use { } aliquots of blood, bubo aspirates and sputum samples will be stored initially at the site and then transported to ipm. sample custody will be maintained by the investigators, and decisions regarding the use and transfer of samples to the research laboratory will be made by the toc. biological research samples will be stored indefinitely, and approval from the sponsor and/or ethics committees, as appropriate, will be sought prior to destruction. the bubo will be aspirated, and a rdt for plague will be performed on site. this rdt for the detection of the f antigen of y. pestis is manufactured by ipm. the test has been used routinely over the past decade and has been validated for bubonic plague. site staff, clinical research associates and clinical study nurses will be trained on the use and interpretation of the rdt before the start of the study and during annual retraining. the remaining bubo aspirate will be stored in cary blair transport medium, packed according to the required safety level using the materials already provided (triple packaging), then sent to the cpl at ipm for a second rdt analysis as well as for molecular and bacteriological diagnosis. molecular diagnosis will first be performed by real-time pcr (qpcr) targeting fragments of the pla and caf- genes of y. pestis, and only samples with inconclusive results will be confirmed by conventional pcr (cpcr) targeting genes pla, caf- and inv . bacteriology will be done with direct culture and amplification in mice followed by biochemical identification of the suspected strain by api e and by bacteriophage lysis test, then antibiotic susceptibility testing of each y. pestis isolated strain according to the kirby bauer method interpreted according to the clsi reference. sputum samples will not be tested on site with rdt but will be sent directly to the cpl. biological results will be sent back to the physician responsible for the patient. a malaria rdt will be performed on all patients at admission. a -ml venous blood sample will be taken at admission (d ), d and d for plague serology (for children, a -ml venous blood sample will be taken at these time points). in addition, for subjects with positive serology at d , serological follow-up will be carried out at months (m ) after their inclusion. two types of serological techniques will be performed: serology by immuno-enzymatic assay method (elisa, anti-f antibodies, igg) which is validated and will be performed at the cpl, and serological analysis by xmap-luminex multiplex technique (magpix) to simultaneously detect antibodies (igm and igg) directed against several y. pestis antigens will be performed at the immunology of infectious diseases unit (imi) at ipm. a serological test (on magnetic beads, xmap-luminex) to detect and quantify the f circulating antigen of y. pestis will also be performed. only anti-f igg elisa technique will be considered for case classification. statistical methods for primary and secondary outcomes { a} primary endpoint analysis the final classification of cases will take into account all clinical and biological data in accordance with who recommendations adapted in with the introduction of qpcr as a diagnostic tool [ , ] . the data will be analyzed using four analysis populations: (i) intention to treat (itt), (ii) intention to treat infected (itti), (iii) per-protocol (pp) and (iv) perprotocol infected (ppi). intention to treat infected (itti) will be the primary analysis population. an infected patient is a confirmed or probable case. a case is probable if the rdt or qpcr or serology (anti-f igg elisa) is positive but without evidence of seroconversion or a fourfold increase in antibody titre. a case is confirmed if the rdt and qpcr are positive or if the culture is positive or if there has been a seroconversion or a fourfold increase in antibody titre on two separate serological samples (either between d and d , between d and d or between d and d ). the criteria for exclusion from the per-protocol analysis population will be specified in advance in the statistical analysis plan (sap). the proportion of patients who meet the pre-established definition of therapeutic response in the streptomycin plus ciprofloxacin group will be subtracted from the proportion of patients who meet the pre-established definition of therapeutic response in the ciprofloxacin group. a positive difference in proportions would imply a higher response in patients treated with ciprofloxacin, compared to streptomycin plus ciprofloxacin, and a negative difference would imply a lower response in patients treated with ciprofloxacin. if the lower limit of the one-sided . % confidence interval (ci) around the difference in proportions is not less than − . , ciprofloxacin will be considered non-inferior to streptomycin plus ciprofloxacin. descriptive summaries of the data will be provided by randomization arm for the pre-specified secondary outcomes. the data will be summarized by randomization arm using the mean ( % cl) and median (interval) for continuous or discrete data and raw data with percentages and % cis for binary variables. the average time to resolution of fever and pain at the bubo site will also be summarized. no formal interim analysis is planned. it is unlikely that the trial can be stopped prematurely, as a smaller sample will not provide the precision required to demonstrate non-inferiority within the pre-specified margin. no additional analyses are planned. methods in analysis to handle protocol non-adherence and any statistical methods to handle missing data { c} missing data handling: primary endpoint if a patient has had a resolution of fever and shows a ≥ % reduction in bubo size on or after day , but is lost for follow-up before day , he or she will be considered to have had a therapeutic response. any deviations from the protocol will be fully documented on a protocol deviation form that will be sent to the sponsor and stored in the trial master file and on site in the study site file. plans to give access to the full protocol, participant-level data and statistical code { c} there are currently no plans to grant public access to the full protocol, participant-level dataset or the statistical code. composition of the coordinating centre and trial steering committee { d} ipm will jointly coordinate the trial with the sponsor, the university of oxford. as well as a central trial coordination team to coordinate the day-to-day activities of the trial, ipm will deploy one clinical research associate and two clinical study nurses to each district who will assist the sites with the recruitment, data collection and conduct monitoring activities. ipm will also be primarily responsible for data management but supported by the university of oxford. the university of oxford will coordinate the sponsor activities, coordinate the trial committees and provide day-to-day management and oversight of the trial. the toc is coordinated by the university of oxford and is composed of members from each of the study partners to review the operational management of the trial and discuss any urgent matters related to trial conduct as and when they arise. the group meets on a weekly basis to discuss study progress. the trial steering committee provides overall supervision for the trial on behalf of the trial sponsor and to ensure that the study/trial is conducted according to the guidelines for good clinical practice (gcp) and all relevant regulations and local policies. a data safety monitoring board has also been established to maintain oversight of patient safety and data integrity within the trial. composition of the data monitoring committee, its role and reporting structure { a} an independent data safety monitoring board (dsmb) has been established to safeguard the interests of trial participants, to assess the safety of the interventions during the trial and to assist and advise the principal investigator (pi) so as to protect the validity and credibility of the trial. the dsmb will meet remotely ( ) upon the opening of the trial, ( ) at the end of each plague season and ( ) streptomycin and ciprofloxacin are approved drugs with well-established safety profiles. serious adverse events/ serious adverse reactions (saes/sars) and suspected unexpected serious adverse reactions (susars) will be reported in accordance with the international guidelines and applicable national legislation. all adverse events (aes) observed or reported by the patient should be recorded on the ae form within the crf. in the event of an adverse event or reaction, the investigator is responsible for providing an appropriate medical response, providing a causality assessment and monitoring the progress of the ae until it has resolved. in the event that an adverse event is deemed serious, according to the international definition (ich), it should be notified immediately to ipm via the sae report form within h of the site becoming aware of the event. newly arising saes must be reported from the point of consent until d . all saes must be followed up by the site until resolution (including when resolution occurs after d ). the following events, which are either classified as treatment evaluation criteria or inclusion criteria, are exempt from reporting as an sae: respiratory symptoms (cough, tachypnea, haemotysis) intestinal symptoms (diarrhoea, vomiting) fever bubo pain at bubo's site in addition, the following circumstances should not be notified to the sponsor: medical/surgical/hospital procedure planned before inclusion a condition present before inclusion or discovered at inclusion (e.g. malaria discovered at inclusion and requiring hospitalization) all saes/sars reported to ipm will be notified to the sponsor within one business day and will be reviewed and evaluated by two clinical reviewers. if the reviewers have concerns about the saes submitted, an emergency meeting of the dsmb and the toc will be convened to discuss the management of the sae and the potential impact on the trial. all susars will be reported to the centre national de pharmacovigilance in madagascar within days for fatal or life-threatening reports and within days for other serious, unexpected sars for all initial reports. none. plans for communicating important protocol amendments to relevant parties (e.g. trial participants, ethical committees) { } all protocol amendments will be discussed by the toc which includes members from each of the study partners ahead of submission to the respective ethics committees. it is the responsibility of each institution to obtain their own ethical approvals. only once all ethical approvals have been obtained will the amendment be implemented. it will be the responsibility of ipm to disseminate the changes to the protocol to participating sites. the principal publication of the study will be in the name of the investigators with full credit assigned to all collaborating investigators, research coordinators and institutions. oral or written communications will acknowledge all trial stakeholders, either in the list of authors or in the acknowledgements, depending on their respective contributions to the trial and the presentations made. a drafting committee will be formed at the time of data analysis. the order of authors at publication will be equally contributed, by the importance of contribution or in alphabetical order. results from the trial will be published in open-access journals, and the data will be available for sharing. conducting clinical trials of plague is operationally challenging, mostly because cases occur in rural places with difficult access to health facilities, especially because cases occur mostly during the rainy season, and are scattered over a vast territory. communication and network coverage may be erratic, and security issues may also occur. this requires adequately staffed, welltrained study teams to be deployed over a number of peripheral health facilities, with logistical challenges and inherent high costs. this, plus the fact that drugs are registered for plague (based on the 'animal rule') and that national and international guidelines exist (based on empirical regimens), along with the general lack of funding options for plague, has essentially discouraged the conduct of treatment trials. where necessary, obtain the approval of the above parties for all substantial amendments to the original approved documents. written informed consent to participate will be obtained from all participants. author details recruitment start date: planned february planned recruitment end date additional file . : patient information sheet adverse event; c: celsius; caf- gene: chromatin assembly factor- ci: confidence interval; cioms: council of international organizations of medical sciences; clsi: clinical & laboratory standards institute; cpl: clinical plague laboratory; crf: case report form d: day; dfid: department for international development; dsmb: data and safety monitoring board; elisa: enzyme-linked immunosorbent assay f : fraction antigen; fda: food and administration; gcp: good clinical practice; gdpr: general data protection regulation igm: immunoglobulin m; inv gene: chromosomal invasin gene ppi: per protocol infected; qpcr: quantitative polymerase chain reaction rct: randomized clinical trial; rdt: rapid diagnostic test; rr: respiratory rate sae: serious adverse event; sap: statistical analysis plan; toc: trial operations committee; who: world health organization plague around the world in epidemiological characteristics of an urban plague epidemic in madagascar report on the outbreak of bubonic plague: being a report based upon observations on cases of bubonic plague treated at the municipal hospital for infectious diseases at arthur road treatment of plague with gentamicin or doxycycline in a randomized clinical trial in tanzania successful treatment of human plague with oral ciprofloxacin drugs@fda: fda approved drugs world health organization. plague around the world in effect of antibacterial therapy on the epidemic threat of experimental pneumonic plague in monkeys evaluation of the effectiveness of antibacterial substances in treating an experimental form of bubonic plague in monkeys world health organization. plague around the world publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations the authors would like to acknowledge the contribution of dr. feno rakotoarimanana and inès vigan-womas for their advice and contribution to this protocol. received: april accepted: july this protocol was written by po with assistance from jb and based on the main study protocol written by po, ag, te, mr, rr, as, lb, al and jb. clinical guidance was provided by po, rr, mr and as. all authors have approved the submitted manuscript. the authors declare that they have no competing interests. this trial is supported by the department for international development and wellcome [grant number /z/ /z]. the role of the funder is to provide financial support for the trial. a representative of the funder will act as an observer on the trial steering committee. the trial collaborators are responsible for the design, management and analysis of the trial and the publication of the trial results. as sponsor of the trial, the university of oxford is the owner of the data. on the conclusion of the trial, oxford will provide a copy of the trial dataset to the trial investigators at institut pasteur de madagascar, centre hospitalier universitaire joseph raseta befelatanana and centre d'infectiologie charles mérieux (antananarivo) and the trial statistician at the london school of hygiene and tropical medicine in order for the trial team to fully participate in the development of the main publication. not applicable-no details, images or videos relating to an individual person have been collected as part of this publication. the investigators will ensure that this study is conducted in accordance with the current version of the declaration of helsinki ( version) and the applicable principles of the international committee on harmonization of good clinical practice (gcp-ich e ). the protocol, the informed consent form and the crf have received approval from the madagascar biomedical research ethics committee, and the protocol, patient information sheet and informed consent form have received approval from the oxford tropical research ethics committee and the ethics committee of the london school of hygiene and tropical medicine. the chief investigator will submit and, key: cord- -utqyuy n authors: zamani, efpraxia d.; pouloudi, nancy; giaglis, george m.; wareham, jonathan title: appropriating information technology artefacts through trial and error: the case of the tablet date: - - journal: inf syst front doi: . /s - - - sha: doc_id: cord_uid: utqyuy n the concept of appropriation is of paramount importance for the lasting use of an information technology (it) artefact following its initial adoption, and therefore its success. however, quite often, users’ original expectations are negatively disconfirmed, and instead of appropriating the it artefact, they discontinue its use. in this study we examine the use of it artefacts following negative disconfirmation and use grounded theory method techniques to analyse blogposts, collected between march – july , to investigate how users appropriate or reject the tablet when technology falls short of users’ expectations. our findings show that users overcome negative disconfirmation through a trial and error process. in doing so, we identify that users appropriate the tablet when the attained benefits significantly outweigh the risks or sacrifices stemming out of its use. we discuss our contribution within the context of the appropriation literature, and highlight that the success of it lies with the user’s success in identifying personal use scenarios within and across diverse contexts of use. contemporary information technology (it) devices are boundary spanning and accommodate different contexts of use, covering both professional and personal use scenarios. such it devices can be smartphones and tablets, among others, and their adoption and use is largely volitional (schmitz et al. ) , which means that the individual user is able to decide and control their use. this indicates a great heterogeneity of potential uses, and further signifies increased user control over a device's adoption, modification, and even rejection. these two points create an important challenge where the user can easily switch between contexts, with it having to satisfy their requirements irrespective of the changing environments. this is important because the success of an it artefact resides with the user identifying a benefit in it use against the background of personal use scenarios. for this reason, today there is a large body of research that examines why users accept an it artefact and how they make use of it (barnett et al. ) . often, these studies draw from theories, such as the technology acceptance model (davis and warshaw ) , the unified theory of acceptance and use of technology (venkatesh et al. ) and their variations (kim and garrison ; e.g., venkatesh et al. ; venkatesh and davis ) . these theories place the emphasis on the factors that drive user decision with regard to the acceptance or rejection of the technology (aggarwal et al. ) . however, because these theories place the emphasis on the preliminary stages of user interaction, less or no emphasis is inescapably given in how users actually make use of the technology, which exerts a greater impact on its viability (venkatesh et al. ) . the aspect of how users make use of it is typically examined by post-adoption user behaviour studies (zamani et al. ) , and often through the lens of appropriation which explains how users adapt and modify the it or even refine their it use in order to achieve their goals (clark ) . studies often focus on the fit among user, task and it (e.g., barki et al. ) or the variations of appropriation acts and adaptations (e.g., pallud and elie-dit-cosaque ) . however, existing research has two main shortcomings. first, there is a key assumption that a fit can be achieved via appropriation. second, research mainly focuses on organisational systems within enterprise-focused contexts (e.g., a. bhattacherjee and premkumar ; jasperson et al. ) . even when the unit of analysis is set at the individual level, the user is approached as an organizational member (e.g., ahuja and thatcher ) . as a result, appropriation studies often miss the particularities of it where appropriation is not necessarily side-stepping top-down imposed use patterns but equally an effort to satisfy user goals. in this paper, we are interested in seeing "what is the process of appropriation of it devices by individual and volitional users following negative disconfirmation?" this is of distinct importance. disconfirmation denotes the discrepancy between the user's original expectations and their perceptions regarding the actual performance of the it device post-usage (a. bhattacherjee and premkumar ) . positive disconfirmation suggests that the user is pleasantly surprised but negative disconfirmation is crucial as it often leads to discontinuance behaviour and the abandonment of the it device. it is thus imperative to understand how, rather than abandoning the it device, the user can be successful and incorporate it successfully, within their portfolio of other it devices. appropriation signifies the situation where the user has managed to overcome the said discrepancies, and made the it device their own, using adaptations and modifications (a. k. barrett ; clark ) . most importantly, appropriation leads to habitual norms and routines (dennis et al. ) , and as such, to the lasting use of the it device (wu et al. ) , which is what makes an it device fairly successful. we use the tablet as an exemplary case, and specifically the ipad. we chose to focus on the ipad because since its launch, it has been consistently popular among the mass consumer market, and because it has attracted the attention of both practitioners and end users (zamani et al. ) . we use grounded theory method (gtm) techniques specifically because our aim is to develop a theory grounded on the data, using systematic ways for data collection and analysis, while taking stock of prior research and with the objective to enrich the appropriation literature. our empirical material consists of blog posts, covering years. we analysed this material following the approach put forward by volkoff et al. ( ) by drawing iteratively from the relevant literature and our data, and examining the empirical material through the lenses of competing theories. this resulted in the identification of behavioural patterns that are compatible with the concept of trial and error. our contributions are twofold. first, we introduce trial and error as a new variation of appropriation that addresses some of the observed shortcomings of the existing variations. second, we study user behaviour with contemporary, highly popular it devices. as their use is not necessarily organizationally mandated, their success relies on the users identifying personal use scenario(s) that can serve them within and across diverse contexts of use. against this background, understanding the processes users go through while moving from negative disconfirmation to appropriation is of crucial importance for the success of the examined it devices within these diverse contexts. in what follows, we first discuss prior work on appropriation. we then present the methods used and our findings. then, we discuss our findings in relation to the existing literature and theorise on trial and error. we conclude our paper with the contributions of our research and its limitations. appropriation is a prerequisite for the sustained and lasting use of it systems as it feeds into the formulation of norms and routines (dennis et al. ) by supporting users in developing personal use scenarios (mäkelä and vellonen ; wu et al. ). as such, appropriation is core for the success of it artefacts. in the next section we examine the different variations of appropriation. clark ( ) defines appropriation as the "situation where the user starts by recognizing the potential value of a particular it and manages to narrow the absorption gap between the requirements of the it and its own limited capacities" (p. ). similarly, carroll and fidock ( ) describe the concept as "seeking a relationship with the technology so that it provides benefit to the user through supporting practices, enabling new -and beneficial -practices or removing ineffective practices" (p. ). existing research emphasizes the impact of appropriation on performance (e.g., beaudry and pinsonneault ; desanctis and poole ) with several studies analysing its different variations, ranging from workarounds (e.g., alter ) and adaptations (e.g., elie-dit-cosaque and pallud ) to improvisations (e.g., mcgann and lyytinen ) . by far, the most well research variations are those of adaptations and workarounds. according to beaudry and pinsonneault ( ) , adaptation guarantees a fit between technology and user, as a result of the user changing routines and habits, enriching their skillset, and changing the technology with the aim to achieve their goals. schmitz et al. ( ) have further extended this concept through the lens of adaptive structuration theory and discuss that, as a result of the technology's malleability, users adapt the technology and their tasks either subtly and progressively (exploitive adaptation) or more exploratively, by reinterpreting the technology and its features (explorative adaptation). the explorative type of adaptation is quite similar to the enhanced use of it, proposed by bagayogo et al. ( ) , where the user attempts to find new ways of using features of an it system. this may include using previously unused or underutilised features (bagayogo et al. ) . the exploitive type of adaptation strongly resembles the deep structure usage, proposed by desanctis & poole (desanctis and poole ), whereby both approaches focus on the extent and the intensity of the use of it features for task completion, and the user efforts to progressively use more and more it features and functions. while in all the aforementioned approaches there is a common element of learning (kwahk et al. ) where users recognise and put in use new approaches to task completion (barki et al. ) , there are differences as well. namely, enhanced use of it emphasizes that it use may change over time (bagayogo et al. ) , whereas schmitz et al. ( ) and beaudry and pinsonneault ( ) focus more on the identification of a fit among task and technology or technology and user. on the other hand, deep structure usage is a rather more demanding form of usage (tams et al. ) , as it entails that users comprehend how the system is structured (burton-jones and straub ) . in the effort of identifying a fit and appropriating an it device, it is not used as originally designed (schmitz et al. ) . user modifications may or may not be in the spirit of the original design (desanctis and poole ), and essentially become workarounds (alter ) as the user engages with it outside recommended rules (ferneley and sobreperez ) . workarounds are often seen as a form of resistance behaviour within an organizational setting. yet, numerous studies to date show that, when workarounds become stable over time, they are merely an indication of an inadequately designed information system (ferneley and sobreperez ) , and should be seen as integral for the completion of day-today work (e.g., azad and king ) . as workarounds are often developed for tackling in situ newly emergent shortcomings, they are closely related to revisions of it use (sun ) and improvisation acts (morrison ) . revisions of it use concern primarily the way users revise their it use due to novel and challenging situations when users are faced with discrepancies or other deliberate initiatives (sun ) . such revisions allow users to meet their needs and overcome inadequacies of it and processes (lee et al. ) . within this context, users may try new features, substitute older ones, or combine some and even repurpose them, in their attempt to make better use of existing and new it features, functions and extensions (sun ) . in this sense, revisions of it as an appropriation variation shares a common focus with enhanced it use on how it use may change over time, as it emphasizes that "a person's [technological features in use] is always in flux" (sun , p. ) . such novel uses of it in light of shortcomings, resource shortages and other workplace challenges can equally take the form of improvisation acts (morrison ) . combining, recombing and repurposing it denotes an attempt to try out different things in order to solve problems. in doing so, users may cast a wide net that goes well beyond what they know, which allows them to innovate and identify novel solutions (scheiner et al. ) . while trying to find new ways of using it, users essentially try to innovate (tams et al. ) , and in doing so they make it their own (ahuja and thatcher ) . workplace innovation with it suggests that, users seek to incorporate it in their processes (wu et al. ) while attempting to do away with the restrictions enacted by it itself (schmitz et al. ) . the difference between trying to innovate with it and improvisation is that first places emphasis on one's goals about the outcome of the interaction (ahuja and thatcher ) , while the second entails thinking and acting "simultaneously and on the spur of the moment" (ciborra , p. ) . therefore, while trying to innovate can be seen as one's behavioural coping toward identifying new uses for existing systems and ways to support new tasks (wu et al. ) , improvisation is often seen as unpredictable because users need to work with whatever is available (ciborra ) as a result of e.g., resource shortages (morrison ) . table offers a summary overview of the different appropriation variations presented here, offering additional information with regard to the context of study and the unit of analysis typically adopted by existing studies. we note that, while some studies focus on the individual level, the majority focuses on organizational contexts where it use is mandatory. we further highlight that while the previous discussion has highlighted their differences, the common denominator across all variations is that of the user making the it their own, irrespective of the exact process adopted. in what follows we highlight the current shortcomings of the appropriation literature. first, all variations recognize that how users use it may be different from the originally designed instrumental use. this innovating with it allows users to identify new it uses for task completion. it entails that the system's value can only be realized when it is fully integrated into one's workflow (ahuja and thatcher ) . trying to innovate is goal-directed and can be a team learning behaviour (magni et al. ) and is influenced by one's personal innovativeness (haug et al. ). users revise their it use as a result of emerging situations and discrepancies, with the aim to overcome disconfirmation and appropriate it (sun ) . it revisions may also stem out of the introduction of new it features, functions and extensions that support the system's increased use (bagayogo et al. (ferneley and sobreperez ) . they can generate both stability and fluidity, (re) creating routines (rossi et al. ) improvisation is about thinking and acting "simultaneously and on the spur of the moment" (ciborra , p. ) , with the aim to find viable solutions (ciborra ) . therefore, it requires expertise, competence and intuition (fernandes ) , so that users are able to identify relevant solutions (orlikowski ) . it offers flexibility at the individual level (rossi et al. ) level of analysis organizational members (barki et al. ; beaudry and pinsonneault ) individual ( suggests that users move from 'technology-as-designed' to 'technology-in-use' by engaging with the technology, exploring its potential and modifying, eventually, its features as per their requirements, which is what leads to the appropriation of it (lapointe and beaudry ) . second, all variations are focused on identifying a fit among task, technology and user. this is done either by modifying the technology, the task, user behaviour or any combination of these. the assumption here is that a fit does exist and can be found by the user. however, a user may follow the same processes that are said to lead up to appropriation, without achieving the desired outcome, i.e., appropriation itself, eventually rejecting the it. this possibility is under investigated in the existing literature. third, the extreme majority of studies focuses on enterprise-level systems and/or on organizational contexts (see table ). for example, improvisation studies focus on the improvisational capability of organizations (e.g., chatterjee et al. ) , or the improvisation acts of organizational members during fire-fighting situations (alblas and langerak ; repenning ) . such an emphasis is expected considering that investments in enterprise software are costly (bagayogo et al. ) and reluctance, or even failure, to adopt enterprise it endangers performance and profitability. this assumes that users, while attempting to appropriate the it, will prioritise the requirements of the organization over their own. it also assumes that there is a restricted set of it systems which has been imposed top-down by the organization. as such, even when the unit of analysis is that of the individual, users are treated as organizational members with the assumption that their commitment to make things work may be low. however, individual users under their volitional control, may proceed at their discretion with the appropriation of it; in doing so, they may employ a multitude of adaptations and modifications, if and when required (schmitz et al. ), without constraints imposed by a top-down hierarchy, they may not abandon the process when they identify a good enough solution, or abandon the process altogether should they consider that a tentative solution is either non-existent, or too difficult (swann ) . further, under the user's volitional control, where a top-down description of what constitutes suitable use does not exist (schmitz et al. ) , the acts of adaptation, exploration, and appropriation can be more flexible and indeed lead to rejection without (fear of) sanctions; time and flexibility allow users to discover possibly unexpected usages that can increase their satisfaction with the it artefact. as such, research should focus more on use contexts beyond the typical organizational so as to appreciate the richness of the phenomenon, how it may be adapted when its use is not mandated and users are able to choose from a wider set of action possibilities. we seek to understand "what is the process of appropriation of it devices by individual and volitional users following negative disconfirmation?" this research question entails a broad focus around post adoption use behaviour, but is squarely rooted on the discrepancies users experience between initial experiences and realities in use. this broad focus further allows us to consider the problems users may encounter while attempting to fulfil a task, and the different solutions they may enact with the it device in question, or any other it device at their disposal. to address this question, we designed a qualitative study using grounded theory method (gtm) techniques. gtm is often used for describing process-based phenomena, and offers data collection and analysis that support the emergence of theory from the data, while being guided by existing relevant theories (urquhart ) . we follow a bottom-up approach in line with the gtm paradigm (e.g., m. barrett and walsham ; boudreau and robey ; volkoff et al. ), meaning that trial and error emerged from our preliminary analysis of the collected data, when we observed that users, first, had an expressed interested in using the tablet for various reasons and that failing to use it in an as-is fashion, they were trying out different things. this observation led us to proceed with subsequent data collection in order to identify either similar or dissimilar behavioural patterns, where the second phase of data collection was influenced by the concepts emerging from the first phase (theoretical sampling) (urquhart ) . our gtm study was designed specifically around the tablet and occurrences of negative disconfirmation with it, in order to capture the processes of appropriation following such occurrences. we chose to focus specifically on the ipad. the ipad, as an exemplar case of a tablet is particularly popular among users, offers a fairly consistent user experience across its numerous generations (zamani et al. ) , and thus allows for maximum similarly in the data, which in turns leads to generating and verifying basic properties and conditions for our core constructs (urquhart ) . for our gtm study, we followed the stages of analysis as proposed by glaser (glaser ) and charmaz (charmaz ) of coding and theorising around the open (or initial), selective (or focused) and theoretical codes, writing up memos and theorising around these, too, and finally, integrating and linking up our codes and core categories through a constant comparative method, finally developing our trial and error theory of appropriating. these are discussed in detail in the next two subsections. when we began looking into the tablet and how users use it, we noticed that many bloggers were documenting their experience with the tablet in their personal blogs. within them, the tablet users were offering narratives of their everyday life, sharing their goals and experiences with the tablet as well as detailed accounts of their interaction, the problems they were encountering and the employed strategies towards solving these. we therefore considered safe to assume that these bloggers were able to provide candid descriptions in their blog posts of their everyday interaction with the device, of the it uses they were hoping or were successful in developing with the tablet, and of the actions that allowed them (or not) to appropriate it (hookway ; zamani et al. ) . the empirical material of this study therefore consists of blog posts. blog posts often contain narratives of everyday life, where bloggers share their experiences and their goals with their readership. in our case, the collected blog posts are authored by ipad users who offer detailed accounts of their interaction, the problems they encountered as well as the solutions they implemented towards resolving them. the material was collected in two stages; the first spans the period between march -august , and the second between january -july . the beginning of our study ( ) and our first data collection phase (march -august ) overlapped with when the first tablets were introduced to the mass consumer market (i.e., the ipad). further details on data collection can be found in appendix . during this phase, our preliminary data analysis showed that, while users were keen to identify a fit for the tablet into their everyday, the process was rarely straightforward. instead, users exhibited a highly explorative behaviour, where they were trying different use scenarios, often incorporating the ipad into their device portfolio in novel ways. the use of gtm allowed us to identify negative disconfirmation as a fairly relevant conceptual category for our study, and where appropriation and rejection are outcomes of a trial and error process where the user tries out different things in order to identify solutions to this negative disconfirmation. in light of this, we proceeded with a second phase of data collection (january -july ), following theoretical sampling of the same type. (urquhart ) . our aim was to increase and verify the usefulness of each of the emerging categories and establish the conditions for each. as such, theoretical sampling for the second stage entailed focusing exclusively on sampling blog posts that would help us achieve meaning and content saturation (hennink et al. ) while selectively sampling with the emergent core category in mind (glaser ) . for both data collection staged, we examined the collated blog posts against our inclusion criteria. namely, each blogpost had to: a) contain a rich description of the blogger's interaction with the ipad, b) describe voluntary use of the device for both personal and professional use scenarios, c) contain a description of negative disconfirmation i.e., the user attempted to use the device in a particular way but for some reason failed to do so, and d) describe an underlying effort to overcome disconfirmation. these criteria allowed us to collect material that contains contextual and processual information, supporting us in addressing our research question. the chronological difference between the two stages means that, while during the first stage, the tablet was still a novel device, during the second stage, users had become directly or indirectly familiar with it, and had a clearer idea about their goals and expectations regarding the device as a result of others' experiences and advertising. however, the purpose of our study is not to compare and contrast expectations, where the 'starting point' of each experience would be undoubtedly critical. instead, we are only interested in examining how users attempt to overcome negative disconfirmation through trial and error behaviour and regardless of the specificities of the technical features that prompted the disconfirmation. therefore, we do not consider the different tablet generations as a critical element for our interpretations, precisely because the data reveal a consistent behaviour of trial and error across the different generations of the technology. all in all, the final data pool consists of blog posts, authored by unique english-speaking tablet users (table , appendix ). of them, bloggers are male, are based in north america (usa and canada), and the majority of the remaining blog posts are authored by europe-based bloggers. most users have managerial positions or freelance. we began our analysis with a preliminary examination of our data and moved to open, selective and theoretical coding following the glaserian paradigm (glaser and strauss ) and in line with urquhart's and fernandez's recommendations ( ) . while open coding, we coded, line by line, or at word level, and more rarely full paragraphs, often using in vivo labels and drawing from existing literature (see fig. for some examples). this was done by the first author in consultation with the second author, discussing the relevance of the used labels and the consistency of the coding. an overview of the process of data collection, analysis and interpretation can be found in table . the stage of open coding is critical as it acts as a sensitizing device for data collection and analysis, and for further examination of the existing literature (m. barrett and walsham ) . indeed, during this stage, we noticed that users were recounting some disconfirmation, most often than not, a negative one, with the tablet, following which they begun trying out different solutions in order to overcome it and address the experienced issues. in many instances, this led to further 'errors' in their interaction with the tablet. we moved to selective coding by focusing our coding around the codes that seemed to relate to the emerging categories, while identifying their variants, how they relate to each other (hekkala and urquhart ) , drawing iteratively from the literature, constantly reviewing and revising the evolving coding scheme (wiesche et al. ) . the descriptions of our codes and core categories can be found in table . in this section we present our findings, organized around our three core categories of negative disconfirmation, trial and error and outcomes, in order to illustrate how negative disconfirmation emerges, the process of trial and error behaviour, and the conditions for each of the two main outcomes of this process. inspired by other researchers (e.g., korica and molloy ; vaast and levina ) , we use vignettes. vignettes are often used for the presentation of findings as they constitute concrete examples, which are carefully selected as "illustrations and exemplars of particular concepts" (swan et al. (swan et al. , p. , and in this case serve as a way to provide a rich description of different examples of trial and error behaviour without decoupling these from their familiarisation: review of the empirical material: memoing and note taking, emergence of initial ideas (trial and error) open coding: initial coding line by line, occasionally at word level (cf. table ) selective coding: open codes organised around core/initial ideas (trial and error), grouping codes together, primarily guided by the preliminary research question (cf. table ) . reflection: review of codes and themes. the coding scheme was reviewed by the first two authors to ensure it reflects the emerging themes, that codes are mutually exclusive and that they are exhaustive (miles and huberman ) . negative disconfirmation emerged as particularly prominent. we examined theoretical saturation (not achieved) ➔ a second data collection was decided on the basis of theoretical sampling to achieve it. second data collection jan -jul new data were collected, focusing specifically on negative disconfirmation occurrences, and a) newer generations of ipads, b) volitional contexts of use, where c) the ipad is used both for professional and personal use cases by similar types of users (boundary conditions), with a view to achieve meaning and content saturation of our codes (hennink et al. ). this stage was guided by theoretical sampling (urquhart ) and helped us achieve theoretical saturation. new material was added in the main pool, and open coded line by line, or at word level. selective coding redone: integration of new selective codes, reframing of previous ones, identification of the properties and the components of the core categories (c.f., table ). reflection: the coding scheme was reviewed for consistency by the two first authors. theoretical saturation was examined (no new themes emerging and the theoretical categories were saturated as a result of coding). theoretical coding: reflective elaboration of relationships among categories (glaser ) via constant comparison, using evidence from the data, building on our memos, on the basis of glaser's coding families (glaser (table ) , and integrative diagrams (fig. ) , revisiting the literature and developing findings. "the conventional wisdom on tablets is that they're for consumption not production. you can absorb text quickly and well, for example, but writing is a chore. extract from blogpost "a week with an ipad pro" (b ) "at this point i don't think any working professionals are going to be able to go all-in on the ipad pro as a their daily driver. there are just too many walls and ceilings to bump into right now. however, for a casual user, this device could very well be all that you need." /coded at "too many walls and ceilings to bump into" extract from blogpost "ipad pro review" (b ) fig. examples of open coding trial and error schema goals "skeptical" (e.g., b , b , b ), "what always wanted the iphone to be" (e.g., b ), "primary machine broke down/replacing" (e.g., b , b , b ), "it experience (e.g., b )" users begin interacting with the ipad, having some goal in mind. this goal may be very tangible and specific (e.g., replace a pre-existing it (or not) device) or fuzzy and highly explorative (e.g., explore the potential of the tablet). as such, negative disconfirmation seems to be the result of unachievable goals, and gaps between goals and realities in use. comparing "comparing to paper" (b , b ), "comparing to books" (e.g. b , b ), "comparing to a laptop" (e.g., b , b , b ), "comparing to using it with mouse and keyboard" (e.g., b , b , b ), "none that i've tried work all that well" (b ) in setting their goals, tablet users may be influenced by reviews, advertisements, others' experiences as well as their own past experience with similar or dissimilar devices. negative disconfirmation surfaces when the user is unable to achieve their goal (e.g., they are unable to read an e-book while lying in bed), and when they compare the tablet with other it and non it devices (e.g., comparing the ipad to the kindle and physical books, and the resulting experience). it is noted that comparison is continuous: from the moment the user begins interacting with the tablet all the way to finally appropriating or rejecting it, and during their trial and error efforts. "using a smaller keyboard" (b ), "to record notes during patient interviews, both by typing and with a stylus" (b ), used only while "at a table or another flat service [surface, sic]" (b ) trial and error is what users go through in the face of negative disconfirmation in order to overcome it. they do so by adapting the device e.g., using external add ons), augmenting it by using third-party applications, and even adapting their tasks and workflows. this behaviour is influenced by the user's experience with it, and prior experiences, and this being a tentative solution suggests that a) it is one of the many possibly equivalent solutions towards overcoming negative disconfirmation, and b) it is later reviewed for its applicability and can potentially lead to further problems (errors). error non-tolerable errors: "lack of speed and accuracy" (b ), " [not] easy for me to mix and match my favourite instruments" (b ), "more fatiguing compared to pen and paper" (b ) tolerable error: "there wasn't enough power" (b ), "quasi-mobile device, but it's not recognized as one" errors denote problems stemming as a result of the tentative solutions (being not good enough or raising further problems that prohibit the user from achieving their goals). as a result, disconfirmation persists or intensifies. • there may be non-tolerable errors, where the tentative solution is not good enough or a solution does not exist. • there may be tolerable errors or no errors (the tentative solution does not impede further interaction and use). appropriating "at first, i used simplenote to sync with scriverner. eventually, i found a better solution, using scrivener, dropbox, and elements. this last solution has worked well for me since i discovered it." (b ) appropriation surfaces as the user transitions to new workflows, by adapting their tasks and their behaviour to the tablet's requirements, or equally employing tentative solutions that augment the tablet (e.g., hardware or software add ons) and produce no errors or only tolerable errors. this suggests that the user overcomes negative diconfirmation and achieves their goals, and that the user integrates the tablet into their everyday. rejecting "i gave up and borrowed laptops (one per continent) to do all of my posts, including when i was covering our keynotes at tnw conference. (…) however, in the near future at least, i will haul my laptop on any trip i go on where i'll be blogging" (b ), "i will probably never try reading another book on the ipad again: destroying one of my greatest pleasures with constant discomfort seems like a ridiculous thing to do to myself again." (b ) users reject the tablet because they cannot overcome negative disconfirmation: they continue comparing the new to the old way of completing tasks, and they either deem the tentative solutions as not good enough or the errors as non-tolerable. as a result, they often regress to their old routines. contextual conditions. we chose the particular vignettes because they are rich descriptions of negative disconfirmation, trial and error and outcomes, but also show the variety of contexts within which users tried (and possibly failed) to appropriate the ipad. through the analysis of our findings, we see that initial negative disconfirmation with the tablet surfaces in indicative different contexts of ipad use, as a result of a discrepancy between the user's expectations and goals in using the tablet and the tablet's actual performance. below, we present two vignettes to illustrate how negative disconfirmation surfaces. dale (b ) acquired an ipad with the aim to use it as his sole device while travelling for work-related purposes. he is a frequent flyer and he was motivated to use the ipad as a result of his battery capacity and light-weight format, that could allow him to be more mobile and remain productive for longer while on the go: "the two most important factors were that the ipad has a killer battery ( + hours no matter what i'm doing) and that it is slim and only . pounds. compare this to my + pound and well over one inch thick laptop that gets at best hours of use (on a -cell battery) -basically i bought the ipad in part to be used as my travel laptop replacement for these reasons" (b ). with this in mind, he expected that the ipad could serve him well while covering conferences around the world. however this was not the case. dale explains that, despite of its strong points, the ipad did not perform as expected and that it could not function as the sole device. the ipad did not fit well with his workflow because it did not allow him to complete important tasks as part of his job, which results in his negative disconfirmation and him using other devices in order to keep up with his duties: "i took notes at the dc conference on the ipad, which turned into three posts. however, and here is the main moral to this post -all these posts came at best hours after the sessions because i didn't actually post any of these stories to wordpress using the ipad. (…) so to make a long story short, i gave up and borrowed laptops (one per continent) to do all of my posts, including when i was covering our keynotes at tnw conference." (b ). gordon (b ) purchased the tablet with the aim to explore and experiment with it while recovering after his operation. contrary to dale, gordon does not have a clear goal with "face the congregation at all times" (b ), "wonderful opportunities for "social" internet surfing" (e.g., b , b ), "a screen that connects me with people" (b ), "once you get used it that, you realize how efficient you are with the lack of distraction." (b ) the outcome of appropriation as a result of the trial and behaviour is subject to the user identifying benefits in using the tablet: identifying benefits is a condition for the appropriation of the tablet. some of the benefits is the use of the device together with others, without isolating themselves from their environment, and without hindering their social interactions. identifying benefits allows users to develop their personal use cases, persevere in finding a tentative solution and evaluate overall more favourably the tablet despite initial disconfirmation. feeling restricted "too many walls and ceilings to bump into" (b ), "apple will sit and control what you can do with the advice" (b ), "the size of the device doesn't let much freedom for taking many photos" (b ), "inability to listen to a video in the background" (b ) the outcome of rejection is more likely when users feel as if the tablet restricts them in some way. missing features and functions entail that the user either will have to work around them (tentative solutions), or accept them (tolerable errors). if this is unacceptable though (a tentative solution does not exist or the error is non-tolerable), the user feels as if the tablet is designed in a way that restricts their activity, especially when compared to other devices. note: numbers in brackets denote the id number of the blog post. the complete list can be found in table in appendix regard to his tablet use. instead, he clearly notes that he was motivated to acquire it for two main reasons: the first was his prior experience with his iphone, and the second his desire to find out about the ipad's merits and potential: "i bought an ipad last week because i love my iphone so much (…) and also because i figured that, since i was going to have a lot of time on hands recovering from my surgery, it would be fun to have a cool new toy. (…)" (b ). while the goal is not as specific as in dale's case, gordon similarly compares his tablet experience to his experience with other it and non-it devices, and notes that the tablet performs less satisfactorily, in part due to its form factor: "the first day i had it, i rented a movie i have always loved, movie_ , and tried to watch it for over an hour before simply giving-up. (…) a laptop, buy [sic] the way, would have been much easier because you can adjust and hold the angle screen more easily." (b ). based on the collated narratives, we see that users acquired the tablet with the expectation to either use it within an occasionally well-defined use scenario (vignette ) or experiment with it in an attempt to explore its potential (vignette ). in both cases, negative disconfirmation ensues a comparison. first, users assess the tablet's success in helping them achieve their goals and meet their expectations, and compare how they used to complete tasks and other activities with their other it and non it artefacts to how their workflow changes by using the tablet instead. further, regardless of whether users acquired the tablet for exploitation (vignette ) or exploration (vignette ), negative disconfirmation denotes that the it artefact fails to meet the user's goals and expectations. trial and error is a sequence of attempts to bridge the gap between goals and actual experience, and while the user tries out one or more tentative solutions, in order to overcome their negative disconfirmation. the following vignettes illustrate trial and error behaviour, where ipad users try out different tentative solutions with the aim to tackle their initial negative disconfirmation. peter (b ) is a musician and music editor who has been using his macbook pro and a specialist application (mainstage) to emulate "the sounds of pro keyboards like roland rd pianos and synths when playing live". he is now interested to see if an ipad-centred setup can work equally well for live performances, and therefore replace the laptop. to do this, peter needs to use "a decent "real" midi controller keyboard", but, to his disappointment, on the one hand, "ipads have neither usb nor midi inputs", while on the other hand, such larger keyboards typically requires an external power source (negative disconfirmation). to overcome this, he turned to an adaptor for connecting a keyboard to the tablet that uses the ipad as a power source (tentative solution). while "[t] his worked well, and the ipad was able to power the keyboards for hours", peter "encountered a small glitch when [he] first plugged in [his] midi controller", when his tablet showed an error message that "there wasn't enough power" (tolerable error). however, he felt confident that this "was meant to work"; therefore he "started experimenting" (trying), and discovered that the error was due to the sequence of plugging in cables and adaptors. he next raises the issue of latency as there is a delay between him stroking a key and receiving a response from the tablet. this is however "hardly noticeable, or unnoticeable" (tolerable error) and although he does "have the occasional problem (error) [ …], resetting the ipad makes it responsive again" (tentative solution). to better control sounds and effects, peter had to map the keyboard on the ipad application. yet, while comparing the ipad-centred versus the macbook pro-centred set up, he explains that "there doesn't seem to be a way of mapping all of those useful buttons, knobs and sliders on my keyboard to do anything useful" (negative disconfirmation). after further attempts (trying), he arrives at the conclusion that this is a limitation of the application rather than of the keyboard (no solution). michael (b ), a medical student during his clinical year, has been finding the tablet both useful and versatile for his studies and patient care. while at the hospital, he needs access to medical records that are stored securely into a dedicated content management system. however, he was not able to access this system directly from his tablet (negative disconfirmation). to overcome this obstacle, he tried out using the citrix receiver, a freeware desktop-virtualization package (tentative solution). this allows him to access the centralized host and to "tap into [the] emr system" (no error). however, "[t] here are some ways where [he's] been less than impressed with ipad". michael notes that the ipad is "not a very good input tool" due to the "lack of speed and accuracy" in capturing information while talking to his patients (negative disconfirmation). he has tried "to record notes during patient interviews, both by typing and with a stylus" (tentative solution), but he doesn't consider this set up as satisfactory because, while being "too busy making sure that the […] notes [a] re accurate", he feels the tablet is a barrier between the patient and himself (non-tolerable error). in addition, he compares the tablet to a regular notepad, and deems that "[p] aper and pen is still superior in a lot of cases". harriett (b ) is a litigator and a consultant who switched from the ipad mini to the ipad pro, with the aim to see if she could "be doing more with [the] ipad" while on the go and while meeting clients. she begins her narrative by saying that the ipad mini "has never been [her] preferred device […] [for] productivity related activities", such as preparing presentations, legal briefs and the like. she considers that the larger form factor and the recently made available multitasking features have made "these types of tasks easier […] than they ever were before". however, she describes her personal experience as a "compromise" when compared to her laptop experience. she suggests that "the biggest problem with the ipad pro was it was just too darn big". she could only use it "at a table or another flat service [surface, sic]", and being almost as big as a laptop, carrying the ipad pro required some effort: "it wasn't something that could be thrown in a purse and taken on a whim […] . if i was going to go the trouble, i personally would have preferred to have my mac" (negative disconfirmation). harriett reflects meeting with clients and describes "typing directly on the glass [as] a clunky experience" (negative disconfirmation). she considered pairing an external keyboard (tentative solution), but in her opinion this would make the tablet even more similar to a laptop, where there'd be a physical barrier between her and her client, which is "the situation [she] was trying to avoid by using the ipad in the first place" (non-tolerable error). she further compares note taking on the ipad with a stylus to note taking on a notepad and describes the former as "more fatiguing compared to pen and paper due to having less friction and having to apply more pressure to control the pencil" (non-tolerable error). russ (b ) is a design professional who uses the ipad as his "primary machine", having been a windows user for years. he had been unpleasantly surprised in the beginning. due to his profession, he often has to import pictures and videos from his camera to edit them on his devices. however, he quickly realized that he cannot import these files from the camera's sd card to the ipad (negative disconfirmation). to overcome this obstacle, he tried out a camera connection kit (tentative solution), which seems to be working well (no error). however, with regard to image editing, russ notes that "the biggest issue is image resizing […] . i've found it impossible to resize an image to a specific pixel value without also having to calculate the height value too"] (error). he has attempted to find an easier way around this, but after trying out different third-party applications (tentative solution), none of which seems to work (non-tolerable error). despite that such problems require him to employ "cumbersome methods to achieve the desired results", his productivity has not decreased as a result of him using the ipad as his primary device. these vignettes ( - ) illustrate that users apply a tentative solution so as to try and overcome their initial negative disconfirmation. following this, users may be faced with further issues, namely tolerable (vignette ) or non-tolerable errors (vignettes , , ), which prompt subsequent trials, until users identify a good enough solution or consider that no solution exists. equally they may be faced with no errors (vignettes , ) at all, when the applied tentative solution is considered a good enough one. in what follows, we describe appropriation and rejection as the two major outcomes of trial and error, and we discuss the conditions for each outcome. through trial and error behaviour, users move from tentative solutions to good enough ones, that help them overcome negative disconfirmation. in such cases, these solutions entail the adaptation and modification of the technology, where the user augments the tablet with e.g., external keyboards (vignette ), specialized applications (vignettes , , ), and adapts and modifies their own habits and routines (vignettes ). in doing so, the outcome of trial and error may be the appropriation of the it artefact or its rejection. in the vignette that follows, we illustrate the outcome of appropriation and highlight its conditions. garry (b ) has been an ipad user for some time. during this time, he has moved much of his work his macbook pro and imac to the ipad. he notes however that, "there were a few things [he] needed such as a keyboard case, writing app, etc." in order to do so. in his blogpost, he takes his readers through his journey of how he chose his current set of applications, as well as how he augmented his tablet with an external keyboard that makes him feel "like typing on a macbook pro" while offering "multiple viewing angles like a laptop". he chose these solutions as a result of trial and error, like all other users, and he further explains that while he "had no problems finding those things, the real challenge was in changing [his] os x-centric mindset". this entailed "unlearn [ing] some of [his] long-time habits" and "judg [ing] ios on its own terms rather than constantly comparing it to os x." as a result, garry notes: "the ipad is a more personal experience and i tend to have it with me wherever i am in the house. (…) since i got my ipad air , i have hardly even picked up my iphone plus when i'm in the house. and the ipad has cut down the use of my imac drastically, and mostly left the desktop computer relegated to work duties. don't get me wrong, i still love my imac and iphone plus, but neither of them can compete with the ipad air for certain uses such as games, reading, comics, etc." garry recognises that the ipad can be of value to him, offering a more personal experience, and specifically for some activities such as reading and gaming. to "narrow the absorption gap" (clark , p. ) , he has opted for an external keyboard and applications, such as the kindle app, which enable him to support these practices (carroll and fidock ) . similarly to garry, russ (vignette , b ) has also appropriated the device. he has migrated to the ipad almost the entirety of his computing activity by adjusting his habits and routines as well as modifying the device itself through add-ons: "there are a few areas, such as image resizing, which it severely lacks, but there are cumbersome methods to achieve the desired results on the tablet." in russ' case however, the reason for being motivated to identify these "cumbersome methods" and to employ them on a daily basis, has been the increase of his productivity as a result of the lack of multitasking: "it's lack of side-by-side apps (i.e the traditional mac os x setup) means you actually end up focusing more on the work in hand, because there's nothing that's distracting you across the screen." going back to the definition of appropriation, the stories of garry and russ clearly illustrate that, appropriation of the tablet suggests that the users make the technology 'their own', despite their initial negative disconfirmation, specifically because they identify some benefit in using the it artefact (carroll and fidock ; clark ). russ, for example, having been disappointed that the tablet doesn't allow easy image editing, explains that it is now his preferred device because it has increased his productivity, as it offers a more focused interaction, without distractions. as users learn how to use the new it device, identifying some beneficial use is of paramount importance and is the tipping point for eventually appropriating or rejecting the device. in those cases that users are unable to identify any benefits in using the ipad, or when such benefits come with sacrifices users are unwilling to make, the outcome of the trial and error behaviour is that of rejection. rejection suggests that the tablet cannot support the user in meeting their expectations and achieving their goals. this may mean that the tablet cannot reasonably substitute a previously owned it or non-it artefact or that it doesn't improve one's workflow in some way. one's golas and their experiences with other forms of it, and even with non-it solutions (e.g., pen and paper), have an impact on how an interaction is experienced because they weigh in in how users make sense and act about technology (kendall et al. ; orlikowski and gash ) and is quite applicable when rejecting the ipad. harriett (b , vignette ) acquired the tablet in an attempt to uncover new it-centred use scenarios. without a specific use in mind, she suggests that the large form factor of the ipad was the main issue that caused her negative disconfirmation, and for failing to integrate it in her everyday workflow. its size meant further constraints in relation to portability, ergonomics and connectedness with clients, all of which were obstacles to her productivity. the tentative solutions she identified could only intensify her negative disconfirmation and lead to further errors. as such, she was unable to identify any benefit in using the device as she considered that her past practices of using e.g., a legal notepad, posed fewer restrictions. this resulted in her returning the tablet. another user, dwayne (b ), similarly to harriett, notes that the restrictions he was faced with while using the tablet, exceeded by far the benefits he was able to identify. even though he was "positively surprised by the ipads capabilities" as far as battery life goes, he was not that impressed by the fact that, as a linux user, he was unable to access itunes, which would allow him to load all he needed to the ipad. it is interesting to note however that, rejection and appropriation seem to exist along a continuum, rather than being binary outcomes of trial and error. for example, michael (b , vignette ), having been using the tablet for different usages, considers the device "invaluable", because it allows him to take stock of every possible moment while at work. but he is less content when it comes to how the tablet has been serving him during his rounds because the tablet acts like a barrier between him and his patients, which he consider as a restriction and prioritises ultimately achieving a better communication with the patients rather than his convenience ("who wants a medical student (someday physician) who focuses more on a computer than on the person?"). as such, while the ipad enables new usages and offers some benefits (e.g., studying while on the go, accessing information), it does lead to lead to ineffective practices, too, thus he "put [s] away the ipad" when being with patients. in other words, through michael's example we see that users may appropriate the device for some use scenarios (in this case, accessing health records while on the go), but reject it for others. therefore, the conditions under which trial and error leads to rejection rather than appropriation, have to do with perceived restrictions, and the extent to which these restrictions seem to balance out any potential benefits. in such instances, the perceived restrictions are the reasons for gaps between goals and reality, i.e., they prohibit users from using the tablet according to their initial goals. this is particularly clear in harriett's case who notes that, while she could augment the tablet in a way that would allow her to use it as desired, doing so would outweigh the benefits she was after (portability and flexibility). in this study, we focused on the volitional use of it in order to understand if and how users overcome initial negative disconfirmation, and why some users appropriate the it artefacts, whereas others fail or refuse to do so. drawing from grounded theory method techniques, we have identified two core categories, trial and error, and outcomes, whereby trial and error is observed following negative disconfirmation, and the iterations of trials and errors result in either appropriating or rejecting the it artefact. fig. offers an illustration of how the core categories (trial and error and outcomes) have been build up on the basis of their relationships. as the figure shows, trial and error is composed of feedback loops (iterations), where a preliminary tentative solution may lead to different types of errors (tolerable, non-tolerable errors) or, indeed, no errors. when the feedback loop 'breaks' (dashed line), trial and error leads to the outcomes of appropriating or rejection, depending on the conditions. if, through trial and error and despite the experienced errors, the user has managed to identify some benefits, the most probable outcome will be that of appropriation. alternatively, if the user perceives being restricted, and especially if the experienced errors are deemed as non-tolerable, then the most probable scenario will be that of rejection. what is interesting and begs consideration is that trial and error takes place against the backdrop of constant comparison, whereby the user is comparing the new against the old workflow, their new to their old experience, and ultimately the ipad to other artefacts, which are not necessarily it (e.g., comparison may entail comparing reading a book on the ipad versus reading an actual hardcover book). trial and error has been quite an influential concept, first appearing in the s, when behaviourists observed that consecutive efforts (trials) help overcome an obstacle or solve a problem (costall ). shirahada and hamazaki define it as "the process of continuous knowledge creation and acquisition until success is achieved" (shirahada and hamazaki , p. ). rerup and feldman draw attention to the fact that people compare outcomes to targets, and then revise their routines as necessary so that they can meet those targets (rerup and feldman fig. connecting trial and error to outcomes following negative disconfirmation findings show, as the majority of users have an expressed interest in integrating the tablet in their workflow, upon experiencing negative disconfirmation they proceed with trying out different things in order to overcome it. in doing so, they combine and recombine their available it devices and objects, in order to identify ways to 'make technology work'; when they are successful in this endeavour, they transition from the 'technology as designed' to the 'technology in use' (elbanna and linderoth ) , which echoes existing conceptualizations of appropriation (ahuja and thatcher ) . however, when they are unable to identify ways to make it work in line with their goals, then the iterations of trial and errors lead to rejection instead of appropriation. as a result, trial and error may be observed when users are faced with anomalies and try out new things to overcome problems (mcgann and lyytinen ). we consider trial and error to be a useful concept to discuss voluntary adoption of it. our findings show that, as users begin their interaction with the it artefact, they are required to update their perceptions, exploring what they can do with the it and how they can do it. previous studies have found that users may adapt themselves (i.e., changing their routines), or the it artefact (i.e., using external devices, additional applications etc.), or any combination of the two (e.g., barki et al. ; beaudry and pinsonneault ) , in their endeavour to see what actions are possible (bagayogo et al. ; sun ) . equally, it may be argued that the observed iterative attempts to identify and apply tentative solutions are nothing more a variation of the adaptation cycles, proposed by sun in his seminar paper regarding user revisions of technology (sun ) . these iterative attempts in assessing tentative solutions are in turn reminiscent of the body of work on workarounds (e.g., choudrie et al. ; ferneley and sobreperez ; koopman and hoffman ) , as they entail that users tweak in subtle and less subtle ways the it artefact, incorporating bundles of applications and other fixes (choudrie et al. ; ferneley and sobreperez ; koopman and hoffman ) . there are fewer similarities between trial and error and improvisation. trial and error is primarily based on exploitation; as our findings show, tablet users seek to implement tentative solutions based on what they know and their experiences. in contrast, existing literature suggests that when improvising users tend to be more exploratory (scheiner et al. ) . further, as improvisation is generally timesensitive (pavlou and el sawy ) , users may be able to pursue a limited number of iterations. yet, as our findings show, users may proceed at their discretion with as many trials they want or need to identify tentative solutions; it is telling that some users (peter, vignette ) pursuit several iterations of trial and error, whereas others (harriett, vignettes ) are less committed in identifying a solution. as such, echoing prior literature on trial and error, we posit that the only event that seems to stop the iterations is the identification of a good enough solution (miner et al. ; rerup and feldman ) or the belief that a solution may not exist (swann ) . we thus consider that trial and error can function as an umbrella concept, consolidating essentially the different appropriation variations, specifically for the study of volitional use of it devices. at the same time, however, trial and error does not assume a priori that appropriation will be the only possible outcome. our analysis shows that despite all users' expressed intention to integrate the tablet into their workflow (vignettes and ) , some of them were unable to find a good solution that can help them overcome their negative disconfirmation. therefore, trial and error, despite its problem-focused and solution-focused nature, and in contrast to existing appropriation variations, does not assume that there is a fit between user, task and technology, which eventually will be achieved. instead, trial and error can be an open ended process which may result to the simple abandonment of the device. this contribution is further supported by our findings if we consider that our analysis showed that appropriation and rejection exist along a continuum, whereby a user may appropriate the tablet for certain tasks, and reject it for certain others. indeed, as in michael's case (vignette ), trial and error may lead to partial appropriation (or indeed partial rejection), if the user considers that the tablet is inappropriate in some way within a particular context of use, but at the same adequate or ideal for others. this challenges the rationalist view of technology, whereby adoption, rejection and behaviours in between are mostly seen as decision situations (riemer et al. ) , and further highlights the socially constructed and co-constructed nature of technology (leonardi and barley ) , where the technology and its use evolve together (richter ). this draws attention to the conditions for each outcome. first, the essential component of appropriation is the identification of benefits (carroll and fidock ; clark ) . these benefits motivate users to persist in identifying and trying out different tentative solutions despite their negative disconfirmation. such benefits may relate to productivity, performance, and convenience among other things (dang et al. ) , all of which are directly related to the needs and wants of the individual user. in light of this, wyatt ( ) posits that individual users are more likely to adopt some kind of a technology if they consider that there are some benefits in its use, and they will only do so, if those benefits significantly outweigh any risks that use may entail (e.g., time wasted). indeed our findings clearly show that time savings, and increases in productivity are deemed simply not enough in light of other sacrifices (such as lugging around a heavy object, being unable to focus on the primary activity or to properly interact with others), then the outcome of the trial and error behaviour is that of rejection (michael and harriett, vignettes , ) , where the user discontinues its use for some or all tasks. this suggests that the tablet as an it artefact is not defined or interpreted by its functions and the included or excluded features in its design; instead, users assess it in relation to more practical terms (i.e., if, how, when it can support them) and on its potential place within their wider sociomaterial practices (kendall et al. ; riemer and johnston ) . our theory of tablet appropriation through trial and error accounts for some of the shortcomings that presently exist within the it appropriation literature. specifically, by casting a wide net around negative disconfirmation and focusing on cases of tablet users with an expressed interest to incorporate the tablet within the constellation of their pre-existing it devices, we have identified how and why trial and error may result in the appropriation or the rejection of the tablet. the use of such devices is often volitional rather than mandated, and thus, users can freely abandon the device if they consider that other options are better. in contrast, existing studies that focus on some type of appropriation appear to do so with an underlying assumption that eventually users will be able to identify a fit between the task, the technology and their work practices, through modifications, adaptations, workarounds and the likes. however, this constricted lens often excludes the possibility that such a fit may not exist, and that therefore, despite efforts to appropriate the it device, users may eventually reject it. our theory of trial and error manages to holistically account for both possibilities and explain their conditions within a volitional, individual use, which is a somewhat neglected aspect of is use, despite the ubiquity of tablets. in light of this, we consider that our trial and error theory further contributes to work done by schmitz et al. (schmitz et al. ) in relation to malleable it and adaptation behaviours in two ways. like the adaptive structuration theory they propose, our theory addresses voluntary use of it for both personal and professional use contexts. both approaches uncover rich use scenarios, iterative, exploratory and exploitive behaviours, and numerous types of adaptions and modifications that can be combined with each other toward achieving appropriation. however, we extend this work by formally incorporating the possibility for the rejection outcome. our second contribution relates with the characteristics of this trial and error process. contrary to organisational systems, such as enterprise resource planning (erp) systems, contemporary devices are quite heterogenous in that they can fulfil a number of professional and private usages. while this heterogeneity may influence appropriation in a positive way it can also have a negatively affect if the learning costs are high. relatedly, devices such as tablets, smartphones and the likes, are platform-based devices, anchored to an ecosystem of complementors, apps and users. a user may benefit by others' trial and error learning (e.g., through their blogposts and forum comments) and, equally, by the availability of hardware extensions and software apps, all of which may expand their usability options and help them identify new usages. we also note that trial and error with devices within a volitional context of use is not typically conditioned by external stakeholders who could otherwise enforce temporal, and other restrictions on it use. as a result, trial and error is an iterative behaviour that may continue unobstructed up until the point where the user feels either content or too disheartened to carry on. finally, at its core, trial and error has the comparison of task completion with or without the particular device, which is often used as a means to halt trial and error (and move towards rejection), or move towards appropriation. studying how users use it artefacts when technology falls short holds great potential for it designers, manufacturers and organisations because it opens up a window into why and how users appropriate or reject the said artefacts. at the moment, there are numerous it devices in the market, competing for consumers' attention by promising increased productivity and performance, on top of a pleasant user experience. however, not all of them prove to be as successful. as technology becomes more and more consumerised, personal devices are used for both personal and professional use scenarios (dang-pham et al. ) and such is the case with devices like the tablet. users are able to exert increased control over it adoption and use, because, within volitional contexts of use like bring-your-own-device schemes and consumerised environments, they can make their own choices (doargajudhur and dell ; hovav and putri ) . they are thus able to abandon one it artefact for another, which may offer a better workflow or may simply be more familiar with it. our findings can be used by it designers, manufacturers, practitioners as well as marketers to develop interventions and provide incentives that will facilitate appropriation over rejection. for example, designers could aim at alleviating feelings of being restricted by offering additional connectivity means or reducing existing barriers. equally, marketers and organisations could develop interventions for highlighting and maximising the identification of benefits of using a particular it artefact, especially when there are concerns in relation to a competitor and potential losses from switching. in conclusion, in a world where the success of an it artefact depends on continuous use and its appropriation and integration within one's workflows, our study offers an opportunity to better understand how individual users succeed or not in appropriating the tablet. most importantly, it lays the foundations for future studies by offering a grounded-on-the-data theory that can be applied within our other contexts. our study comes with limitations. while focusing on user behaviour post negative disconfirmation, we have not addressed its impact on a more holistic level so as to consider user experience and satisfaction. these concepts hold great significance for the design of it and can possibly influence the benefit-driven nature of the user (wyatt ) , where such benefits may be more intangible. a second limitation stems from the nature of our empirical material and our methods, both of which are greatly influenced by the study's context and particularism (davison and martinsons ) . our analysis builds on blog posts; as such, the pool of our users represent an intersection of tablet users and bloggers. in addition, these users are authored by mostly male, north america-based ipad users, who hold upper-level managerial positions or are freelancers. this means that our findings are specific to the boundary conditions of this particular demographic. to an extent, this group may be considered homogeneous. however, this is far from the truth. several studies to date have shown that cultural values and national cultures play an important role in how people choose and make use of it, from using online social networks to smartphone devices and applications (e.g., chu et al. ; george et al. ; gupta et al. a ). lastly, as evident from our findings, users have at their disposal a number of it devices, and can thus afford to use either one of them, depending on their desire and preference. therefore, our findings should be treated with caution, since they reflect chiefly interactions of generally affluent professionals who afford to experiment or 'play around' with it devices that are fairly expensive. in this study, we have developed a theory of trial and error when users respond to moments of negative disconfirmation. using gtm, we illustrated the trial and error process that users go through when exploring whether and how a new it artefact fits within their personal and professional lives against the context of a larger portfolio of multiple it artefacts. we have developed our theory from the ground up, where trial and error emerged as a process directly from the data. we consider that the first step should be the validation of our theory across other contexts, including different it devices and different users. in our study we have focused within the volitional context of use. however, given that tablets are actively being used within organisations and issued as corporate devices, it would be interesting to explore how the findings from this research may apply when the users are the employees of an organization and it use is mandated; organizational culture would be an interesting concept to explore (gupta et al. b; scheibe and gupta ) . in addition, it would be interesting to investigate how trial and error may unfold when users are less affluent, and restricted to use one particular device only. in addition, because of the importance of cultural values in it use, future studies should look into underlying differences across nationalities and/or ethnicities, focusing on the role of national culture, as a potential explanatory driver for appropriation and rejection and further theorise around our research questions. equally, we consider that an obvious future step would be a different type of generalisation attempt: rather than generalising to a different population and gauging differences at the basis of cultural values, one could examine whether the theory of trial and error for appropriation is replicable and remains valid for other types of it devices. this would be of additional interest for the post covid- world, where workers are already moving to working-from-home arrangements, and where the choice of it artefacts may be made under their own volitional control and at a time when our dependence on technology has been made abundantly clear both for remaining professionally active and socially connected (seetharaman et al. ) . early to adopt and early to discontinue: the impact of self-perceived and actual it knowledge on technology use behaviors of end users moving beyond intentions and toward the theory of trying: effects of work environment and gender on post-adoption information technology use the impact of design debugging on new product development speed: the significance of improvisational and trial-and-error learning theory of workarounds enacting computer workaround practices within a medication dispensing system institutionalized computer workaround practices in a mediterranean country: an examination of two organizations enhanced use of it: a new perspective on post-adoption information system use-related activity: an expanded behavioral conceptualization of individuallevel information system use five-factor model personality traits as predictors of perceived and actual usage of technology technological appropriations as workarounds: integrating electronic health records and adaptive structuration theory research electronic trading and work transformation in the london insurance market appropriation of information technology: a requisite for improved individual performance understanding user responses to information technology: a coping model of user adaptation understanding changes in belief and attitude toward information technology usage: a theoretical model and longitudinal test understhanding information systems continuance: an expectation confirmation model blogging platform posterous to shut down on april th enacting integrated information technology: a human agency perspective cascading feedback: a longitudinal study of a feedback ecosystem for telemonitoring patients with chronic disease reconceptualizing system usage: an approach and empirical test beyond resistance to technology appropriation constructing grounded theory. a practical guide through qualitative analysis strategic relevance of organizational virtues enabled by information technology in organizational innovation understanding individual user resistance and workarounds of enterprise social networks: the case of service ltd a systematic review on crosscultural information systems research: evidence from the last decade notes on improvisation and time in organizations. accounting, management and information technologies the labyrinths of information: challenging the wisdom of systems anglo-american innovation how lloyd morgan's canon backfired have your cake and eat it too? simultaneously pursuing the knowledge-sharing benefits of agile and traditional development approaches examining the impacts of mental workload and task-technology fit on user acceptance of the social media search system investigating the diffusion of it consumerization in the workplace: a case study using social network analysis user acceptance of computer technology: a comparison of two theoretical models context is king! considering particularism in research design and reporting understanding fit and appropriation effects in group support systems via meta-analysis capturing the complexity in advanced technology use: adaptive structuration theory impact of byod on organizational commitment: an empirical investigation the validity of the improvisation argument in the implementation of rigid technology: the case of erp systems the formation of technology mental models: the case of voluntary use of technology in organizational setting user adaptation and is success: an empirical investigation among french workers. international conference of information systems eureka moments in the works of claudio ciborra resist, comply or workaround? an examination of different facets of user engagement with information systems the effects of communication media and culture on deception detection accuracy theoretical sensitivity: advances in the methodology of grounded theory the discovery of grounded theory: strategies for qualitative research the effects of national cultural values on individuals' intention to participate in peer-to-peer sharing economy relationships between it department culture and agile software development practices: an empirical investigation grandma's new tablet -the role of mobile devices in trying to innovate in it everyday power struggles: living in an iois project code saturation versus meaning saturation: how many interviews are enough? qualitative health research human documents research: from the diary to the blog this is my device! why should i follow your rules? employees' compliance with byod security policy a comprehensive conceptualization of post-adoptive behaviors associated with information technology enabled work systems understanding technology as situated practice: everyday use of voice user interfaces among diverse groups of users in urban india investigating mobile wireless technology adoption: an extension of the technology acceptance model work-arounds, make-work, and kludges making sense of professional identities: stories of medical professionals and new technologies understanding mandatory is use behavior: how outcome expectations affect conative is use identifying it user mindsets: acceptance. resistance and ambivalence email adaptation for conflict handling: a case study of cross-border interorganisational partnership in east asia what's under construction here? social action, materiality, and power in constructivist studies of technology and organizing do electronic health records affect quality of care? evidence from the hitech act innovating with technology in team contexts: a trait activation theory perspective designing for appropriation: a diy kit as an educator's tool in special education schools the improvisation effect: a case study of user improvisation and its effects on information system evolution qualitative data analysis organizational improvisation and learning: a field study a paradox of progressive saturation: the changing nature of improvisation over time in a systems development project the problem with workarounds is that they work: the persistence of resource shortages improvising organizational transformation over time: a situated change perspective technological frames: making sense of information technology in organizations user responses to new system implementation: a bricolage perspective. international conference information systems (icis the "third hand": it-enabled competitive advantage in turbulence through improvisational capabilities understanding fire fighting in new product development routines as a source of change in organizational schemata: the role of trial-and-error learning corporate social networking sites-modes of use and appropriation through co-evolution australasian conference on information systems acis . th australasian conference on information systems acis what is it in use and why does it matter for is design? eliciting the anatomy of technology appropriation processes: a case study in enterprise social media on the role of information overload in information systems (is) success: empirical evidence from decision support systems balancing fluid and cemented routines in a digital workplace the effect of socialization via computer-mediated communication on the relationship between organizational culture and organizational creativity organisational and individual unlearning in identification and evaluation of technologies capturing the complexity of malleable it use: adaptive structuration theory for individuals being (more) human in a digitized world trial and error mindset of r&d personnel and its relationship to organizational creative climate tumblr's porn ban could be its downfall-after all, it happened to livejournal. the verge understanding user revisions when using information system features: adaptive system use and triggers the object of knowledge: the role of objects in biomedical innovation what happens when learning takes place? interchange how and why trust matters in post-adoptive usage: the mediating roles of internal and external self-efficacy grounded theory for qualitative research. a practical guide using grounded theory method in information systems: the researcher as blank slate and other myths multiple faces of codification: organizational redesign in an it organization a theoretical extension of the technology acceptance model: four longitudinal field studies user acceptance of information technology: towards a unified view extending the two-stage information systems continuance model: incorporating utaut predictors and the role of context technological embeddedness and organizational change grounded theory methodology in information systems research understanding user adaptation toward a new it system in organizations: a social network perspective non-users also matter: the construction of users and non-users of the internet accommodating practices during episodes of disillusionment with mobile it publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations we collected the empirical material in two stages, between march -august , and january -july during the first stage, we collected blogposts, authored by unique bloggers. during the second stage, we collected in addition blogposts, authored by unique bloggers. the complete casebook of the study is shown in table (appendix ).for both stages, our search strategy entailed initiating a google search at first, using the following keywords: "experience" and "ipad" and "blog".we initially conducted this research in a nondiscriminatory manner, in order to get a preliminary idea about the themes bloggers tend to discuss. we then focused specifically on the main blogging platforms, i.e., wordpress. com, medium.com, blogger.com, tumblr.com, posterous.com (now defunct). within these platforms, we used the search functionality as well as the hashtag or the tag functionality to identify additional relevant posts (snowball sampling).the collated blog posts were then examined against our inclusion and exclusion criteria. we excluded any blogpost that could be seen as being a technical review, as affiliated directly or indirectly with apple inc. or as containing indications that the blogpost has been endorsed in some way or sponsored by any of the manufacturers/developers of any of the products and/or services mentioned in the blog.relatedly, to be included, each blogpost had to: a) contain a rich description of the blogger's interaction with the tablet, b) describe voluntary use of the device within both professional and personal use scenarios, c) contain a description of negative disconfirmation i.e., the user attempts to use the device in a particular way but failing to do so for one or more reasons, and d) describe an underlying effort to overcome disconfirmation. these criteria allowed us to collect material that contained contextual and processual information, supporting us in addressing our research question.specifically for the second stage of data collection (january -july ), which was driven by theoretical sampling, we purposefully sampled blogposts with the aim to enrich the meaning of our existing emerging codes, rather than expand the reach of our evolving theory. in other words, we aimed at identifying additional cases, where blog authors were discussing the same concepts, so that we could densify our theory, by verifying the usefulness of the core categories and establishing the core conditions for each.considering the nature of our empirical material, it is critical to note that some of the blogging platforms are now defunct, and several of the blogs are not online anymore. for example, posterous.com, once a very popular blogging platform, shut down in early (bishop ) . with it, lots of our empirical material vanished. similarly, when in late , tumblr.com announced the ban of a certain type of content, millions of posts vanished from the platform, which led to a mass migration of users onto other platforms (stephen key: cord- -ztgrxucb authors: ben-michael, eli; feller, avi; stuart, elizabeth a. title: a trial emulation approach for policy evaluations with group-level longitudinal data date: - - journal: nan doi: nan sha: doc_id: cord_uid: ztgrxucb to limit the spread of the novel coronavirus, governments across the world implemented extraordinary physical distancing policies, such as stay-at-home orders, and numerous studies aim to estimate their effects. many statistical and econometric methods, such as difference-in-differences, leverage repeated measurements and variation in timing to estimate policy effects, including in the covid- context. while these methods are less common in epidemiology, epidemiologic researchers are well accustomed to handling similar complexities in studies of individual-level interventions."target trial emulation"emphasizes the need to carefully design a non-experimental study in terms of inclusion and exclusion criteria, covariates, exposure definition, and outcome measurement -- and the timing of those variables. we argue that policy evaluations using group-level longitudinal ("panel") data need to take a similar careful approach to study design, which we refer to as"policy trial emulation."this is especially important when intervention timing varies across jurisdictions; the main idea is to construct target trials separately for each"treatment cohort"(states that implement the policy at the same time) and then aggregate. we present a stylized analysis of the impact of state-level stay-at-home orders on total coronavirus cases. we argue that estimates from panel methods -- with the right data and careful modeling and diagnostics -- can help add to our understanding of many policies, though doing so is often challenging. to limit the spread of the novel coronavirus, governments across the world implemented extraordinary "non-pharmaceutical interventions," such as closing non-essential businesses and imposing quarantines. the specific policies -and the decisions to lift them -varied widely, with particularly dramatic variation within the united states. learning about the impact of these policies is important both to inform future policy decisions and to construct accurate forecasts of the pandemic. there are well-established epidemiologic methods for estimating the impact of an intervention that occurs in a single location and at a single time point, dating back to john snow and the cholera epidemic in london. often known as difference-in-differences, the basic approach constructs counterfactual outcomes using group-level longitudinal data from the pre-and post-policy change time periods and from localities (e.g., states) that did and didn't implement the policy. variants of this approach are known as panel data methods and include event studies and comparative interrupted time series models. there is no clear consensus in epidemiology, however, on how to proceed when many localities implement a policy over time, sometimes known as staggered adoption. fortunately, epidemiologic researchers are well accustomed to handling similar complexities in studies of individual-level interventions, with decades of research on strong non-experimental study designs, including methods that handle confounding and variation in timing of treatment across individuals. "target trial emulation" emphasizes the need to carefully design a non-experimental study in terms of inclusion and exclusion criteria, covariates, exposure definition, and outcome measurementand the timing of all of those variables. in this paper, we argue that policy evaluations using panel data need to take a similarly careful approach to study design, which we refer to as "policy trial emulation." the main idea is to construct target trials separately for each "treatment cohort" (states that implement the policy at the same time) and then aggregate. we illustrate this approach by presenting a stylized analysis of the impact of state-level stay-at-home orders on total coronavirus cases. we believe this new connection to trial emulation is an important conceptual advance, though the underlying statistical methods we discuss are well-established, and many of these points have been made in other contexts. we argue that estimates from panel methods -with the right data and careful modeling and diagnostics -can help add to our understanding of policy impacts. the underlying assumptions, however, are often strong and the application to covid- anti-contagion policies is particularly challenging. we now describe key steps in conducting "policy trial emulation," which are necessary and obvious when designing a randomized trial and are becoming more common in the design of nonexperimental studies. we argue that these steps are just as important when evaluating policies with aggregate longitudinal panel data. we illustrate the key idea with a stylized policy evaluation: measuring the impact of us states adopting a "shelter in place" or "stay-at-home" policy on covid- case counts. these orders urge or require citizens to remain at home except to conduct essential business, e.g., grocery shopping or exercise; we use nyt tracker to define policy enactment dates and obtain daily case counts. no irb approval was needed for this use of publicly available aggregate data. first, we must have a consistent definition of the exposure. specifically, there is only one form of treatment, and the outcome we see under a particular policy environment is equal to the potential outcome under that policy environment. for stay-at-home policies, different states enacted different requirements that broadly fall under this header, and the nyt definition is just one. this introduces a trade-off. we could consider multiple types of treatment separately, such as closing schools versus closing non-essential businesses. however, this greatly expands the dimensionality of the specific exposure under consideration. instead we consider "packaging treatments" together, allowing for some variation in the specific implementation of the policy across units. as a result, the estimated effect averages over different policy-specific effects within the data, which may be less interpretable and may violate the consistency assumption. second, there is growing evidence that stay-at-home orders had only modest impacts on individual behavior -in many states, individuals reduced their mobility even in the absence of official policy changes. thus, we focus here on emulating an intent-to-treat (itt) analysis, where individuals are randomized to treatment conditions, but the amount of treatment actually received (e.g., the dose) may differ across people. the itt is often relevant for examining whether the policy is effective overall, regardless of specific implementation. if we had data on, e.g., state-wide implementation of the policy or the level of adherence as determined by mobility measures, we could conduct an analysis analogous to a per protocol effect, e.g., estimating the effect of "full" policy implementation (i.e., all individuals following the stay at home guidelines). the trial emulation framework helps clarify the additional assumptions necessary for conducting a per protocol analysis in this context. finally, an important complication is reasoning about interactions between units, e.g. people with covid- traveling across state lines and spreading infection. the trial emulation framework makes clear that we must pay close attention to this when defining our target trials: nearly all standard tools for policy evaluation assume no interference between units, that is, a state's outcome only depends on that state's intervention. violations of this assumption, sometimes known as spillovers or contagion, complicate the definitions of units and exposures and can lead to bias. understanding and explicitly modeling these violations is paramount when studying policies to control infectious diseases. for example, holtz et al. use travel and social network information to account for spillovers across states. we refer interested readers to the large literature on observational causal inference with interference in general and on panel data methods in particular. after defining units and exposures, the next step in the trial emulation framework is to define the estimand of interest. as discussed above, we focus on the itt for treated states. formally, let !" be an indicator that state i has a stay-at-home order at time t; and let !" be the corresponding observed outcome. we can express the causal quantity of interest via potential outcomes: !" ( !" = ) is the outcome if the stay-at-home order is enacted, and !" ( !" = ) is the outcome if the order is not enacted. the causal contrast of interest is then a difference between these potential outcomes, !" ( ) − !" ( ), averaged over the states that implemented the policy and over post-treatment time periods. we also focus on the impact of "turning on" these policies, but of course states also turn them "off." just as in the individual exposure case, modeling individuals or locations turning exposures on and off is complex. if that seems ambitious in a trial setting it is often even more ambitious in a non-experimental context! the next step in trial emulation is to define "time zero", i.e., the point in time when individuals or states would have been randomized to treatment conditions. this is crucial for clearly distinguishing baseline (pre-treatment) measures from outcomes (post-treatment): inappropriately conditioning on or selecting on post-treatment variables can cause as much bias (including immortal time bias) as can confounding. . in standard target trial emulation, time zero is often defined based on when individuals meet the specified eligibility criteria and is applied equally to both treated (exposed) and control (unexposed) units. in policy trial emulation, states are often "eligible" to implement a policy at any point, though this can also occur in standard target trial emulation. for treated states, we typically use the date the policy is enacted as time zero; for comparison states, however, analysts essentially need to identify the moment in time that a state "could have" implemented the policy but did not. for states that enact stay-at-home orders on march ; we similarly set march as "time zero" for the comparison states. in the covid setting, however, we might instead want to measure time since the start of the pandemic in a specific location ---given the sudden emergence of the pandemic, case counts are essentially undefined before this time. this presentation is in line with many of the epidemiologic models. we refer to this as case time and, as an illustration, index time by the number of days since the th confirmed case. calendar dates with fewer than cases and case times after april th, are unshaded. the choice of time zero will depend on the context. in the covid setting, case time is more consistent with models of infectious disease growth but is also more sensitive to measurement error. calendar time is more natural for accounting for "common shocks" to states, such as changes in federal policy, which occur on a specific date. moreover, for some specific questions, such as impacts on employment outcomes, this distinction might not be relevant. for ease of exposition, we focus on calendar time in the main text and give analogous results in case time in the appendix. the next step in policy trial emulation is to clearly define the outcomes, both the measures themselves and their timing ( in the notation above). in a typical trial, the outcomes might be something like "mortality within months." in our covid case study we focus on two different outcome measures: (a) the (log) number of cases, and (b) the log-ratio of case counts from the previous day. the first is a measure of the cumulative effect, while the latter is a measure of the day-by-day changes in growth. we focus on log transformed data because exponential disease growth can result in different pre-intervention trends on the raw outcome scale; we further discuss the risk of pre-trends below, and present results for raw case counts and case growth in the appendix. data quality is also a key concern. care needs to be taken to select outcomes that can be measured accurately; in particular, differential changes in the testing regime across states over time can lead to the illusion of an effect. finally, we focus on outcomes that are measured at the aggregate (e.g., state) level, which is natural given the outcomes we examine. however, it is sometimes more reasonable to consider targeting a cluster-(rather than individual-) randomized trial in which treatment occurs at some aggregate level but outcome data are measured at the individual level data. the same trial emulation principles apply, and hierarchical modeling can be used to account for the multilevel structure. we now describe a single target trial, and then turn to describing a sequence of nested target trials, which more fully captures the staggered adoption setting. here the goal is to estimate the impact for a single cohort of states that enact a stay-at-home order at the same time. for now, we focus on the five states that enacted the policy on march , : connecticut, louisiana, ohio, oregon, and washington. in a randomized trial, researchers have the "luxury" of knowing that the exposed and unexposed units are similar on all baseline characteristics in expectation. this is not the case in nonexperimental studies such as policy evaluations. thus, a key step in policy trial emulation is carefully selecting comparison units. we discuss statistical approaches for adjusting for differences in the next section. an important consideration in selecting comparison units is the length of follow up -once a comparison state enacts the treatment, it is no longer a useful comparison without strong modeling assumptions. only days passed between when the first and last states enacted stay-at-home orders, and if we compare the march cohort to late-adopting states, we can observe effects for at most days. in general, across cohorts, the longer the follow up, the fewer available comparison states. * due to the lag between virus exposure, symptoms, and testing, we expect stay-at-home-orders to have delayed or gradual effects on the number of confirmed cases and case growth. how we define the start of the "outcome" time period (to then allow for different patterns of effects) is intimately related to the question of defining time zero, discussed above. therefore, we compare the treated cohort to the eight "never treated" states, allowing for estimates of longer-term effects. † in principle, we could use "not yet treated" states in the comparison group, dropping states from the comparison group at the time they enact the policy. however, the set of "not yet treated" states will change throughout the follow up period. it may then be difficult to assess whether changes in * the choice of time scale is especially important if comparing with late adopters, since a state might be a valid comparison in calendar time but not case time: ohio's stay-at-home order was enacted after california in calendar time but before california in case time. † these are arkansas, north dakota, south dakota, iowa, nebraska, oklahoma, wyoming, and utah. effects are merely due to the changing composition of the comparison states. additionally, for each set of comparison states at each follow up period we will need to perform the diagnostic checks we describe below, potentially leading to an unwieldy number of diagnostics. once we have specified the target trial, the final stage is estimating the treatment effects and evaluating diagnostics for underlying assumptions. while similar to standard trial emulation contexts in many ways, there are particular nuances and complications in the policy trial emulation setting given the longitudinal time-series nature of the data and the relatively small number of units (e.g., states). we illustrate this setup with simple estimators, especially the canonical "difference-in-differences" estimator; we discuss alternative estimators below. against shared "shocks" between the two cohorts -e.g., changes in national policy or testingit does not adjust for any differences in pre-intervention cases. difference-in-differences combines these two approaches. first take the pre/post estimate of a decrease in the log growth rate by . in the march cohort and compare it to a pre/post change in the never treated states, a decrease of . . taking the difference of these differences (hence "difference-in-differences") yields an estimated reduction of the log growth rate by . , or an percentage point decrease in the growth rate (table ) . formally, this estimator is: where &$ ---and %$ ---denote the pre-and post-treatment average outcomes for cohort , and = ∞ denotes the never treated cohort. the key to the differences-in-differences framework is a parallel counterfactual trends assumption: loosely, in the absence of any treatment, the trends for the treated cohort would be the same as the trends for the never-treated states, on average. this assumption is inherently dependent on the outcome scale: if trends in log cases are equal, then trends in "raw" case numbers cannot also be equal. this assumption would be violated if: • anticipation: behavior changes before the state-wide shutdown, since pre-time zero measures would no longer truly be "pre-treatment." in the case of stay-at-home orders, there is strong evidence of such anticipatory behavior. finally, this approach relies entirely on outcome modeling and therefore differs from many common methods in epidemiology that instead model treatment, especially inverse probability of treatment weighting (iptw). recent "doubly robust" implementations of difference-indifferences also incorporate iptw and therefore rest on different assumptions than outcome modeling alone, including the positivity assumption that all units have a non-zero probability in being in each of the treatment conditions. standard difference-in-difference models avoid positivity by instead relying on a parametric outcome model that potentially extrapolates across groups. tool: it estimates the effect averaged over the entire post-treatment period. by combining various x did estimators we can estimate how effects phase in after march . first, we pick a common reference date, often the time period immediately preceding treatment (here, march ) . then for every other time period we estimate the x did relative to that date. concretely, to estimate the effect k periods before/after treatment we compute the x estimator: where ($ ---is the average for cohort , periods before/after treatment. figure shows these estimates for the march cohort, sometimes known as event study plots, with uncertainty quantified via a leave-one-unit-out jackknife. finally, for ≥ we estimate a different treatment effect for each period succeeding treatment, without imposing any assumptions on how we expect the treatment effects to phase in or out. from figure we see that in this single target trial there is insufficient precision to differentiate the effects from zero, let alone to distinguish a trend. we now estimate the overall average impact by repeating the trial emulation approach for all states that eventually adopt a stay-at-home order. as above, the first step is to divide treated states into cohorts based on adoption date. for each cohort, we then emulate a single target trial, selecting the same eight never-treated states as comparisons for every target trial. finally, we aggregate results across these target trials. these are nested target trials in the sense that each target trial can have a different starting point and length of follow up. this approach is sometimes known as stacking or event-by-event analysis in the econometrics literature. the specific approach we implement here is equivalent to that in abraham and sun ( ) and callaway and sant'anna ( ) without any covariates. in each single target trial, we estimate a series of two-period difference-in-differences estimates for that cohort as in the previous section. there are many ways to aggregate these estimates across cohorts. here we aggregate based on days since treatment (sometimes called "event time"): where is the total number of cohorts, $ is the number of treated units in cohort , and is the total number of treated states. we refer to these as nested estimates. figure shows the estimates from this approach, both for the effect on log case growth and on log cases. as with the single target trial, estimates to the right of zero are the treatment effects of interest, and estimates to the left of zero are "placebo estimates" of the impact of the treatment prior to the treatment itself. for the left panel (log case growth), the placebo estimates for the ten days before a state enacts a stay at home order are precisely estimated near zero; however, placebo effects prior to this are highly variable and show a downward trend over time. this suggests caution in interpreting the negative estimates to the right of zero. for the right panel (log cases), the placebo estimates are even more starkly different from zero, suggesting that this would not be an appropriate analysis and that the estimated effects are likely merely a reflection of these differential trends. epidemiologists regularly confront settings where multiple jurisdictions adopt a policy over time and the data available are aggregate longitudinal data on those and other jurisdictions. policy trial emulation provides a principled framework for estimating causal effects in this setting. the specific approach we advocate is not new; there is a growing literature in statistics and econometrics proposing robust methods for panel data. here we show that these ideas fit naturally into the trial emulation framework, especially the notion of aggregating across multiple target trials. as a result, we can leverage recent methodological advances to enable more sophisticated estimation that allows for looser assumptions (e.g., parallel trends conditioned on covariates), including inverse propensity score weighting, doubly robust estimation, synthetic controls, and matching. we could also impose stronger modeling assumptions on the time series, e.g., a linear trend, such as in comparative interrupted time series. one approach we caution against is the common practice of using regression to fit a longitudinal model to all the data, with fixed effects for state and time. as with individual data with timevarying treatments and confounders, naive regression models can mask important issues. in particular, it has been shown that the coefficient in this pooled model estimates a weighted average over all possible x difference-in-differences estimates, where the weights can be negative. moreover, some of these estimates are not in the spirit of trial emulation, e.g., by comparing two states that are entirely post treatment. in practice, these complications mean that the sign of the estimated effect can be flipped relative to the nested estimate. while some approaches, such as "event study models," are less susceptible to this critique, we believe that the trial emulation framework we outline here is more transparent and less prone to error, partly by being explicit about all the causal contrasts. the issues that we highlight are just some of many major challenges in estimating policy effects more generally, including: differences in the characteristics of states that do and don't implement the covid- pandemic adds additional complexities to these policy evaluations. for example, the disease transmission process, and the up to two-week lag in the time from exposure to symptoms, makes it difficult to identify the precise timing of expected effects. data on outcomes of interest are also limited or problematic; for example, case rates need to be considered within the context of the extent of testing. finally, methods that do not account for spillovers and contagion are likely to be biased in this setting, and so properly addressing interference is a key methodological and practical concern. these issues -and the strong underlying assumptions -suggest caution in using difference-indifference methods for estimating impacts of covid- physical distancing policies. at the same time, the policy trial emulation framework suggests a rubric by which we can assess the quality of evidence presented in these studies. we anticipate that high-quality panel data methods will add to our understanding of these policies, especially when considered alongside other sources of evidence. see how all states are reopening designing difference in difference studies: best practices for public health policy research confounding and regression adjustment in difference-in-differences electronic medical records can be used to emulate target trials of sustained treatment strategies avoidable flaws in observational analyses: an application to statins and cancer per-protocol analyses of pragmatic trials available at using difference-in-differences to identify causal effects of covid- policies. unpublished covid- ) data in the united states the consistency statement in causal inference: a definition or an assumption? epidemiology does water kill? a call for less casual causal inferences observational data for comparative effectiveness research: an emulation of randomised trials of statins and primary prevention of coronary heart disease dependent happenings: a recent methodological review interdependence and the cost of uncoordinated responses to covid- . working paper vaccines, contagion, and social networks problem of immortal time bias in cohort studies: example using statins for preventing progression of diabetes policy evaluation in presence of interferences: a spatial multilevel did approach, crei università degli studi roma tre. working paper estimating the number of infections and the impact of non-pharmaceutical interventions on covid- in european countries. imperial college covid- response team the design of clustered observational studies in education mostly harmless econometrics: an empiricist's companion. economics books difference-in-differences with variation in treatment timing estimating dynamic treatment effects in event studies with heterogeneous treatment effects an honest approach to parallel trends the effect of minimum wages on low-wage jobs specifying a target trial prevents immortal time bias and other self-inflicted injuries in observational analyses synthetic difference in differences matching and regression to the mean in difference-in-differences analysis health policy data science lab. difference-in-differences evaluating methods to estimate the effect of state laws on firearm deaths: a simulation study correcting under-reported covid- case numbers: estimating the true scale of the pandemic appendix figure : nested estimates of the impact of statewide stay-at-home orders on log cases and log case growth, in case time. standard errors estimated with the jackknife. key: cord- - nwpic d authors: rennie, katherine j.; o’hara, james; rousseau, nikki; stocken, deborah; howel, denise; ternent, laura; drinnan, mike; bray, alison; rooshenas, leila; hamilton, david w.; steel, alison; fouweather, tony; hynes, ann-marie; holstein, eva-maria; oluboyede, yemi; abouhajar, alaa; wilson, janet a.; carrie, sean title: nasal airway obstruction study (nairos): a phase iii, open-label, mixed-methods, multicentre randomised controlled trial of septoplasty versus medical management of a septal deviation with nasal obstruction date: - - journal: trials doi: . /s - - - sha: doc_id: cord_uid: nwpic d background: septoplasty (surgery to straighten a deviation in the nasal septum) is a frequently performed operation worldwide, with approximately , performed annually in the us and , in the uk. most septoplasties aim to improve diurnal and nocturnal nasal obstruction. the evidence base for septoplasty clinical effectiveness is hitherto very limited. aims: to establish, and inform guidance for, the best management strategy for individuals with nasal obstruction associated with a deviated septum. methods/design: a multicentre, mixed-methods, open label, randomised controlled trial of septoplasty versus medical management for adults with a deviated septum and a reduced nasal airway. eligible patients will have septal deflection visible at nasendoscopy and a nasal symptom score ≥ on the nose questionnaire. surgical treatment comprises septoplasty with or without reduction of the inferior nasal turbinate on the anatomically wider side of the nose. medical management comprises a nasal saline spray followed by a fluorinated steroid spray daily for six months. the recruitment target is patients, recruited from up to sites across scotland, england and wales. randomisation will be on a : basis, stratified by gender and severity (nose score). participants will be followed up for months post randomisation. the primary outcome measure is the total snot- score at months. clinical and economic outcomes will be modelled against baseline severity (nose scale) to inform clinical decision-making. the study includes a recruitment enhancement process, and an economic evaluation. discussion: the nairos trial will evaluate the clinical effectiveness and cost-effectiveness of septoplasty versus medical management for adults with a deviated septum and symptoms of nasal blockage. identifying those individuals most likely to benefit from surgery should enable more efficient and effective clinical decision-making, and avoid unnecessary operations where there is low likelihood of patient benefit. trial registration: eudract: – - , isrctn: . registered on march . (continued from previous page) discussion: the nairos trial will evaluate the clinical effectiveness and cost-effectiveness of septoplasty versus medical management for adults with a deviated septum and symptoms of nasal blockage. identifying those individuals most likely to benefit from surgery should enable more efficient and effective clinical decision-making, and avoid unnecessary operations where there is low likelihood of patient benefit. keywords: nasal septum, nasal obstruction, septoplasty, turbinates, mometasone furoate, clinical trial, cost-effectiveness, process evaluation background septoplasty is surgery to straighten the nasal partition between the two nostrils (the septum). septoplasty is a commonly conducted operation worldwide, with approximately , operations performed annually in the us and approximately , in the united kingdom (uk) [ , ] . most of these are carried out for nasal blockage and associated symptoms such as a snoring and sleep disturbance. nasal blockage is one of the commonest complaints presenting to otolaryngologists. however, the causes may be multiple, and several may be co-existent. septal deviation or lesions in the nasal passages, such as nasal polyps or enlarged adenoids or turbinates, may cause a 'fixed' sensation of blockage. 'fluctuating' blockage symptoms may be caused by inflammatory conditions of the nasal epithelium such as infective or allergic rhinitis. in addition, the 'nasal cycle', a spontaneous physiological congestion and decongestion of the nasal cavity, compounds the challenge in characterising and assessing nasal patency [ ] . the impact of the 'nasal cycle' can be mitigated by measuring nasal airflow following therapeutic nasal decongestion [ ] . ideally, the septum runs down the centre of the nose. if it is not straight, perhaps because of injury or a developmental anomaly, it may narrow one or both sides of the nose and obstruct airflow. a perfectly straight nasal septum in adults is rare and some degree of deviation is an accepted norm. however, in instances where there are symptoms of nasal obstruction and a concomitant deviation of the septum, patients may be offered the septoplasty operation. on the sidewalls of the nose are 'turbinates', tissue structures which are rich in blood vessels and glands. often when the septum narrows one side of the nose, it creates a larger space on the other side, into which the turbinate on that side expands. medical management using topical nasal steroid sprays decongests the nasal lining and may lead to improvement in the symptoms of nasal blockage. however, such treatments are required on a daily, ongoing basis and in practice may not be successful. in addition, side effects of nasal dryness, irritation and bleeding may impact on treatment satisfaction and compliance. when surgery to straighten the septum is carried out, some surgeons also reduce the contralateral turbinate tissue. potential complications of septoplasty include septal perforation, septal adhesions and bleeding [ ] . post-operative pain is common although this is reduced if sutures rather than nasal packing are used [ ] [ ] [ ] . patients typically are advised to take several days off work or usual activities after the operation. septoplasty has no defined selection criteria, particularly in patients whose principal symptoms are sleep related, and clinical practice varies in different centres. the mode of action of septoplasty in sleep-related breathing disorders is not fully understood [ ] [ ] [ ] . the effectiveness of septoplasty with or without turbinate surgery remains unclear and there is a lack of high-quality evidence of its benefit in the literature [ , ] . not all patients improve with surgery. estimates of persistent septal deviation following a septoplasty procedure range from less than % [ ] to % [ ] . where septoplasty fails and further surgery becomes necessary, revision rates are reported to be high [ ] . there is also a lack of robust evidence about the additional benefit of turbinate surgery [ ] . one study showed reduced revision rates for septoplasty when the turbinate tissue is reduced [ ] ; other studies report no added long-term benefit from turbinate reduction [ ] [ ] [ ] . currently, most septal surgery is based on subjective, unstandardised clinical impressions of the contribution of the nasal septum to patients' symptoms. there is also no good comparative evidence regarding alternatives to septal surgery; nor about who might most benefit, to inform patients' and doctors' shared surgical decisionmaking [ ] . whilst it is recognised that that the evidence base for septoplasty is ambiguous [ ] , it is important to take into account the variations between men and women in relation to the operation. firstly, septoplasty is more common in men [ , ] and, secondly, there is a known gender influence on response to nasal-patient reported outcome measures [ ] . the aim of nairos is to establish, and inform guidance for, the best management strategy for patients with nasal obstruction associated with a deviated nasal septum, via a randomised controlled trial (rct) of surgery versus medical management across sites in both secondary and tertiary hospitals across england, scotland and wales. to establish, and inform guidance for, the best management strategy for participants with nasal obstruction associated with a deviated septum, via a randomised controlled trial comparing the clinical and costeffectiveness, of nasal septoplasty plus/minus (±) contralateral turbinate reduction versus medical management. the study objectives are split into three different aspects: clinical effectiveness, economic evaluations and mixedmethod process evaluation. clinical effectiveness to measure clinical effectiveness according to: subjective self-report rating of nasal airway obstruction heterogeneity of estimated treatment effect specifically according to severity of obstruction and gender objective measures of nasal patency number of adverse events (aes) and additional interventions required technical failure in the surgical arm how well those agreeing to enter the trial reflect those screened for eligibility the cost-effectiveness of each intervention the cost-utility with outcomes reported as incremental cost per quality adjusted life year (qaly) gained a longer-term economic model to assess costs and health consequences beyond -month follow-up period all economic analyses will be conducted from the perspective of the national health service (nhs) and participants mixed-methods process evaluation of the trial and interventions our mixed-method process evaluation will identify, describe, understand and address: barriers to optimal recruitment, and potential solutions to address these, through integration of the quintet recruitment intervention (qri) [ , ] participants' and healthcare professionals' experiences of trial participation and the interventions under evaluation factors likely to influence wider implementation of trial findings the design, measured outcomes and analysis of the process evaluation and qri are detailed later in this manuscript. a multicentre, randomised controlled, open-label trial, incorporating a qualitative process and economic evaluation. participants will be randomised on a : basis between septoplasty, with or without turbinate reduction, versus medical management (isotonic saline nasal spray (sterimar) and mometasone nasal spray) of nasal obstruction. participants in the medical management arm will be asked to use the nasal sprays twice daily for weeks, then once daily for the remainder of the -month period. recruitment will take place over months, with trial completion complete at months (submission of final report). the trial will take place in nhs hospitals across scotland, england and wales (see the isrctn registry number ). an overview of the nairos schedule of events patient pathway is shown in fig. . adults (aged ≥ years) referred by their general practitioner (gp) to ear, nose and throat (ent) secondary care outpatient clinics who are found to have a deviated septum on nasendoscopy and reduced nasal airway as indicated by a nose score ≥ . ent staff will also be recruited for participation in a process evaluation. the nairos eligibility criteria are listed in table . hospital researchers will proactively identify nairoseligible patients through triage of referral letters of rhinology patients to the ent department, and issue an invitation to attend a research clinic. patients attending a research clinic will, where possible, have been sent the patient information sheet (pis) with their appointment details, and have been directed to the patient information video, available at www.nairos.co.uk. all patients will have been given a minimum of h after receiving the pis to decide whether or not they would like to take part. the main pis can be found in additional file . consent a delegated member of the research team will undertake informed consent discussions with the opportunity for the patient to ask any questions and discuss the trial in more detail. patients will be invited to give informed, written consent in three stages. firstly, consent to undergo screening (eligibility). secondly, consent to have the discussion about the nairos trial with the investigator audio-recorded and their details passed onto • any prior septal surgery • systemic inflammatory disease or the use of any current oral steroid treatment within the past weeks • granulomatosis with polyangiitis • nasendoscopic evidence of unrelated associated pathology, e.g. adenoid pad, septal perforation, chronic rhinosinusitis indicated by the presence of polyposis or pus • any history of intranasal recreational drug use within the past months • breast-feeding, pregnancy or intended pregnancy for the duration of involvement in the trial • bleeding diathesis • therapeutic anticoagulation (warfarin/novel oral anti-coagulant (noac) therapy) • clinically significant contraindication to general anaesthesia • patients known to be immuno-compromised • those in whom an external bony deformity substantially contributes to the nasal obstruction a member of the qualitative team for a telephone interview. finally, eligible patients are invited to give consent for the main trial, and to also give consent to potential future sharing of their anonymised data with other researchers not related to the nairos study. the patient informed consent form can be found in additional file . screening screening data used to assess eligibility will include: clinical examination (including nasal endoscopy) nasal obstruction symptom evaluation scale (nose) scoreconfirmation of total ≥ age baseline recording of four core features at endoscopy of the undecongested nose • the side of the maximum convexity • one main site of deflection on each sideanterior/ posterior/upper/lower/all) • confirmation that there is no excluding inflammatory processpus/polyps/adenoids • magnitude of observer-rated airway block (< %; ≥ %) if the participant is unable to complete the endoscopic examination without topical preparation, it can be performed after the airway assessment of the decongested nose. the nose scale is a validated five-item, unifactorial self-report of nasal-block severity which has been applied in previous research and audit studies [ , ] . the three recognised nose-derived categories of baseline severity used will be: - = moderate, - = severe, - = extreme [ ] . for nairos, it is anticipated that baseline severity will be the most important determinant of outcome. those with a nose score of less than will be excluded from nairos on the basis of having symptoms that are too mild to warrant inclusion. randomisation at the baseline visit, consenting, eligible patients will be randomised on a : basis using random permuted blocks of variable length. stratification will be by gender and baseline severity (nose score). randomisation will be administered centrally by the newcastle clinical trials unit (nctu) web-based system. the treatment allocation is open label and the randomisation system will provide a unique trial identifier for each participant via email to a delegated member of site staff. participants will be randomised between: . septoplasty with or without unilateral turbinate reduction . medical management participants allocated to the septoplasty group will undergo surgical correction of the nasal septal deviation ± unilateral reduction of the inferior turbinate on the concave side. a preliminary secondary care feasibility exercise revealed that there is considerable variation in surgical practice around the uk; rates of contralateral turbinate reduction varied between nairos centres from to % of septoplasties. as a pragmatic study, nairos does not ask surgeons to change their usual practice in relation to contralateral turbinate reduction. nairos surgeons may or may not carry out unilateral turbinate surgery on the wider side, according to their assessment of the individual patient airway. intention to reduce one turbinate will be recorded prior to randomisation. details of the actual surgery performed will also be collected. participants will have a closed septoplasty, will be sutured, not packed, and will be a day case (where possible). the recommended post-operative twice-daily regimen will be of saline douche plus naseptin nasal cream (or if the patient is allergic to the peanut content of naseptin, bactroban % ointment). participants will be recommended to take a few days off work. nasal-steroid and saline sprays should not be part of routine standard post-operative care for nairos. any additional medication required by participants will be recorded as concomitant medication. surgery must be carried out anytime up to weeks (+ weeks) after randomisation. the additional -week window is to allow for extenuating circumstances only, such as unexpected patient or clinical reasons that necessitate a delay in surgery. reasons for delays to surgery will be collected and reported. the surgical intervention will be performed by surgeons who have completed their training. patients randomised to the medical management arm will be asked to use a combination of an isotonic spray with a full twice-daily dose of a fluorinated steroid spray (mometasone furoate) which is a typical maximal medical therapy regime over a -month period. preparatory work by the chief investigator indicated that most patients referred from their gp have never used this sustained combination therapy. sterimar isotonic nasal spray dose: one spray (metred dose) into each nostril prior to using the mometasone nasal spray. mometasone nasal spray dose: mcg (two sprays) into each nostril twice daily for weeks, followed by mcg (two sprays) into each nostril once daily or mcg (one spray) into each nostril twice daily for the remainder of the -month period. participants who wish to discontinue their allocated treatment, but remain in the trial, may access other treatments via the standard local nhs route. such participants will be followed up as per their allocated treatment intervention arm. participants in the surgical arm who wish to pursue medical treatment will not receive the trial investigational medicinal product (imp) prescription. participants in the medical arm who wish to receive surgery and remain eligible for septoplasty should be added to the elective nhs waiting list. the primary analysis is comparison of the comprehensive, validated sino nasal outcome test- (snot- ) [ ] patient-reported scores at months from randomisation (− weeks to + weeks), with complete follow-up of participants to months post randomisation. snot- is a commonly employed patient reported outcome measure in the assessment of patients with pathologies of the nose and sinuses [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] and was first applied to septoplasty in [ ] . our ppi work found that patient symptoms mapped better to the snot- than to the nose and that patients preferred the snot- measure. to maximise collection of primary outcome measure, participants who cannot attend the -month follow-up visit may complete snot- by post. secondary outcome measures can be categorised into patient-reported, safety, economic, exploratory and qualitative. patient reported outcome measures (proms) proms will be used to measure long-term change in nasal patency and quality of life: snot- subscales (rhinologic, sleep, ear/facial pain, psychological) at months nose scale at months double ordinal airway subjective scale (doass)administered post nasal decongestant use only at months. doass is a subjective comparator of right and left nasal patency [ ] allowing direct comparison with the spirometry measures safety outcomes safety outcomes will be measured by the number and characteristics of any aes, and surgical complication/failure and re-intervention within months. economic outcomes economic outcome measures include: qaly gained using the -item short form health survey (sf- ) questionnaire ( -week recall), further converted into qalys using the health economy survey derived from sf- (sf- d) algorithm [ ] , at months, and aes avoided use of and timing of additional interventions in primary and secondary care recorded by health care utilisation questionnaire at months and months number of days unable to undertake usual activities recorded by health care utilisation questionnaire at months and at months incremental cost per change in snot- at months costs to nhs and participants at months longer-term economic model to assess costs and health consequences beyond the trial exploratory outcome measures two of the most common objective measures of nasal patency, used in some overseas healthcare systems to assess likely benefit from septoplasty, are peak nasal inspiratory flow rate (pnif) and nasal partitioning ratio (npr) [ ] . pnif and npr will be used in this trial as exploratory outcome measures. all sites will be provided with two devices to measure two different measurements of nasal patency: pnif, measured with a pnif meter (peak nasal inspiratory flow (pnif) meter; gm instruments, kilwinning, uk) npr, measured using the nv rhinospirometer (nv rhinospirometer; gm instruments, kilwinning, uk) the two standard measurements will each be made before and after decongesting the nasal turbinate tissue with xylometazoline at baseline and at and months following randomisation. pnif measures the peak flow rate of air through both nostrils during inhalation using a pnif meter with a face mask. the participant holds the mask over the nose and mouth, closes the mouth and inhales maximally (sniffs). pnif has been shown to respond to septoplasty/turbinectomy [ ] and can, therefore, be used for an overall assessment of nasal airflow impairment, and as an objective outcome measure from surgery. however, pnif does not differentiate between the two nostrils. bench testing shows the nv rhinospirometer to be an accurate and precise objective marker of airflow symmetry [ ] . the nv rhinospirometer has two separate channels to measure the volume of air passing through each nostril, hence deriving the npr, the difference between right and left volumes divided by the sum. npr ranges from symmetrical ( ) to completely unilateral (± ). the npr appears to predict the septal surgery outcome [ , ] . comparison of npr during both maximal inhalation and normal tidal breathing will allow the comparative utility of these two measures to be compared and demonstrate any change in nasal function following treatment. qualitative outcomes qualitative outcomes will be identified through observations of training and nairos meetings, interviews with health professionals and participants, and audio-recording of recruitment discussions. the trial schedule of events is presented as a flow diagram (fig. ) and using the standard protocol items: recommendations for interventional trials (spirit) figure [ ] (fig. ) . participants recruited to the main trial will be followed up for months from the point of randomisation. data including the number of participants screened, approached and interested in taking part will be collected via a log completed by site staff conducting screening. assessments pre-randomisation eligible patients who consent to participate in the main trial will have the following outcome measures administered prior to randomisation: site nursing staff will record details of any concomitant medication and aes during a phone call at weeks after surgery has taken place, and at all scheduled trial visits. medical management arm data as a pragmatic trial using standard treatment as part of the medical management arm, precise assessment of any mometasone furoate spray and sterimar spray residuum will not take place. participant compliance with the imp does not form part of the trial monitoring plan. participants will be asked at the -month follow-up visit (visit ) to estimate how many bottles of the sterimar and mometasone furoate spray they used. site nursing staff will record details of concomitant medication and any aes during a phone call at weeks post randomisation and at all scheduled trial visits. data will be handled, computerised, stored and archived in accordance with the general data protection regulation ( ), and the latest directive on good clinical practice (gcp) ( / /ec). patient-identifiable data will remain at each site and not be collected as part of the trial dataset. patient identification on data collection tools used during screening will be through a unique sequential screening number allocated by site staff. patients recruited to the main trial will additionally be identified by a unique trial identifier number generated by the randomisation system. data will be transcribed and npr files uploaded by site staff to the trial's secure, password-limited, validated macro™ database (elsevier). the participant trial record, including completed paper data collection tools, will be archived at site for years following the end of the trial. audio-recordings will be archived for years. the trial will be conducted in accordance with the medicines for human use (clinical trials) regulations and subsequent amendments. all parties must abide by these regulations and the international conference on harmonisation-good clinical practice (ich-gcp) guidelines. participants who withdraw their consent from the trial, or are withdrawn by the investigator, will not be replaced. all data collected up until the point of withdrawal will be retained for nairos research purposes, and consent will be sought for this (additional file ). the snot- minimal clinically important difference (mcid) in the national comparative audit of surgery for nasal polyposis and chronic rhinosinusitis was . [ ] . septal surgery is reported variously as showing reductions in total snot- scores above ( points) [ ] or below ( points) this boundary [ ] . in the absence of a specific figure for septoplasty mcid, nairos has assumed a clinically relevant reduction being at least points. reported standard deviations (sd) of the snot- score were [ ] (in external septoplasty) to [ ] in septorhinoplasty, nairos assumed the larger, more conservative sd. sample size calculations were based on a t test for superiority assuming equal variance across groups, a conservative estimate given the primary analysis is based on adjustment for stratification covariates, increasing power. the target recruitment of participants allows for % drop-outbased upon experience from our unit's two prior septal surgery audits [ , ] . the remaining participants ( per arm at completion), are required to show a -point [ ] difference in overall snot- score between arms, with % power and % type i error, assuming a sd of . primary outcome the primary analysis is comparison of snot- scores at months by randomised treatment arm (immediate surgery vs medical management). mean overall scores will be presented by treatment group. the associated significance of any observed difference will be calculated in multivariable regression models adjusting any treatment effect by stratification factors, gender and nose severity at baseline. secondary analysis of the primary outcome measure will adjust for the influence of baseline severity snot- score as a continuous covariate, planned turbinate reduction as a binary covariate and other important demographic and clinical covariates at randomisation (including, but not exclusively, age, body mass index (bmi), smoking, endoscopic features). non-linear relationships between continuous baseline measures and outcome will be addressed by simple, and possibly more complex, fractional polynomial transformations. the nairos model will generate a linear predictor score of patient outcome weighted according to the statistical importance of each covariate. each patient's linear predictor score will be compared against observed score for internal validation. this model will be used to explore recommendations for treatment options. the importance of baseline severity, as a continuous distribution of nose score at randomisation, may be further explored graphically by subpopulation treatment effect pattern plots (stepp analysis) [ ] to display the predicted point estimates of any treatment effect (with % ci) over the range of nose values (range - in nairos participants), further informing any patient selection guidance and recommendations. primary statistical analyses will be carried out on an intention-to-treat basis. the number of ineligible participants and reasons for ineligibility will be reported. a sensitivity analysis may be conducted and reported if the number of ineligible participants or participants not receiving the allocated treatment is excessive. participants may choose to discontinue the treatment to which they have been allocated, and may also ask that they receive an alternative treatment as per local standard nhs care. the implication of such treatment adjustments, which typifies surgical trials, is that the intention-to-treat analysis will produce a conservative estimate of the effect of septoplasty. non-compliance (including receiving the alternative treatment) may be addressed using an 'as treated' approach or complier average causal effect (cace) approach, since the intention-to-treat analysis under noncompliance is biased when the intervention effect is large [ ] . statistical methods for withdrawal of participants, based on statistical censoring, may be considered. tests of heterogeneity will assess robustness of the overall treatment effect across stratification subgroups, and by intention to perform unilateral turbinate reduction. there are no formal interim analyses of the primary outcome measure and there are no formal statistical stopping rules. decisions regarding continuation of the trial will be made at dmc meetings held every months. decisions will be made on the basis of information presented in a statistical report that includes analysis of formal data snapshots, including safety data. analysis of secondary outcomes analyses of secondary outcomes will follow a broadly similar strategy. these will include the data at -month follow-up from the other outcomes (snot- subscales, nose, doass, sf- ) and that for all outcomes at -month follow-up. subjective scales, tabulated by arm and overall at randomisation, -month and -month follow-ups, will be compared by both summary statistics and graphical representation. multiple regression will be used to investigate longitudinal outcome scores between treatment groups at follow-up time points. variation between participants will be included as a random effect with an assumed normal distribution. analysis will include the stratification factors of baseline severity and gender. further adjusted analyses will include terms for baseline values of the scores and key demographic and clinical covariates. adverse events will be tabulated according to world health organisation (who) common terminology criteria for adverse events (ctcae) grade version . . number of severe (ctc grade , or ) will be reported as a proportion of all aes. number of participants experiencing at least one severe ctcae will be reported as a proportion of all participants. surgical complication/failure and re-intervention will be tabulated and will not subject to statistical testing. technical failures from operations where widening of nasal airway was achieved yet the symptoms persist will be reported. analysis of exploratory outcomes three measurements each of pnif and npr during maximal inhalation will be made. either the maximum (pnif) or average (npr) value is used. summary statistics will be presented for pnif and npr by arm and overall, at baseline, -month and month follow-ups. design the process evaluation incorporates the qri and mixed qualitative methods. data collection and analysis will commence during study set-up and continue throughout the trial. randomising patients between surgical and medical arms can be challenging. the qri, based at bristol university, will assist in the identification and methods of addressing such challenges. the qri uses novel qualitative and mixed-method approaches pioneered during the national institute for health research (nihr) health technology assessment (hta)-funded protect (prostate testing for cancer and treatment) study [ ] . these methods have since been applied to several other 'challenging' or controversial rcts in different clinical contexts, all of which have led to insights about recruitment issues and the development of generic and bespoke strategies to optimise recruitment [ ] . the qri will coincide with the study set-up and the first year of recruitment, using qualitative and novel methods to investigate and address recruitment barriers (objective a, below) [ ] [ ] [ ] . qualitative interviews will be conducted throughout the trial to investigate patients' and clinicians' experiences of the study procedures, interventions and barriers to implementing findings into practice (objectives b and c, below). sampling strategy the sampling strategy is informed by current and prior experience [ , , ] . in keeping with the principles of rigorous qualitative research, sampling will be responsive to the study context. in some cases fewer interviews or observations will be conducted, and in others, additional data will be required to accommodate our emerging analysis or study events. numbers of interviews will be guided by 'data saturation'continued sampling until findings become repetitious. objective a: optimising recruitment -qri (study set-up and first year of recruitment) working in close collaboration with the trial management group (tmg), the qri team will assimilate investigational and interventional approaches to understand and address recruitment difficulties in the early stages of nairos. the findings and implications of the qri will continue to be implemented by the tmg and study investigators throughout the remainder of the trial recruitment period. the qri will proceed in two iterative phases: a detailed understanding of the recruitment process will be developed in phase i, leading to tailored interventions to improve recruitment in phase ii. phase i: understanding the recruitment process and how it operates in individual centres. a multi-faceted, flexible approach will be adopted, comprising one or more of the following methods: (a) in-depth interviews, conducted with: members of the tmg (n = - ); clinicians or researchers involved in trial recruitment (n = - ); and eligible patients who have been approached to participate in the trial (n = - ). interviews will explore views on trial processes, perceptions of equipoise, and information about how the protocol is operationalised in clinical centres (b) audio-recording and non-participant observation of consultations during which the trial is discussed with patients, enabling identification of clear and subtle obstacles to recruitment (c) mapping of eligibility and recruitment pathwaysnoting the point at which patients receive information about the trial, which members of the clinical team they meet, and the timing and frequency of appointments. the qri researcher will work closely with the clinical trials unit to compose logs of potential rct participants as they proceed through screening and eligibility phases (d) regular observation of tmg and investigator meetings to gain an overview of trial conduct and overarching challenges (logistical issues, etc.) (e) scrutiny of study documentation (e.g. piss) to identify aspects that are unclear or potentially open to misinterpretation phase : development and implementation of recruitment strategies. anonymised findings from phase i will be presented to the tmg, summarising the factors that appear to be hindering recruitment. a plan of action will be devised in collaboration with the tmg if there is consensus that aspects of practice are amenable to change. interventions will be tailored to the nature of recruitment challenges identified. generic forms of intervention may include 'tips' documents on how to explain trial design and processes. supportive feedback will be a core component of the plan of action, with the exact nature and timing of feedback dependent on the issues that arise. centre-specific feedback may cover institutional barriers, whilst multicentre group feedback sessions may address widespread challenges. individual confidential feedback will be offered where there is a need to discuss specific challenges or potentially sensitive issues. objectives b and c: understanding experiences of septoplasty and non-surgical management we will investigate patients' (n = [ ] [ ] [ ] [ ] [ ] and health professionals' (n = - ) experiences of the interventions and trial participation through qualitative interviews conducted during patient follow-up. where possible, patients for the follow-up interviews will include those interviewed during the recruitment phase; additional participants will be recruited based on purposive and emergent criteria (e.g. patients who have declined their allocated treatment). we will identify any aspects of the care pathway which are problematic for patients or health professionals; and potential barriers and facilitators to wider acceptance and implementation of trial findings. a focus group of gps will explore preliminary trial findings and discuss implications for primary care management of nasal obstruction. our analysis of the implementation of study findings will be informed by normalisation process theory (npt) [ ] . qualitative data management and analysis all interviews will be audio-recorded, transcribed verbatim and edited to ensure anonymity of respondents. contemporaneous field notes from non-participant observation in clinical settings will be edited to ensure anonymity of participants. data will be managed using nvivo software. the analysis will be conducted according to the standard procedures of rigorous qualitative analysis which we have described previously [ ] , including open and focussed coding, constant comparison [ ] , memoing [ ] , deviant case analysis [ ] and mapping [ ] . we will undertake independent coding and crosschecking and a proportion of data will be analysed collectively in 'data clinics' where the research team share and exchange interpretations of key issues emerging from the data. audio-recorded recruitment consultations will be subjected to content, thematic and novel analytical approaches, including targeted conversation analysis [ ] and quanti-qual appointment timing (the 'q-qat method') [ ] . there will be a focus on aspects of information provision that is unclear, disrupted, or potentially detrimental to recruitment and informed consent. all participants who complete the nairos trial, or who discontinue the treatment interventions at any point, will be offered standard, local nhs care in discussion with their local investigator. the sponsor will provide indemnity in the event that trial participants suffer negligent harm due to the management of the trial. this indemnity will be provided under the nhs indemnity arrangements for clinical negligence claims in the nhs. the trial steering committee (tsc), data monitoring committee (dmc), trial statistician, data manager and other members of the central trial team as required will have access to the full trial dataset. individual site trial datasets will not be available to individual site investigators prior to publication of the main trial results. all requests for data should be directed to the corresponding author for consideration. access to the anonymised final trial dataset may be available following review; we will retain exclusive use until publication of major outputs. the results of the trial will be presented at topic-specific national or international conferences and published in a general medical journal with the monograph published by hta. authorship of all publications will be on a named individual authorship basis. for each publication, all individuals who fulfil the authorship definition for the publishing journal or site will be included as individually named authors. authorship order will be decided by the chief investigator and tmg. a lay summary of results and the hta report will be available on the nairos website. members of the patient and public involvement (ppi) panel will review results and they will be involved in writing lay summaries of results for dissemination to relevant patient groups. nctu staff will monitor trial conduct and data integrity to ensure that the trial is conducted in accordance to the latest directive on gcp ( / /ec). this will be detailed in a data management plan and a monitoring plan approved by the trial sponsor. safety reporting delegated nursing staff will interview participants to collect and record any aes. this will take place at every trial visit (n = ), and also via safety phone calls; weeks after randomisation for medical management arm participants, and weeks after septoplasty for surgical arm participants. serious adverse events (saes) will be assessed for any relationship to the treatment intervention (causality), and expectedness (by reference to the reference safety information (rsi)) of any serious adverse reactions (sars). only a qualified medical doctor, delegated to do so at site, may assess the causality and expectedness of each sae. trial management group a trial management group, facilitated by nctu, will convene approximately monthly throughout the duration of the trial. members will consist of key nctu staff, the chief investigator, local clinical co-applicants, trial statisticians, a sponsor representative and staff representing health economics, qualitative and quintet recruitment intervention teams. independent data monitoring committee an independent data monitoring committee (dmc) has been appointed to provide an independent review of participant safety and data endpoints. the independent members comprise two statisticians and a clinician. the dmc will meet at least annually, and report directly to the trial steering committee (tsc). trial steering committee a tsc has been appointed to provide overall independent supervision of the trial. members consist of an independent chair, two independent clinicians, an independent statistician, an independent health economist and three patient representatives. the tsc will meet least annually, after a dmc meeting. all substantial changes to the protocol were approved by the local uk hra research ethics committee, and standalone minor changes (version . ) were approved by the health research authority (hra), prior to implementation at sites. the current, full protocol is available to view on the trial funder's website: https://fundingawards.nihr.ac.uk/search. a summary of key changes to the protocol during the trial is listed in table . there is a paucity of evidence underlying the indications for septoplasty in the uk. at present, the decision to perform septoplasty is based on the clinician's subjective estimation of the impact on the affected nasal airway caused by a deviated septum. in addition, there is a lack of evidence of the impact of a standardised topical medical treatment regimen on the nasal airway in the presence of a septal deflection. at a time of rising healthcare costs and increasing scrutiny on the requirement to justify clinical interventions there is an urgent need to answer these questions. the aim of nairos is to perform a rct to compare surgical treatment to a standardised dual medical therapy (sterimar spray and mometasone spray) and estimate the effectiveness based on subjective nasal symptoms, objective airway measurements and the impact on quality of life. furthermore, a number of other interactions will be measured at baseline, and months following randomisation. the impact of known covariates including sex, turbinate enlargement and subjective degree of nasal obstruction will be assessed. nairos is a pragmatic 'real-world' trial, researching a common surgical intervention against a contemporary comparator in such a way that the results will be generalisable to nhs patients in whom it is offered. however, limitations are anticipated in both treatment arms. in the surgical arm clinicians may vary in their assessment and documentation of the nasal septum deflection. it is also recognised that there are shortcomings in objective measurements of the nasal airway [ ] . in the medical arm we are not monitoring quantities of nasal-steroid used and instead relying on patientreported use. nairos will also compare the cost-effectiveness, to the patient and the nhs, of both the medical and surgical arms of the trial. the challenges and barriers to patient recruitment will be analysed by the quintet recruitment intervention with a view to identifying and minimising these. a qualitative evaluation will explore the views of participants and staff and their experience of the intervention to enable us to shape guidelines and inform clinical decision-making in patients with a deviated nasal septum. the overarching aim will be to shape future guidance on the management of a deviated nasal septum in an nhs setting. the nairos trial is currently working to protocol version . , dated january . recruitment began on january , and is due to end on january . supplementary information accompanies this paper at https://doi.org/ . /s - - - . • specification of the location of the rsi within the mometasone smpc . , nov • specification of the decongestant spray to be used alongside the nasal patency measurements (xylometazoline) and classification of the decongestant spray as a nimp • clarification of the exclusion criteria regarding the use of orally administered steroids and updating the exclusion criteria to exclude patients who have an external bony deformity that is likely to make a substantial contribution to the nasal obstruction . , jun • update to the mometasone rsi • update the exclusion criteria from any history of intranasally administered recreational drug use to any history of intranasal recreational drug use within the past months • clarify the clinical examination procedure to state that patients who request local anaesthetic for nasal endoscopy may have the nasal endoscopy assessment carried out after the other trial assessments have been completed • clarification of the timing for the surgical intervention to state that patients randomised to septoplasty must have their septoplasty anytime within weeks of randomisation . , dec addition of a -week window to the timeline for surgery for use in extenuating circumstances (i.e. weeks + weeks) • change the window for the -month visit from ± weeks to − weeks/+ weeks to maximise collection of the primary outcome measure • clarification of management of patients between the -month and -month follow-up visits • clarification of management and options for participants who wish to discontinue with their allocated treatment and explore other surgical or medical treatments as part of standard nhs care • state that discontinuation of allocated treatment does not constitute withdrawal from the trial. update to the rsi for the surgical intervention ambulatory sinus and nasal surgery in the united states: demographics and perioperative outcomes admitted patient care -england nasal patency: problems in obtaining standard reference values for the surgeon complications in septoplasty based on a large group of patients is nasal packing necessary after septoplasty? a meta-analysis open versus endoscopic septoplasty techniques: a systematic review and metaanalysis quality of life and symptoms before and after nasal septoplasty compared with healthy individuals surgical treatments for snoring nasal obstruction considerations in sleep apnea nasal septoplasty with submucosal diathermy to inferior turbinates improves symptoms at months postoperatively in a study of one hundred and one patients septoplasty for nasal obstruction due to a deviated nasal septum in adults: a systematic review suturing of the nasal septum after septoplasty, is it an effective alternative to nasal packing? clinical outcomes of nasal septal surgery at high altitude revision septoplasty: a prospective disease-specific outcome study septoplasty with concomitant inferior turbinate reduction reduces the need for revision procedure outcomes of septoplasty a randomised clinical trial of turbinectomy for compensatory turbinate hypertrophy in patients with anterior septal deviations a randomized clinical study: outcome of submucous resection of compensatory inferior turbinate during septoplasty quality improvement report: improving design and conduct of randomised trials by embedding them in qualitative research: protect (prostate testing for cancer and treatment) study. commentary: presenting unbiased information to patients can be difficult optimising recruitment and informed consent in randomised controlled trials: the development and implementation of the quintet recruitment intervention (qri) outcomes after nasal septoplasty: results from the nasal obstruction septoplasty effectiveness (nose) study. otolaryngol head neck surg development of a severity classification system for subjective nasal obstruction psychometric validity of the -item sinonasal outcome test the sino-nasal outcome test (snot): can we make it more clinically meaningful? otolaryngol head neck surg does septoplasty enhance the quality of life in patients? prevalence of sinonasal outcome test (snot- ) symptoms in patients undergoing surgery for chronic rhinosinusitis in the england and wales national prospective audit functional and cosmetic outcomes of external approach septoplasty is the sino-nasal outcome test- a suitable evaluation for septorhinoplasty? evaluation of benefits of nasal septal surgery on nasal symptoms and general health gender-specific differences in chronic rhinosinusitis patients electing endoscopic sinus surgery can the sino-nasal outcome test (snot- ) be used as a reliable outcome measure for successful septal surgery? assessment of subjective scales for selection of patients for nasal septal surgery the estimation of a preference-based measure of health from the sf- the use of nasal spirometry as an objective measure of nasal septal deviation and the effectiveness of septal surgery use of peak nasal inspiratory flowmetry and nasal decongestant to evaluate outcome of septoplasty with radiofrequency coblation of the inferior turbinate the accuracy and reproducibility of rhinospirometry in detecting flow asymmetry in a nasal cavity model the value of bilateral simultaneous nasal spirometry in the assessment of patients undergoing septoplasty spirit explanation and elaboration: guidance for protocols of clinical trials nasal septal surgery: evaluation of symptomatic and general health outcomes patterns of treatment effects in subsets of patients in clinical trials reporting attrition in randomised controlled trials practice based, longitudinal, qualitative interview study of computerised evidence based guidelines in primary care the national randomised controlled trial of tonsillectomy in adults (nattina): a clinical and cost-effectiveness study: study protocol for a randomised control trial distributed decision making: the anatomy of decisions-in-action clear obstacles and hidden challenges: understanding recruiter perspectives in six pragmatic randomised controlled trials capturing users' experiences of participating in cancer trials evaluating complex interventions and health technologies using normalization process theory: development of a simplified approach and web-enabled toolkit a feasibility study incorporating a pilot randomised controlled trial of oral feeding plus pre-treatment gastrostomy tube versus oral feeding plus as-needed nasogastric tube feeding in patients undergoing chemoradiation for head and neck cancer (tube trial): study protocol the constant comparative method of qualitative analysis the quality of qualitative research. london: sage constructing grounded theory: a practical guide through qualitative analysis it's not just what you say, it's also how you say it: opening the 'black box' of informed consent appointments in randomised controlled trials a simple technique to identify key recruitment issues in randomised controlled trials: q-qat-quanti-qualitative appointment timing outcomes of septal surgery a systematic review of training programmes for recruiters to randomised controlled trials springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations the authors would like to thank the nairos site principal investigators for their continuing input into the nairos trial. nairos authors' contributions sc is the trial chief investigator and senior author. sc, jo'h and jaw led the funding application and protocol development. ds was the trial senior statistician during the funding bid and advised on trial design. dh is the trial senior statistician and advised on trial design, and leads on the statistical analysis plan. tf is the trial statistician. lt is the senior health economist, yo is the health economist, nr is the senior qualitative researcher and lr leads the quintet recruitment intervention. dwh contributed to the ppi for the funding bid. as, kjr, a-mh, e-mh and aa provided trial and database management and trial monitoring. md and ab developed the pnif and npr measurement protocols. kjr drafted the manuscript for this publication. all authors contributed to protocol development. all authors have revised the manuscript and given approval for the final version. this trial is funded by a national institute for health research health technology assessment programme grant (funder reference / / ). the funder did not contribute to this manuscript, but provided review and approval prior to submission for publication. data-sharing is not applicable to this article as no datasets were generated or analysed for this article. the main pis and main informed consent form are presented as additional files and , respectively.ethics approval and consent to participate conduct of the trial will be in accordance with the recommendations for physicians involved in research on human subjects adopted by the th world medical assembly, helsinki and later revisions. informed, written consent will be given by all participants prior to any trial screening procedures and participation in the trial.a favourable ethical opinion has been granted from the uk health research authority research ethics committee (north east -newcastle and north tyneside ; study reference approval number /ne/ ). as per the uk hra process, this central rec review is applicable to all trial sites; local ethical review is not required. we will not commence recruitment at sites until local research and development confirmation of capacity and capability is received. the trial has also received approval from the uk medicines and healthcare products regulatory agency (mhra; study reference number - - ). the trial has been included in the national institute for health research clinical research network (nihr crn) portfolio (nihr clrn study id: ). the trial sponsor is the newcastle upon tyne hospitals nhs foundation trust, freeman hospital, freeman road, high heaton, newcastle upon tyne, ne dn. the trial sponsor has delegated responsibility for trial management to nctu, including trial design; review and approval of all localised patient-facing documentation prior to implementation at each site; collection, analysis and interpretation of data; writing of the protocol publication and final clinical report manuscripts. the sponsor did not contribute to this manuscript, but provided review and approval prior to submission for publication. not applicable. the authors declare that they have no competing interests.author details key: cord- -jjye t j authors: ingraham, nicholas e.; tignanelli, christopher j. title: fact versus science fiction: fighting coronavirus disease requires the wisdom to know the difference date: - - journal: crit care explor doi: . /cce. sha: doc_id: cord_uid: jjye t j nan s ince december , countries have quickly shifted from spectators to victims of the severe acute respiratory syndrome coronavirus (sars-cov- ) outbreak. the coronavirus disease (covid- ) pandemic has overwhelmed healthcare systems and crippled economies around the world, including the united states. unlike china, many countries had time to prepare for their epidemics and, as of march , remain in the early stages. time has allowed the scientific community to bear arms and mount expedited research efforts in unforeseen fashion. the collaboration between academic institutions and government agencies is unprecedented and inspiring. countless hours and sleepless nights from academics around the world have resulted in a united front to mitigate morbidity and mortality from covid- . despite these concerted efforts, another pandemic, in its own right, threatens to destroy the meticulously built scientific juggernaut surrounding covid- . those are alternative facts. this commentary uses a recent study of hydroxychloroquine to demonstrate the dire need for randomized clinical trials, but more importantly, to explore the potential consequences of misinformation, how fear fuels its impact, and offer guidance to maintain scientific integrity without relinquishing hope. chloroquine, and its less toxic metabolite hydroxychloroquine, are chemotherapeutic agents used to treat malaria ( ). chloroquine/hydroxychloroquine also inhibits certain inflammatory pathways resulting in the treatment of multiple rheumatological disorders ( ). the sars-cov outbreak in led to yet another therapeutic potential when chloroquine was found to inhibit sars-cov attachment by altering the binding protein and receptor required for entry ( ) . despite other in vitro studies identifying a potential role in killing infected cells ( , ) , in vivo trials have yet to confirm these findings ( ) . in fact, preclinical trials of chloroquine in other viruses, dengue fever, and chikungunya, demonstrated enough promise to warrant further clinical trials; however, the preclinical effects did not translate to humans ( , ) . as of march , there remains no randomized control trial in humans with evidence that chloroquine or hydroxychloroquine is beneficial in sars-cov or sars-cov- . to date, the only randomized control trial with published data found no benefit (viral clearance) between the patients treated with chloroquine compared with patients treated with a placebo (c. jun, unpublished observations, ). although an abstract published online february , touts efficacy with chloroquine in patients, there remains no publication of these data ( ) . however, a recent nonrandomized, observational study by gautret et al ( ) found decreased viral load in patients treated with hydroxychloroquine compared with a control group chosen from another hospital. these findings are noteworthy for many reasons, specifically for reasons not commonly noted. the lack of randomization and adjustment in the study by gautret et al ( ) introduces selection and confounding bias, respectively. this alone should temper any interpretation, but the more subtle discretion regarding the six excluded patients from the treatment (hydroxychloroquine) group, warrants an equivalent amount of skepticism. of these patients, one died, three clinically deteriorated requiring intensive care admission, one withdrew secondary to a drug-related complication, and another was lost to followup. excluding these patients drastically biases the results toward a benefit. in fact, if included, the number needed to harm with hydroxychloroquine is six (hydroxychloroquine . % vs control % harm), which is arguably of greater clinical significance compared with decreasing unadjusted viral load. furthermore, the mechanistic primary outcome with an implausible sample size calculation (a presumed % efficacy) creates enough methodological concerns to prevent anything beyond the hypothesis-generating conclusions from this study. the limitations stated above are not groundbreaking and were likely appreciated by the regular and seasoned academic readership. unfortunately, the "readership" of not only this article but all academic literature has broadened immensely over the past months. the world is, understandably, grasping for a discovery; while the scientific community operates under a worldwide microscope in desperate need of a critical appraisal filter. despite its methodological limitations and lack of clinical relevance, the article by gautret et al ( ) continues to trend on social media days after its publication. this article and its repercussions reinforce the adage that "a lie can travel halfway around the world while the truth is still putting on its shoes. " since its publication, prescriptions for chloroquine and hydroxychloroquine have dramatically increased, standard operating procedures have incorporated hydroxychloroquine as a standard of care, lethal overdoses have occurred, and there are rising concerns that shortages may affect availability for u.s. food and drug administration-approved uses of these medications. premature acceptance of efficacy is not new (swine flu vaccination [ ] or recombinant human activated protein c [ ] ), but it is these prior experiences that influence current standards to require high quality and often multiple randomized control trials to change practice. this high bar contributes to the estimated years it can take for best practices to be translated into medical practice ( ) . this glacial, yet safe and meticulous, pace is unacceptable in our current crisis. lowering the bar and maintaining scientific rigor is possible when the scientific community harmoniously pivots toward a single target. remarkably, we have succeeded in this movement by coming together as an international community. unfortunately, unless we maintain the narrative from our scientific surge, the "bar" we have meticulously repositioned may fall quickly to the floor. inaccurate facts stem from two levels. the ability to identify and suppress both misinformation and disinformation is crucial to maintaining the scientific process. misinformation is incorrect or misleading information ( ), whereas its more devious counterpart, disinformation is false information deliberately and often covertly spread to obscure the truth ( ). the example above represents misinformation. we can safely assume the differing interpretations occurred unintentionally, and we hope to provide guidance for future situations while we await results from randomized clinical trials. misinformation occurs to a greater degree during disasters ( ) . a natural human tendency during a crisis is to find resolution, even when it does not exist ( ) . fear fuels these efforts to dissipate this uncertainty. the limitations of the study by gautret et al ( ) are not lost on seasoned academic researchers. however, despite warnings from healthcare leaders and public health agencies, there continues to be a premature adoption of hydroxychloroquine as treatment based on limited preclinical data and misinformed interpretation of a nonrandomized study. importantly, compassionate and well-intentioned healthcare workers are not immune to these tendencies. arguably, healthcare workers are more vulnerable to misinformation in our current climate, as the careful and curious lens, previously used to critically appraise the literature, is now blurred by their intrinsic passion to "do something. " intentions aside, misinformation is a current public health emergency! if left unchecked, preventable patient morbidity and mortality will occur while simultaneously dismantling the remarkable ongoing efforts to defeat covid- . there may be a risk to the integrity of not only hydroxychloroquine trials but also other investigational drug trials currently ongoing. widespread use and misconception of hydroxychloroquine being a cure may result in reduced enrollment in hydroxychloroquine trials or hesitation to enroll in other drug trials that are not hydroxychloroquine. these potential ripple effects from misinformation pose the greatest threat to our ongoing fight against covid- . merchant and asch ( ) layout countermeasures to combat misinformation. first, it is essential to leverage our position in social media and advocate for transparency ( table ) . accomplishing this will undoubtedly require a unified effort by the medical community given the majority of social media influence resides outside of healthcare. fortunately, we can turn to science to provide some insight. a recent article in scientific reports analyzed the social media mechanisms associated with social bursts during significant worldly events. although the motivation to propagate ("liking" or "retweeting") a topic is commonly influenced by the user's fondness for the topic, during significant events the influence is shifted and amplified based on their degree of trust in the source of information ( ) . we live in a world where an entertainer, politician, or idol's endorsement is echoed to millions. in contrast, a world-renown scientist or medical provider reaches people on the magnitude of thousands. this creates a vacuum of credible medical information. as the world looks to the medical community, they do so through underutilized mechanisms where our voice is barely a whisper. we currently have an ethical responsibility as leaders in our field to be prominent, speak out against misinformation, and deliver the facts. just as we are condensing the timeline for research, we must streamline our professional opinions, in realtime, through social media engagement. similarly, it is imperative that journals expedite responses to misinformation, solicit commentary for controversial topics, and deliver a unified message in collaboration with the authors. resultantly, the scientific community controls the narrative while preserving its veracity. this is crucial because if left unchecked, the narrative appears increasingly malleable as the virus and fear spread. the spotlight is currently on the academic community more than ever. we owe it to the research coordinators, investigational drug service pharmacists, the couriers, the project managers, the statisticians, the trainees performing chart reviews, our mentors, and our mentees, to ensure their unrelenting and ongoing efforts are not in vain. equally as important, we owe it to our family, friends, and community to be the beacon of hope while preserving scientific integrity. a t.s. eliot quote ( ) was recently reframed to reflect our duty as a scientific community, we aim to arrive where we have started-at the bedside of the critically ill and injured patient-caring in ways that were yesterday unimagined, today unknown, and will become tomorrow standard. ( ) dr. tignanelli is a principal investigator for two randomized control trials investigating angiotensin-receptor blockers in the treatment of coronavirus disease among inpatient and outpatients. co-investigators include dr. ingraham. dr. ingraham received funding from national institutes of health and national heart, lung, and blood institute (t hl ). for information regarding this article, e-mail: ingra @umn.edu chloroquine and hydroxychloroquine toxicity new insights into the antiviral effects of chloroquine design and synthesis of hydroxyferroquine derivatives with antimalarial and antiviral activities chloroquine is a potent inhibitor of sars coronavirus infection and spread evaluation of immunomodulators, interferons and known in vitro sars-cov inhibitors for inhibition of sars-cov replication in balb/c mice a randomized controlled trial of chloroquine for the treatment of dengue in vietnamese adults on chikungunya acute infection and chloroquine treatment breakthrough: chloroquine phosphate has shown apparent efficacy in treatment of covid- associated pneumonia in clinical studies hydroxychloroquine and azithromycin as a treatment of covid- : preliminary results of an open-label non-randomized clinical trial swine influenza vaccine and guillain-barré syndrome: lies, damn lies, and...-reply prowess-shock study group: drotrecogin alfa (activated) in adults with septic shock managing clinical knowledge for health care improvement definition of misinformation. . available at definition of disinformation. . available at researchers are tracking another pandemic, tooof coronavirus misinformation how a crisis researcher makes sense of covid- misinformation protecting the value of medical science in the age of social media and "fake news unraveling the origin of social bursts in collective attention exploring the endless frontier key: cord- - wtk y c authors: tini, giulia; duso, bruno achutti; bellerba, federica; corso, federica; gandini, sara; minucci, saverio; pelicci, pier giuseppe; mazzarella, luca title: semantic and geographical analysis of covid- trials reveals a fragmented clinical research landscape likely to impair informativeness date: - - journal: front med (lausanne) doi: . /fmed. . sha: doc_id: cord_uid: wtk y c background: the unprecedented impact of the covid- pandemic on modern society has ignited a “gold rush” for effective treatment and diagnostic strategies, with a significant diversion of economic, scientific, and human resources toward dedicated clinical research. we aimed to describe trends in this rapidly changing landscape to inform adequate resource allocation. methods: we developed an online repository (covid trial monitor) to analyze in real time the growth rate, geographical distribution, and characteristics of covid- related trials. we defined structured semantic ontologies with controlled vocabularies to categorize trial interventions, study endpoints, and study designs. analyses are publicly available at https://bioinfo.ieo.it/shiny/app/covidct. results: we observe a clear prevalence of monocentric trials with highly heterogeneous endpoints and a significant disconnect between geographic distribution and disease prevalence, implying that most countries would need to recruit unrealistic percentages of their total prevalent cases to fulfill enrolment. conclusions: this geographically and methodologically incoherent growth casts doubts on the actual feasibility of locally reaching target sample sizes and the probability of most of these trials providing reliable and transferable results. we call for the harmonization of clinical trial design criteria for covid- and the increased use of larger master protocols incorporating elements of adaptive designs. covid trial monitor identifies critical issues in current covid- -related clinical research and represents a useful resource with which researchers and policymakers can improve the quality and efficiency of related trials. standard and effective approaches for covid- prevention and treatment are not available to date, despite the magnitude of the pandemic and the similarities with the past coronavirusassociated diseases sars and mers ( ) . so far, initial trials with antivirals or other potentially effective drugs such as chloroquine have not yet clearly demonstrated superior efficacy over alternative treatments ( ) ( ) ( ) , and the disease remains associated with devastating morbidity and mortality. a wide variety of intervention strategies have been proposed, aiming at different mechanisms (viral or host processes), disease stages (early, advanced, or prevention), and intervention modalities (medical or non-medical). as covid- -devoted resources grow, quantifying the potential impact of covid- trials becomes a relevant matter for global and national health policies. however, quality research on clinical trials is rendered difficult by the lack of a standardized definition of trial parameters. data reporting in trial repositories is notoriously plagued by internal inconsistencies, especially for "free text" fields that contain key information like inclusion criteria or study endpoints ( ) . general medical ontologies like mesh terms provide an all-encompassing framework but may be inadequate to capture relevant distinctions for specific fields; covid-related terms were only introduced in late march, and their use is only recommended and not mandatory for trial definition. in the present work, we defined structured semantic ontologies with controlled vocabularies to categorize trial interventions, study endpoints, and study designs, and we conducted an analysis of the growth rate, geographical distribution, and trial characteristics of covid- -related trials, highlighting a number of relevant features that may impair the possibility of obtaining reliable and transferable results within the current framework. we formulate proposals for more rational trial designs against this rapidly changing landscape. we identified , relevant studies (including interventional, observational, and other) combining entries from the who and clinicaltrials.gov databases ( figure a) . from january, (the date of the first study posted), the cumulative increase in the number of studies ( figure b ) and the projected enrolled patients ( figure c ) have been growing logistically. we analyzed the funding source on the interventional trials from clinicaltrials.gov for which this information was available ( table ) and found that a high percentage ( , . %) are funded by public agencies, ( . %) by industries, and ( . %) by private-public collaborations. comparison with a disease of comparable magnitude like influenza or cancer shows how this ratio of industry vs. non-industry is highly unusual (influenza / , . %, p = . × − ; cancer / , . %, p < . × − ); instead, no significant differences were found for privatepublic collaborations (influenza, %, cancer %, p-values are non-significant). for subsequent analyses, we focused on interventional trials (n = ), although data have been collected for all trials and are available in supplementary table . trials were opened in different countries. at the national level, the united states was the nation with the highest number of trials, followed by china (figure a ). we calculated a simple "trials per patient" index (tpp) for each country by dividing the number of available trials by the number of cumulative covid- cases in the country. this index may help to gauge the feasibility/accessibility trade-off for trials ongoing in that country: a high index (=many trials relative to the patient population) suggests unrealistic enrolment needs (in other words, it is unlikely that all trials will fulfill the required enrolment), whereas a low index suggests low access to experimental treatments. trials per patient (tpp) were unevenly distributed among and within nations ( figure b) , with a gini coefficient equal to . . of the trials with available location information, the vast majority were monocentric ( , . %), while were multicentric. of those, just were opened in more than locations ( figure c) . the correlation between the cumulative projected patient enrolment and the actual case prevalence in each state was poor (pearson= . ). with current case prevalence, most countries would need to recruit extremely high and possibly unrealistic percentages of their total prevalent cases to fulfill enrolment ( figure d ). early-phase studies (phase and - ) were under-represented in both numbers and patients (figures a,b) . to better describe and capture the semantic heterogeneity of trial characteristics, we defined ontologies with controlled vocabularies for interventions, study designs (supplementary table ), inclusion criteria (supplementary table ) , and study endpoints (supplementary table ). among trials aimed at active treatment, a significant share ( / ) do not require pcr-confirmed diagnosis as inclusion criteria ("suspected, " figure c ). primary endpoints are qualitatively (clinical or virological, radiological, or other laboratory variables) and quantitatively ( proportion, time-to-event, quantity) heterogeneous; "hard" endpoints containing mortality either use the incommensurable limited to clinicaltrials.gov. "oth/ind" represent studies that were funded by a collaboration between industry and public sources, "oth/nih" studies were funded by public sources and nih, and "u.s.fed/oth" are those funded by public sources and the us fed. quantitative who ordinal scale or proportional measures ( figure d) . we categorized all interventional treatments under terms. randomization is common but not prevalent among most interventions ( figure e) ; chloroquine, immune-modulating agents (expanded in figure f ), and antivirals (expanded in figure g ) are the most investigated, with , , and studies, respectively. we present quantitative, updated, and semantically organized measures of covid- -related trials. we highlight a number of peculiar characteristics of this clinical research landscape: extremely rapid growth, substantial geographical and methodological incoherence, an unusual funding pattern, prevalence of monocentric trials, and extreme heterogeneity in the interventions tested. these characteristics are unprecedented in the history of clinical research, a consideration that prevents meaningful comparison with the research landscapes of other prior major outbreaks. the main limitation of our analysis is represented by the heterogeneity in terms of quality and quantity of the available information. the source databases often use non-overlapping trial categorization methods, and many of the records have missing, misspelled, or imprecise wording, potentially causing relevant selection biases. we attempted to mitigate these by forcing information through controlled vocabularies, a procedure that may result in loss of information. we argue that several of the planned trials are unlikely to provide high-quality results for the following reasons. first and foremost, the unrealistic percentages of total prevalent cases needed to fulfill planned enrollment at the national level imply that several trials are unlikely to reach target sample sizes, with severe loss of statistical power or study termination. this has in fact already been observed with the recently published remdesivir trial in china, which failed to complete enrolment, leading to conflicting interpretations ( ). (d) relationship between projected national enrolment and current cumulative confirmed cases by state. reference lines project the percentage of all confirmed cases to be enrolled. if a point sits on the % line, it means that % of all confirmed cases must be enrolled in a trial to satisfy enrolment projections for that country. geographical fragmentation will magnify local and studyspecific confounding in demographics, comorbidities, and the availability of healthcare resources, which are known to impact covid- outcome heavily ( ) ( ) ( ) . variegated endpoints and inclusion criteria will inhibit the possibility of adequately comparing and meta-analyzing treatments across trials. proper dose-finding trials are scarce, giving rise to a risk of under-or over-treating patients and of underestimating potentially risky drug-interactions. finally, the scientific soundness of classical randomized designs in a scenario where the control arm may be rapidly changing ( ) is ethically and methodologically questionable. our analysis provides quantitative grounds for concerns raised in the early days of the covid- pandemic in commentaries ( ) ( ) ( ) that highlighted the difficulties of striking a balance between the need to conduct sound clinical research and the need to take rapid action. this observed disordered growth in clinical research is perhaps expected given the unprecedented medical and socio-economic impact of the covid- pandemic and the absence of homogeneous and clear-cut guidelines on key aspects of covid- -related clinical research, such as what should be considered the gold standard for control arms or the primary endpoints for drug approval. however, we note that the scientific community should prepare the ground for a more ordered development, especially in light of the expected persistence of sars-cov and the likely emergence of other coronavirus-mediated diseases in the long run. a potential solution for some of the above issues is to favor the adoption of adaptive trial design features (inclusion of predefined toxicity/efficacy stopping rules, biomarker-adjusted enrolment, etc.) and the inclusion of multiple phases, interventions, and patient groups under the same regulatory framework, using the so-called "master protocol" model ( ) . advantages of this model include (i) the possibility of comparing the efficacy of multiple interventions against a single, well-standardized control arm, (ii) the possibility of comparing across multiple treatments, particularly relevant in a scenario where time bias is likely to play a major role: mortality is likely to be subject to time-dependent variables such as the icu occupancy ratio or physician experience acquired, (iii) the possibility of skewing enrolment into more effective/less toxic arms as new data are accumulated, (iv) the possibility of introducing novel treatment arms or stratification biomarkers as these are identified in preclinical or translational studies, and (v) the possibility of collecting samples for translational studies under a unified and homogeneous framework, increasing their informativeness. we identified trials with declared adaptive features in their designs (supplementary table ) , among which the most notable is the large solidarity trial promoted by the who to test four treatment options (remdesivir; lopinavir/ritonavir; lopinavir/ritonavir with interferon beta- a; chloroquine or hydroxychloroquine) against standard of care. master protocols are themselves subject to specific biases, in particular the need to adjust for multiple hypothesis testing ( , ) , and often require sophisticated monitoring and logistics that can only be accomplished within large organizations. this calls for stronger interaction between stakeholders like pharma companies, regulatory bodies, funding entities, and patient organizations. in the present rapidly changing scenario, such frameworks may be of particular utility to efficiently discard nonviable hypotheses and prioritize treatment that deserves proper testing on larger scales. experience gained in some fields where master protocols are increasingly adopted, such as oncology ( , ) , may inform trial design. data were downloaded from clinicaltrials.gov and the who international clinical trials registry platform (ictrp https:// www.who.int/ictrp/en/) on april and . data for covid cases by country and for us states were downloaded from the johns hopkins data repository (https://github.com/cssegisanddata/covid- ) and for italian regions from presidenza del consiglio dei ministri-dipartimento della protezione civile (https://github.com/pcmdpc/covid- ) on april . details on ontology definition, geographical analyses, and statistical analyses are discussed in the supplementary methods. all datasets generated for this study are included in the article/supplementary material. gt and lm: collected data, performed analyses, and wrote the manuscript. bd, fb, and fc: performed analyses. sg, sm, and pp: wrote the manuscript. all authors contributed to the article and approved the submitted version. the sars, mers and novel coronavirus (covid- ) epidemics, the newest and biggest global health threats: what lessons have we learned? a rush to judgment? rapid reporting and dissemination of results and its consequences regarding the use of hydroxychloroquine for covid- a trial of lopinavirritonavir in adults hospitalized with severe covid- compassionate use of remdesivir for patients with severe covid- obstacles to the reuse of study metadata in clinicaltrials remdesivir in adults with severe covid- : a randomised, double-blind, placebo-controlled, multicentre trial potential association between covid- mortality and health-care resource availability prediction for progression risk in patients with covid- pneumonia: the call score clinical course and risk factors for mortality of adult inpatients with covid- in wuhan, china: a retrospective cohort study optimizing the trade-off between learning and doing in a pandemic pharmacologic treatments for coronavirus disease (covid- ): a review covid- and risks to the supply and quality of tests, drugs, and vaccines global coalition to accelerate covid- clinical research in resource-limited settings master protocols to study multiple therapies, multiple diseases, or both adaptive clinical trials: advantages and disadvantages of various adaptive design elements master protocols in clinical trials: a universal swiss army knife? master protocols in immuno-oncology: do novel drugs deserve novel designs? the supplementary material for this article can be found online at: https://www.frontiersin.org/articles/ . /fmed. . /full#supplementary-material key: cord- -eikz zz authors: allahyari, abolghasem; rahimi, hossein; khadem-rezaiyan, majid; mozaheb, zahra; seddigh-shamsi, mohsen; bary, alireza; kamandi, mostafa; azimi, sajad ataei; hasanabadi, saeed eslami; noferesti, alireza; shariatmaghani, somayeh sadat; rafatpanah, houshang; khatami, shohreh; imani, afshin jabbar; mortazi, hassan; nodeh, mohammad moeini title: effect of hydroxychloroquine on covid- prevention in cancer patients undergoing treatment: a structured summary of a study protocol for a randomised controlled trial date: - - journal: trials doi: . /s - - -x sha: doc_id: cord_uid: eikz zz objectives: in this study, we investigate the effect of hydroxychloroquine on the prevention of novel coronavirus disease (covid- ) in cancer patients being treated. trial design: this is a multi-centre, two-arm, parallel-group, triple-blind, phase - randomised controlled trial. participants: all patients over the age of from types of cancer are included in the study. patients with acute lymphoid and myeloid leukemias in the first line treated with curative intent, patients with high-grade non-hodgkin's lymphoma treated with leukemia protocols and patients with non-metastatic breast and colon cancer in the first line of treatment will enter the study. the exclusion criteria will include known sensitivity to hydroxychloroquine, weight below kilograms, history of retinopathy, history of any cardiac disease, acute respiratory tract infection in the last months, having covid- in the first two weeks of entering the trial, having diabetes mellitus, having an immuno-suppressive disease other than cancer, having chronic pulmonary disease and taking immuno-suppressant drug other than chemotherapeutic agents for current cancer. this study is performed in five academic centres affiliated to mashhad university of medical sciences, mashhad, iran. intervention and comparator: patients are randomly assigned to two groups; one being given hydroxychloroquine and the other is given placebo. during two months of treatment, the two groups are treated with either hydroxychloroquine (amin® pharmaceutical company, isfahan, iran) or placebo (identical in terms of shape, colour, smell) as a single mg tablet every other day. patients will be monitored for covid- symptoms during the follow-up period. if signs or symptoms occur (fever, cough, shortness of breath), they will be examined and investigated with a high-resolution computed tomography (ct) scan of the lungs, covid- specific igm, igg antibody assay and a nucleic acid amplification test (nat) for the sars-cov- virus. main outcomes: the primary end point of this study is to investigate the incidence of covid- in patients being treated for their cancer over a -month period. randomisation: randomisation will be performed using randomly permuted blocks. by using an online website (www.randomization.com) the randomization sequence will be produced by quadruple blocks. the allocation ratio in intervention and control groups is : . blinding (masking): participants and caregivers do not know whether the patient is in the intervention or the control group. the outcome assessor and the data analyst are also blinded to group assignment. numbers to be randomised (sample size): the calculated total sample size is patients, with patients in each group. trial status: the trial began on april , and recruitment is ongoing. recruitment is anticipated to be completed by june , there has been no change in study protocol since approval, protocol version was approved april , . trial registration: this trial has been registered by the title of “effect of hydroxychloroquine on novel coronavirus disease (covid- ) prevention in cancer patients under treatment” in iranian registry of clinical trials (irct) with code “irct n ”, https://www.irct.ir/trial/ . registration date is april , . full protocol: the full protocol is attached as an additional file, accessible from the trials website (additional file ). in the interest in expediting dissemination of this material, the familiar formatting has been eliminated; this letter serves as a summary of the key elements of the full protocol. intervention and comparator: patients are randomly assigned to two groups; one being given hydroxychloroquine and the other is given placebo. during two months of treatment, the two groups are treated with either hydroxychloroquine (amin® pharmaceutical company, isfahan, iran) or placebo (identical in terms of shape, colour, smell) as a single mg tablet every other day. patients will be monitored for covid- symptoms during the follow-up period. if signs or symptoms occur (fever, cough, shortness of breath), they will be examined and investigated with a high-resolution computed tomography (ct) scan of the lungs, covid- specific igm, igg antibody assay and a nucleic acid amplification test (nat) for the sars-cov- virus. main outcomes: the primary end point of this study is to investigate the incidence of covid- in patients being treated for their cancer over a -month period. randomisation: randomisation will be performed using randomly permuted blocks. by using an online website (www.randomization.com) the randomization sequence will be produced by quadruple blocks. the allocation ratio in intervention and control groups is : . blinding (masking): participants and caregivers do not know whether the patient is in the intervention or the control group. the outcome assessor and the data analyst are also blinded to group assignment. numbers to be randomised (sample size): the calculated total sample size is patients, with patients in each group. trial status: the trial began on april , and recruitment is ongoing. recruitment is anticipated to be completed by june , there has been no change in study protocol since approval, protocol version was approved april , . trial registration: this trial has been registered by the title of "effect of hydroxychloroquine on novel coronavirus disease (covid- ) prevention in cancer patients under treatment" in iranian registry of clinical trials (irct) with code "irct n ", https://www.irct.ir/trial/ . registration date is april , . full protocol: the full protocol is attached as an additional file, accessible from the trials website (additional file ). in the interest in expediting dissemination of this material, the familiar formatting has been eliminated; this letter serves as a summary of the key elements of the full protocol. keywords: covid- , randomised controlled trial, protocol, hydroxychloroquine, acute lymphoid leukemia, acute myeloid leukemia, breast cancer, colon cancer, prophylaxis supplementary information accompanies this paper at https://doi.org/ . /s - - -x. additional file . ghaem hospital, ahmadabad ave, shariati sq publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations we thank the vice chancellor for research of mashhad university of medical sciences for supporting this project. also, the help of clinical research development unit of akbar hospital (affiliated to mashhad university of medical sciences, mashhad, iran) in designing the study and methodological issues is highly appreciated. all the financial resources required for this project have been provided by the vice-chancellor for research of mashhad university of medical sciences (code: ). the funding body has no role in the design of this study, collection, analysis, and interpretation of data and in writing the manuscript. the final dataset of the trial will be available upon request from the primary investigator via e-mail at moeininm@mums.ac.ir, after obtaining the permission of the regional ethics committee. the trial has been approved by the ethical committee of mashhad university of medical sciences, iran. ethics committee reference number is ir.mums.rec. . which was registered on april , . (publicly available at this link) hereby we certify that this trial has received ethical approval from the aforementioned ethical committee as described above. an informed consent is obtained from all participants or their legal guardians prior to recruitment. not applicable. the authors declare that they have no competing interests. key: cord- -chz luni authors: duffett, mark; choong, karen; ng, vivian; randolph, adrienne; cook, deborah j title: surfactant therapy for acute respiratory failure in children: a systematic review and meta-analysis date: - - journal: crit care doi: . /cc sha: doc_id: cord_uid: chz luni introduction: exogenous surfactant is used to treat acute respiratory failure in children, although the benefits and harms in this setting are not clear. the objective of the present systematic review is to assess the effect of exogenous pulmonary surfactant on all-cause mortality in children mechanically ventilated for acute respiratory failure. methods: we searched the medline, embase, cinahl and ovid healthstar databases, the bibliographies of included trials and review articles, conference proceedings and trial registries. we included prospective, randomized, controlled trials of pulmonary surfactant that enrolled intubated and mechanically ventilated children with acute respiratory failure. we excluded trials that exclusively enrolled neonates or patients with asthma. two reviewers independently rated trials for inclusion, extracted data and assessed the methodologic quality. we quantitatively pooled the results of trials, where suitable, using a random effects model. results: six trials randomizing patients were included. surfactant use reduced mortality (relative risk = . , % confidence interval = . to . , p = . ), was associated with increased ventilator-free days (weighted mean difference = . days, % confidence interval = . to . days, p = . ) and reduced the duration of ventilation (weighted mean difference = . days, % confidence interval = . to . days, p = . ). conclusion: surfactant use decreased mortality, was associated with more ventilator-free days and reduced the duration of ventilation. no serious adverse events were reported. acute respiratory failure remains the primary indication for admission to north american paediatric intensive care units (picus) and accounts for significant mortality, morbidity and resource utilization [ ] . respiratory infections, in particular pneumonia and severe bronchiolitis, are the most common causes of respiratory failure requiring mechanical ventilation in children [ ] . alterations in endogenous surfactant play a role in the pathogenesis of many causes of acute lung injury (ali) and acute respiratory distress syndrome (ards) [ ] . surfactant dysfunction, destruction and inactivation have also been demonstrated in children with acute respiratory insufficiency due to bronchiolitis [ , ] . the administration of exogenous surfactant may reduce the need for mechanical ventilation and its associated sequelae by restoring surfactant levels and function. inspired by the success of surfactants in reducing mortality and the need for mechanical ventilation in neonatal respiratory distress syndrome [ ] , investigators have studied exogenous surfactant in other populations with various causes of respiratory failure. trials of surfactant in adults with ali and ards have not demonstrated a mortality benefit [ ] [ ] [ ] [ ] , perhaps due to inherent differences in the aetiology of lung injury in adults, the design features of the trials, the mode and timing of surfactant administration or the type and dose of surfactant used. in children with respiratory failure, the efficacy of exogenous surfactant has been suggested in uncontrolled studies ali = acute lung injury; ards = acute respiratory distress syndrome; fio = fractional inspired oxygen; pao = arterial oxygen tension; picu = paediatric intensive care unit; rsv = respiratory syncytial virus. (page number not for citation purposes) [ , ] . the relatively low mortality rate, the diversity of the study populations and the shorter duration of mechanical ventilation are factors that make large-scale randomized controlled trials in this population challenging to conduct. two of the largest trials were stopped early due to slower than expected enrolment [ , ] . while the use of surfactant in ards/ali has not been previously systematically reviewed, its use in children with bronchiolitis has been [ ] . we anticipated that including trials enrolling children with acute respiratory failure from a variety of causes would result in a heterogeneous population and would increase the generalizability of the results. our confidence in the results of the present review would also be increased if a consistent effect is shown in subgroups and across a spectrum of disease severity. the primary objective of the systematic review is to assess the effect of the administration of pulmonary surfactant compared with no therapy or with placebo on all-cause mortality (at or before hospital discharge) in mechanically ventilated children with acute respiratory failure. we included trials that were prospective, that were randomized, that enrolled children intubated and mechanically ventilated for acute respiratory failure and that compared the intratracheal administration or nebulization of at least one dose of natural or artificial pulmonary surfactant with a placebo or no intervention. we excluded trials exclusively enrolling neonates or patients with asthma. we used the trial authors' definitions of paediatric. the primary outcome measure was all-cause mortality at or before hospital discharge. secondary outcomes were ventilator-free days to day (a composite of mortality and duration of ventilation, defined as days alive and free from mechanical ventilation) [ ] , the duration of mechanical ventilation (from intubation to extubation, death or trial withdrawal), the duration of picu stay, the use of rescue therapy (such as extracorporeal membrane oxygenation, high-frequency oscillatory ventilation, open label surfactant and nitric oxide), and complications and adverse effects as reported by the trial authors. one of us searched for published and unpublished trials, examining trial registries, conference proceedings and the bibliographies of any identified trials and relevant reviews (the search strategy is available upon request). we polled paediatric intensivists and pharmacists at our institution for additional trials. we selected search terms from the keywords and mesh terms of previous surfactant trials and from the generic and brand names of commercially available surfactants. we imposed no language restrictions. one of us screened the title (and abstract if required) of all citations retrieved. we selected citations for further evaluation if they reported the administration of at least one dose of surfactant to at least one child or if the title or abstract did not give enough information to make an assessment. two reviewers independently reviewed all citations meeting criteria for further review and applied the inclusion criteria. disagreements between reviewers were resolved by consensus in consultation with a third reviewer. we considered agreement between reviewers to be acceptable if the kappa value was greater than . . we used the following characteristics to assess the methodologic quality: allocation concealment (sealed envelopes or central randomization were considered adequate), blinding (which of the trial personnel and caregivers were blinded, and the methods used to ensure blinding), completeness of followup (assessed by the number of patients randomized for whom there were no outcomes), similarity of the groups at baseline (with respect to known prognostic factors: age, aetiology, severity of illness as measured by the pediatric risk of mortality score, and immunosuppression), whether a standard or recommended strategy for mechanical ventilation was used, and whether a priori criteria for the use of co-interventions were used. effective blinding of surfactant is challenging because of the large volumes of milky fluid administered, which can often be seen by caregivers in the patients' ventilator tubing or endotracheal tube, particularly during suctioning. we pretested and refined the developed forms on two trials of surfactant therapy for adults, and clarified definitions based on feedback from the reviewers. two reviewers then independently used these forms to abstract trial quality, blinded to the authors, the journal, the country of origin and the results. we resolved any disagreements by consensus in consultation with a third reviewer if needed. after pretesting and refining the forms on two trials of surfactant therapy in adults and clarifying definitions based on feedback from the reviewers, two reviewers then independently abstracted the data. reviewers were only provided with a full-text version of the trials from which the introduction, conclusions and discussion were omitted and from which the author, journal and country of origin were deleted. we thereafter examined these sections of the reports for any missing data. we resolved any disagreements between reviewers by consensus in consultation with a third reviewer if needed. we asked the authors to supply data not included in the published reports. two reviewers performed data entry in duplicate. we quantitatively pooled the results of individual trials when possible. we expressed the treatment effect as a relative risk for dichotomous outcomes and as a weighted mean difference for continuous outcomes with % confidence intervals. we considered effects statistically significant if p < . . a z test was used to statistically test the estimates of treatment effect between groups [ ] . we assessed heterogeneity among trials using the i statistic, and considered an i value greater than % to indicate substantial heterogeneity [ ] . revman . software and a random effects model were used to perform the analyses [ ] . we chose the random effects model because it gives a more conservative estimate of the precision of the treatment effects and because the true effect of the intervention probably varies given the different populations enrolled in these trials [ ] . a subgroup analysis was planned based on the aetiology of respiratory failure (trials enrolling exclusively patients with respiratory syncytial virus (rsv)/ severe bronchiolitis compared with all other trials) if sufficient data were available, because these trials were likely to enrol a younger, more homogeneous, population with a lower predicted risk of mortality. we also planned sensitivity analysis based on methodological features of the included trials (trials reporting adequate allocation concealment compared with all other trials). we identified unique citations, six of which met our inclusion criteria ( figure outlines the reasons for exclusion). most reports excluded enrolled neonates or were retrospective or uncontrolled in design. chance corrected agreement was excellent (kappa = . , % confidence interval = . - . ). table presents a complete description of our quality assessment. only one trial did not report allocation concealment [ ] . although effective blinding of surfactant is challenging, two trials reported blinding of the picu team [ , ] . the two flow diagram of included trials flow diagram of included trials. rcts, randomized controlled trials. groups were generally well matched in terms of baseline characteristics in most trials. the most significant imbalance was the numerically higher number of immunosuppressed patients in the placebo group. these patients had higher mortality ( %) than the immunocompetent group ( %). the authors attempted to adjust for this imbalance with logistic regression, which suggested that the treatment effect seemed to be relatively consistent between the two groups [ ] . only one trial reported a priori criteria for rescue therapy [ ] . table describes the included trials. three trials enrolled exclusively infants with rsv-induced respiratory failure [ , ] or with severe bronchiolitis [ ] . the remaining three trials enrolled a heterogeneous group of patients with ards or ali [ , , ] . while the individual treatment protocols varied, all trials used comparable doses ( - mg/kg phospholipids) of natural or modified natural surfactants and each patient typically received one or two doses. a variety of interventions were used in the control groups: no intervention, air placebo or similar sedation and ventilation manoeuvres without a placebo. although one study [ ] used a modified natural surfactant, all the products used contained surfactant proteins b and c. all studies administered surfactant early in the course of respiratory failure; most patients were treated within - hours of requiring mechanical ventilation. the baseline characteristics of the patients are presented in table . while there was significant heterogeneity among and within trials with respect to age and cause of respiratory failure, we considered the initial pediatric risk of mortality scores and the initial pao /fio ratios to be clinically comparable. mortality data were available for all six trials, randomizing patients and reporting data for patients. there were no deaths reported in the three rsv/severe bronchiolitis trials; thus our estimate is based on three trials randomizing patients, of whom died. in the pooled analysis, surfactant was associated with significantly lower mortality (relative risk = . , % confidence interval = . - . , p = . ). there was no evidence of heterogeneity (i = %) ( figure ). ventilator-free days to day the number of ventilator-free days to day was available for six trials randomizing patients and reporting data for patients. in the pooled analysis, surfactant was associated with significantly more ventilator-free days (weighted mean dif- (figure ). the duration of mechanical ventilation was available for six trials randomizing patients and reporting data for patients. in the pooled analysis, surfactant was associated with a significantly shorter duration of mechanical ventilation (weighted mean difference = . days, % confidence interval = . - . days, p = . ) (figure ) . the duration of picu stay was available for five trials randomizing patients and reporting data for patients. in the pooled analysis, surfactant was associated with a shortened duration of picu stay (weighted mean difference = . days, % confidence interval = . - . days, p = . ), but this difference was not statistically significant ( figure ). data on the use of rescue therapy were available for six trials randomizing patients and reporting data for patients. in the pooled analysis, the surfactant was associated with a significantly lower use of rescue therapy (relative risk = . , % confidence interval = . - . , p < . ). there was no evidence of heterogeneity (i = %). this summary estimate should be interpreted with caution as only one trial reported a protocol for initiating rescue therapy. the decision to use a rescue therapy, particularly an open-label surfactant, may be influenced by knowledge of the patient's allocation; furthermore, only two trials reported blinded caregivers and the methods used to ensure blinding may not be adequate. surfactant therapy was well tolerated (see table ), but only three of the trials reported any definitions or a priori criteria or of collecting adverse events [ , , ] . transient hypotension and transient hypoxia were the most commonly reported adverse events in the largest trial. these responded to a brief adjustment in ventilation, to a slowing of the rate of surfactant administration or to fluid administration. there was no difference in the incidence of air leaks in the two trials that reported this outcome. no patient was withdrawn from any of the trials because of adverse events. we did not pool the data on adverse events associated with the trial interventions from the six trials because of the inconsistent manner in which the events were documented and reported. the effect of surfactant on ventilator-free days, the duration of mechanical ventilation and the duration of picu stay was not significantly different when we compared the three trials that enrolled exclusively patients with rsv/severe bronchiolitis with the three other trials (table ) . a % survival in the bronchiolitis trials subgroup precludes formal subgroup analysis for the primary outcome of mortality. all but one of the included trials reported adequate allocation concealment (defined as sealed envelopes or central telephone randomization). since there were no deaths in this trial we could not assess the effect of inadequate allocation concealment on mortality. pooling the five remaining trials did not change the direction of the effect and did not significantly meta-analysis of trials of surfactant in children with acute respiratory failure: mortality meta-analysis of trials of surfactant in children with acute respiratory failure: mortality. ali, acute lung injury; ards, acute respiratory distress syndrome; % ci, % confidence interval; rr, relative risk; rsv, respiratory syncytial virus. meta-analysis of trials of surfactant in children with acute respiratory failure: ventilator-free days meta-analysis of trials of surfactant in children with acute respiratory failure: ventilator-free days. ali, acute lung injury; ards, acute respiratory distress syndrome; % ci, % confidence interval; rsv, respiratory syncytial virus; sd, standard deviation; wmd, weighted mean difference. change the point estimates for the secondary outcomes of ventilator-free days, duration of ventilation or duration of picu stay (table ). in the present systematic review and meta-analysis of the effect of surfactant for critically ill children with acute respiratory failure we found that surfactant therapy significantly reduced our primary outcome of mortality. surfactant was associated with more ventilator-free days, with decreased duration of ventilation and with less use of rescue therapy as compared with standard therapy. there was no significant difference in the duration of picu stay. surfactant therapy was well tolerated; while transient hypoxia and hypotension were reported during surfactant administration, no study reported any serious adverse events. the patients enrolled in these trials are representative of the heterogeneous group of children with early, severe acute respiratory failure that is seen in clinical practice. these patients had similar severity of illness scores and a similar degree of respiratory failure (as measured by pediatric risk of mortality scores and pao :fio ratios). the heterogeneity of results for our primary outcome of mortality was low. the presence of significant heterogeneity reduces the strength of inferences we can make regarding the effect of surfactant on the secondary outcomes of ventilator-free days, meta-analysis of trials of surfactant in children with acute respiratory failure: duration of mechanical ventilation meta-analysis of trials of surfactant in children with acute respiratory failure: duration of mechanical ventilation. ali, acute lung injury; ards, acute respiratory distress syndrome; % ci, % confidence interval; rsv, respiratory syncytial virus; sd, standard deviation; wmd, weighted mean difference. meta-analysis of trials of surfactant in children with acute respiratory failure: duration of picu stay meta-analysis of trials of surfactant in children with acute respiratory failure: duration of picu stay. ali, acute lung injury; ards, acute respiratory distress syndrome; % ci, % confidence interval; picu, paediatric intensive care unit; rsv, respiratory syncytial virus; sd, standard deviation; wmd, weighted mean difference. duration of ventilation and duration of picu stay. separately pooling the trials that exclusively enrolled patients with rsv/ severe bronchiolitis and those enrolling patients with ards/ ali from a variety of causes did not significantly reduce the heterogeneity. changing ventilation strategies and the use of a variety of natural and modified natural surfactants may have increased the heterogeneity of our results. ventilation strategies, such as the use of lower tidal volumes and earlier use of high-frequency oscillatory ventilation, have evolved significantly in the -year span over which the included trials were conducted [ ] [ ] [ ] . the surfactants used in the included trials were all natural or modified natural surfactants; however, these surfactants may have slightly different effects on oxygenation and compliance due to the differences in phospholipid and surfactant protein composition, which may have influenced individual study results. the strengths of the present review include a comprehensive search strategy, broad inclusion criteria (resulting in a representative, heterogeneous population) and abstraction of clinically important outcomes in duplicate, independently blinded to information that may bias evaluation. the strength of the inference we can make from our subgroup analysis is limited because we were unable to extract all subgroup data from these trials. access to individual patient data would allow better examination of the treatment effect in subgroups of patients and would facilitate further exploration of possible causes of heterogeneity. we found that mortality was very different between the trials that exclusively enrolled patients with rsv/severe bronchiolitis and those that enrolled patients with ards/ali from a variety of causes. we pooled the results because both conditions result in abnormal surfactant function and because of the substantial overlap between the two groups; up to % of children in the ards/ali trials had rsv and up to % of the children in some bronchiolitis studies also had pneumonia. the reduction in mortality and the increased ventilator-free days have important implications as very few trials in paediatric critical care suggest a favourable impact on mortality [ ] . the present review suggests that surfactant could be an important adjunct in the management of paediatric respiratory failure. uncertainty exists, however, about the reproducibility of treatment effects generated from relatively small unblinded trials; questions remain about adverse affects, which may be undetected or under-reported in this literature. also, a large proportion of patients and events are reported in one trial [ ] . furthermore, issues of the optimal dose and the timing of administration, and which patients are most likely to derive benefit, should be studied in further adequately powered multicentre trials. the pediatric acute lung injury and sepsis investigators network is planning a large rigorous randomized trial enrolling children with acute hypoxemic respiratory failure to address these issues. surfactant use decreased mortality, was associated with more ventilator-free days and reduced the duration of ventilation. no serious adverse events were reported. most trials enrolled small numbers of children, and further well-designed and adequately powered multicentre trials are therefore required. • surfactant decreased mortality in a heterogeneous population of children with acute respiratory failure. • surfactant was associated with more ventilator-free days and a reduced duration of ventilation. • no serious adverse events were reported. • further well-designed and adequately powered multicentre trials are required. the feasibility of conducting clinical trials in infants and children with acute respiratory failure adult respiratory distress syndrome in pediatric patients. i. clinical aspects, pathophysiology, pathology, and mechanisms of lung injury harwood jl: abnormal surfactant composition and activity in severe bronchiolitis surfactant abnormalities in infants with severe viral bronchiolitis a controlled trial of synthetic surfactant in infants weighing g or more with respiratory distress syndrome. the american exosurf neonatal study group i, and the canadian exosurf neonatal study group aerosolized surfactant in adults with sepsis-induced acute respiratory distress syndrome. exosurf acute respiratory distress syndrome sepsis study group bovine surfactant therapy for patients with acute respiratory distress syndrome treatment of acute respiratory distress syndrome with recombinant surfactant protein c surfactant safety and potential efficacy of an aerosolized surfactant in human sepsis-induced adult respiratory distress syndrome exogenous surfactant therapy for pediatric patients with the acute respiratory distress syndrome calf's lung surfactant extract in acute hypoxemic respiratory failure in children effect of exogenous surfactant (calfactant) in pediatric acute lung injury: a randomized controlled trial treatment with bovine surfactant in severe acute respiratory distress syndrome in children: a randomized multicenter study efficacy of interventions for bronchiolitis in critically ill infants: a systematic review and meta-analysis statistical evaluation of ventilator-free days as an efficacy measure in clinical trials of treatments for acute respiratory distress syndrome the statistical basis of meta-analysis quantifying heterogeneity in a meta-analysis copenhagen: the nordic cochrane centre, the cochrane collaboration meta-analysis in clinical trials exogenous surfactant supplementation in infants with respiratory syncytial virus bronchiolitis marraro g: multicenter, randomized, controlled study of porcine surfactant in severe respiratory syncytial virus-induced respiratory failure porcine-derived surfactant treatment of severe bronchiolitis instillation of calf lung surfactant extract (calfactant) is beneficial in pediatric acute hypoxemic respiratory failure treatment with bovine surfactant in severe acute respiratory distress syndrome in children: a randomized multicenter study the acute respiratory distress syndrome network: ventilation with lower tidal volumes as compared with traditional tidal volumes for acute lung injury and the acute respiratory distress syndrome high-frequency oscillatory ventilation in pediatric respiratory failure: a multicenter experience prospective, randomized comparison of high-frequency oscillatory ventilation and conventional mechanical ventilation in pediatric respiratory failure alternative outcome measures for pediatric clinical sepsis trials the authors would like to acknowledge the authors of the primary trials (dr marco luchetti, dr jens möller, dr shane tibby and dr douglas willson) for providing additional information or clarification. thanks to john duffett for reviewing the citations and for data entry. the authors declare that they have no competing interests. md conceived of this review. md, kc, vn and djc participated in the design. md and vn extracted data and assessed the quality of the included studies. md, kc, djc and ar helped to draft the manuscript. all authors read and approved the final manuscript. the following additional files are available online: a word file containing a table listing individual trial inclusion criteria and exclusion criteria. see http://www.biomedcentral.com/content/ supplementary/cc -s .doc key: cord- - li v j authors: felix, carol title: clinical research in the time of covid- date: - - journal: int j radiat oncol biol phys doi: . /j.ijrobp. . . sha: doc_id: cord_uid: li v j nan i am writing about what i knowdclinical trials. that said, i think you could substitute "clinical practice" anywhere you see "research" or "clinical trials" in this essay. the covid- pandemic is unprecedented. our response to it is rapidly evolving as we learn more about the pathophysiology of this virus, how to mitigate its effects on public health, and our ability to adjust to new cultural norms. ultimately, a new normal will be established, and this pandemic, in its acute form, will end. but cancers will remain. necessary cancer treatments, such as radiation, will remain. the need for clinical trials, the improved treatments they elucidate, and the us food and drug administration (fda) approval providing public access to these new and better treatments will remain. the pandemic does not end medical research, and it may, in fact, point us in a new and better direction for implementing clinical trials. research is an important aspect of any academic institution. research productivity is a cornerstone of faculty advancement and promotion. this pandemic has already had a huge human and economic toll, globally stopping us in our tracks. it has greatly affected medicine in general, and disrupted new and ongoing clinical trials, prompting the fda in mid-march to issue new guidance for industry, investigators, and institutional review boards. academic institutions will do robust research again, and those institutions most prepared to pick up research at prepandemic levels will be those that follow fda guidelines to facilitate and continue running clinical trials even during the spread of covid- . medical research necessitates collecting large amounts of clinical and other data over a long period of time to determine the safety and efficacy of a given regimen. a whitepaper published in march by science , a pioneer in virtual clinical trials, estimates that there are currently more than " , ongoing clinical trials made up of thousands of trial sites, tens of thousands of investigators and hundreds of thousands of study subjects." furthermore, billions of dollars are invested into these trials, which are critical for bringing new and improved treatments to the public. thus, there is a real need for continuity in medical research. until an effective vaccine for covid- is established, consider that we will likely need to keep in place new social norms such as social distancing and wearing masks. packed clinics and crowded boardrooms will be out of the question. we will also need to consider the psychological effectsdthe fear this virus createsdon our staff and especially our patients. so, how do we run clinical trials in the time of covid- ? the model for doing this can be taken piecewise from the clinical trials network model that we have been building over the last years in the ucla department of radiation oncology. our network demands the ability to remotely conduct and monitor research at participating sites for any specific protocol. to do this virtual or remote research, we need a system that can provide the following , : these virtual research methods combined with mobile phlebotomy services and, when feasible, shipment of supplies directly to patients, make research accessible in the time of covid- and may be better for patients as well. often, research participants are difficult to recruit into trials, citing the inconvenience of coming to ucla for multiple visits. research participants who do consent to participate in trials frequently note the commute time, traffic, costs of parking, and inconvenience of coming to westwood. due to this pandemic, many patients fear (with good reason) coming in at all. research participants, especially those who live further away or even in other states, regularly ask if they can do research procedures near their homes. in creating our nascent network, we have started to implement many of the tools necessary for virtual research, but there is more to do. for the last year, we have been working with ucla's jonsson comprehensive cancer center, investigating essential clinical trials management systems that we hope will be rolled out in the near future. covid- certainly ramps up the demand for it. in addition to having this clinical trials management system for running remote and multisite clinical trials, changes will likely need to be made at a higher level: in staffing. adaptation is the key to success in any evolution. although going back to where we were might seem most comfortable, it should not be the goal. new, smarter, more efficient, more cost-effective, and cleaner solutions will be the way forward. implementing virtual visits and staggering work schedules so that fewer people are in the workplace at any given time seem like obvious solutions as a way to some normalcy. on may , uri alon and ron milo, both professors of computational and systems biology at the wiezmann institute of science in israel, collaborated with eran yashiv, a professor of economics at tel aviv university and the london school of economics, to write a compelling piece for the new york times entitled " - : how to reopen the economy by exploiting the coronavirus's weak spot." the virus's weak spot in the title refers to is its latency perioddon average, there is a -day delay from when a person is first infected to the time that person can infect others. these authors suggest . splitting staff into working groups that alternate shifts; and . working in -week cycles of at work for days and remote work from home for days. as transmission rates drop or increase, the balance of atwork days and at-home days can be adjusted. the benefits of implementing this model include the following: . it predicts the viral contractibility score will reduce to r < . . it reduces the number of staff in the department at any given time, making social distancing easier and curtailing transmission. . it allows sustainable clinical and economic activity. the model works regardless of the accessibility or even accuracy of testing. it is based simply on what we know about the virus' latency period and on social distancing and stay-at-home measures that have proven efficacy. a more hidden benefit is that this model prevents the economic and psychological costs that would be inevitable if research, or any business for that matter, had to regularly reinstate and then shut down every time covid- cases resurged. this viral pandemic is daunting, but there is a way forward. until a vaccine is developed, and likely even after that if covid- becomes a seasonal norm, we will have to weigh the risks and benefits of everything we do, including medical visits and research appointments. however, the risk can be mitigated. intelligent staff scheduling that takes into consideration what we know about the virus' transmission, combined with virtual research and clinical capabilities, at least in part gives us a path for conducting clinical trialsdand perhaps medicine in generaldthrough this health, economic, and social crisis. fda guidance on conduct of clinical trials of medical products during covid- public health emergency: guidance for industry, investigators, and institutional review boards navigating clinical trials in the coronavirus era board on health sciences policy; forum on drug discovery, development, and translation - : how to reopen the economy by exploiting the coronavirus's weak spot key: cord- -wcmc mh authors: rhodus, elizabeth k.; bardach, shoshana h.; abner, erin l.; gibson, allison; jicha, gregory a. title: covid- and geriatric clinical trials research date: - - journal: aging clin exp res doi: . /s - - -x sha: doc_id: cord_uid: wcmc mh nan the sars-cov- pandemic has wrought tremendous upheaval to daily life, spreading rapidly and causing millions to contract covid- . the long-term effects are yet unknown, but scientists are studying the disease and searching for solutions. in geriatric clinical trials research, the problems presented by covid- are multifaceted: older adults face increased risk of morbidity and mortality due to covid- , and so ongoing clinical trials that enroll geriatric participants have been disrupted, appropriately so, in light of these increased risks [ ] . yet, the interruption of ongoing trials may further increase risks to older adults as critical research on treatments for highly prevalent diseases and conditions is slowed or stopped. further, older adults are substantially underrepresented in clinical trials research, and this situation may worsen this discrepancy. scientists in aging appreciate the necessity of older adults' inclusion within clinical trials, but the vulnerability to increased exclusion in clinical trials of this population is high, now more than ever. attention to clinical trial development with special attention to need for inclusion of older adults and precautions is greatly needed to sustain current efforts, at minimum, and ideally, enhance recognition of the value of including older adults in clinical trial research. the covid- crisis affects every aspect of clinical trial research engagement including: recruitment and retention; ability to ensure participant safety while engaged in experimental interventions; study procedures, including consideration of remote assessments, impact on populations with health disparities, and generalizability of future results; outcome measures, including biomarker assessment; impact on the clinical trial workforce, including attrition; impact on dissemination of results and scientific collaborations, which move the clinical trial infrastructure forward; current and future funding allocations; and regulatory considerations in regards to management of altered study conduct and change of outcome measures (fig. ) . the purpose of this article is to highlight the impact of disasters such as the covid- pandemic on geriatric clinical trials research and propose approaches for the scientific community to continue pushing forward. disruptions due to disaster can often result in disruptions to study protocols, challenges to data interpretation, the need for institutional review board (irb) amendments, and the introduction of new biases [ ] . prior research highlights the importance of proactive plans to minimize study disruption, the need to adapt rapidly to changing constraints, and the importance of ongoing communication with contract research organizations and sponsors and to share and develop strategies to mitigate participant-related protocol deviations and violations. application of these actions should all occur in the context of local conditions. researchers have not previously seen this global level of disruption to clinical trials. effects of covid- on geriatric clinical trials research will be long-lasting. trials in their enrollment phase will at a minimum be delayed in achieving full recruitment, leading to lags in, or losses of, discovery. depending on the duration of stay-at-home orders, some studies may be permanently disrupted and discontinued. for more established trials, if attrition leads to final sample sizes much smaller than those required by the a priori power estimates, then these ongoing studies may not have enough statistical power to identify positive findings. trials that involve in-person cognitive assessment face challenges as they move to other modalities for testing, which may influence results. efforts to validate existing measures through remote assessment should continue, to enable meaningful cognitive measures to be assessed remotely. in addition, investment in low cost technology to support these efforts, for participants who may not have remote access, may enable studies to continue with limited disruption, while also ensuring those of lower socioeconomic status are not left behind. clinical trials involving those with cognitive impairment face unique challenges during such crises [ ] . these individuals depend on routine for optimal functioning, as well as access to health care, and supportive community engagement. at a time of social distancing, routines and support systems are dramatically altered. social cohesion and community can offer protections and resiliency for older adults during the time of disaster management. maintaining and supporting active participant and recruitment engagement is essential. for example, the addition of virtual connection, training, and additional contact may prove highly beneficial. efforts to maintain engagement will need to be complemented by efforts to evaluate safety and efficacy. many of the existing safety and efficacy measures within current trials are coming to a halt or being offered through non-traditional means, with unknown validity. individuals continuing to take experimental medications may continue to have checkin safety calls but may have delays in bloodwork and imaging scans that would otherwise provide reassurance regarding any potential adverse medication effects. for infusion studies, medication delivery may be halted, but lingering effects of existing dosages are still possible. careful evaluation of potential drug effects and critical safety measures will optimize the likelihood that safety is not compromised. behavioral studies have additional challenges due to the prolonged proximity of participants and researchers required in some intervention. while the impact of abrupt halting of these trials is yet to be known, maintaining vigilance while planning to resume is essential. there is need for attention to implement gradual, careful restart processes that aim to keep participants safe while minimizing disruption to trials. assessing efficacy will be more complex, as the evaluation will necessitate consideration of history effects within studies including the upheaval to daily routines and changes in cognitive and social stimulation that covid- has created. methodological counterpoints will need to be considered including the role of randomization in distributing the psychosocial effects across groups, the timing of enrollment within the pandemic onset, and testing or controlling for psychological sequelae of covid- , such as controlling for depressive symptoms or the impact of social isolation and loneliness. further, as noted above, changing test administration from in-person to digital modalities that can be conducted remotely out of the office will introduce additional variability into efficacy assessments. in the us, morbidity and mortality early in the covid- pandemic has disproportionately affected older adults and black/african-american and hispanic-american communities [ ] . there are often perceptions in these communities, which arose through a long history of mistreatment by biomedical researchers and healthcare providers, that healthcare offers assistance but also potentially contributes to individual and community harm. if healthcare centers are seen as a focal point for contagion, engagement that requires in-person visits to healthcare facilities may be problematic. existing barriers to engaging hard to reach participants will likely be exacerbated. clinical discoveries should benefit everyone, and diversity and inclusiveness improve the generalizability of clinical trial findings. we must make concerted efforts in recruitment and retention to ensure advancements we have made in regard to increasing the representative population sample engaged in clinical trial research are sustained. recent past and anticipated future successes in clinical trial activity in the area of aging and dementia have relied on biomarker assessments including: brain imaging (mri, pet and other), which requires shared and repetitive usage of common acquisition facilities; retinal imaging and eeg, which require the use of shared equipment that may be difficult to sterilize or sanitize between participants. biomarker outcomes are an increasingly important part of clinical trial research [ ] . discovery of disease-modifying therapies in the area of dementia research, especially preclinical dementia, is dependent on such measures. coordination of using shared medical facilities such as imaging equipment including mri and pet scanners for continued conduct has been a challenge due to concerns of covid- transmission, despite the critical need for the acquisition of essential imaging and safety information mandated by participant status and clinical trial conduct. development of specific and diligent methods to sterilize, sanitize, and convey these safety precautions to the public will allow continued engagement in clinical research activities. as with prior disasters, many research sites in the us have been forced to shut down and/or limit activities as a result of local or state restrictions due to covid- . in addition, site staff in many areas of the us are under 'remote work' or 'stay at home' orders that limit access to their workspace. thus, critical research activity that supports the financial infrastructure of these research sites may be diminished and/ or completely disappear. it may not be possible for some investigators to continue to provide salary support for their well-trained research staff members, not only leading to dissolution of the existing research site infrastructure but also contributing further to the ongoing societal burden of the covid- crisis. such impact may also impede advancement of researchers' careers. the need for the cultivation and maintenance of a workforce trained for enhancing older adults' engagement in clinical trial activities has been overlooked for years. hiring and retaining qualified and experienced study coordinators, recruiters, psychometrists, and even phlebotomists in the geriatric research setting can be extremely difficult. developing such skill sets and desires to work in research may be severely limited by the covid- crisis unless we are able to ensure stability of the underlying research infrastructure. otherwise, we will lose many preciously talented individuals who have entered the field of geriatric clinical research. the call for advocacy is needed for sustained funding mechanisms that offer emergency support and stable institutional standing to ensure continuity and protection of these positions. given the heightened public interest in advancing scientific and medical initiatives to combat the covid- crisis, as well as the financial burden the crisis is posing globally, there is increasing competition for resources that have long been lacking in the field. funding increases for agingrelated research were moving at a rapid pace but funds now may be in jeopardy with this competing need. large-scale funding from the nih is being redirected to directly combat the covid- pandemic. while immediate repercussions on the budget for national institute on aging has not yet occurred, we hope that specific aging-related research continues to receive the funds necessary to move the field forward. representation and advocacy for aging science by stakeholders is imperative to protect current availability of geriatric clinical trial research. as scientists consider lasting implications of covid- , future study design planning and management will likely be altered. specifically, increased utilization of participantcentered design while maintaining rigor and reproducibility is crucial. study design that promotes engagement in a way that meets older adults' unique needs with enhanced ease in access will be needed. current and ongoing recruitment avenues and disaster preparedness plans must acknowledge ways to mitigate decreased contact with older participants who do not use email or computers. further, care for participants that typically occurs during clinical trials under the supervision of medical professionals will need to be adjusted to maintain safety. development, maintenance, and implementation of disaster and business continuity plans within clinical trial design is needed in future planning. mische and wilkerson [ ] describe the need for preparation using risk analyses, response during disasters with implementation of essential functions, recovery efforts, and mitigation of losses to clinical trial data and resources. clinical trial continuity for interventions and care of older adults, especially those with cognitive impairment, is paramount. the vulnerability of older adults to covid- is a critical reminder for the need to prepare for disasters during clinical trial design. future changes in regulatory and industry perspectives on trial conduct could be profound. new trial design methodologies will need to be developed to take into account significant periods of time for subpopulation and/or total population of subjects that may not be able to conduct regular research activities. our patient population regularly encounters unexpected healthcare, social, and/or personal crises that are considered significant disruptions to smooth research conduct. the potential for increasing recognition of such confounds in interpreting clinical trial data could help to smooth regulatory pathways for studies that encounter such obstacles in the future. scientists face unprecedented challenges with long-lasting impacts due to covid- . by learning from prior disasters and epidemics, we can be prepared to address current and upcoming changes. the resilience and ability of researchers to move science forward for the success of advanced care for older adults will have benefits for decades to come. increased acknowledgement of a variety of social determinants of health will allow improved efforts to ensure innovative analytic approaches, as well as structures to support the research workforce and opportunities that keep research dissemination and scientific discovery active and productive. covid- in older adults when disasters strike: navigating the challenges of "sudden science virus is twice as deadly for black and latino people than whites in n.y.c the national institute on aging-alzheimer's association framework on alzheimer's disease: application to clinical trials disaster and contingency planning for scientific shared resource cores authors' contribution all authors contributed to the design, writing, and revision of the resulting manuscript. authors thank matt hazzard and tom dolan, medical illustrators, university of kentucky-college of medicine for visual contributions.funding the first author is funded by nih t ag : "training in translational research in alzheimer's and related dementias (triad)". all authors are affiliated with university of kentucky alzheimer's disease cohort participants funded by nih/nia p ag . availability of data and material is not applicable. conflict of interest the authors declare that they have no conflict of interest. this article does not contain any studies with human participants or animals performed by any of the authors.informed consent for this type of article, formal consent is not required. key: cord- -j ruj authors: dreyfuss, didier title: is it better to consent to an rct or to care?: Μηδεν αγαν (“nothing in excess”) date: - - journal: intensive care med doi: . /s - - - sha: doc_id: cord_uid: j ruj nan twenty-five years ago the belmont report [ ] established a formal distinction between care and research in order to protect patients. the legitimate fear that research might be conducted under the pretence of medical care regardless of whether this increased the risk to patients drove this effort toward clarification. a sound ethical foundation to medical research was deemed essential after the heinous nazi experiments and the abhorrent -year-long tuskegee syphilis study [ ] . at the time that the belmont report was issued randomized clinical trials (rcts) were coming into vogue. an rct is not routine clinical care, even when the treatments in both study arms are consonant with standard practice. thus both the ability of rcts to produce high-quality medical knowledge and the distinction between care and research were considered of paramount importance at the time of the belmont report [ ] . whether research should be incorporated into medical care remains controversial, however, as reflected by several recent and conflicting statements of opinion [ , , ] . i argue here that things have changed since the belmont report, and that a formal distinction between care and research may no longer serve medical or ethical principles in many situations, most notably in the area of critical care. this seemingly provocative position stems from three arguments. first, a large part of the "research" conducted in icu patients consists in evaluating practices or comparing two widely used procedures or treatments [ ] . second, the scientific value and the historical role of rcts may have been overemphasized and their socioeconomic impact misinterpreted. third, analysis of the informed consent process, a mandatory preliminary intended to ensure the autonomy of patients included in rcts, suggests that the cure may be worse than the disease. although informed consent was a major stride towards protecting the rights and dignity of patients, the process may in some instances be perverted into protecting the physicians rather than the patients. the tragic jesse gellsinger case ("teen dies undergoing experimental gene therapy," washington post, september ) reminds us that "informed consent" is not sufficient to protect patients during research [ ] . the informed consent process has many flaws stemming from the frequent complexity of consent forms [ ] , poor comprehension of information by many families who must draw on their limited scientific culture to unravel complicated issues at a time when they are struggling with severe anxiety about their loved one [ ] , and dearth of communication skills among physicians. thus, informed consent to research may be stressful for families and reassuring for physicians. this was not its original purpose. to overcome these problems a reform allowing informed consent to be waived under specific circumstances has been suggested [ ] . although the opinions expressed by the advocates of this approach are worthy of respect, they conflict with the principle of patient autonomy [ ] . a similar wish is expressed by some european physicians who seem very anxious about the putative threat on medical research in emergency that a new european directive supposedly poses [ , , ] . indeed, this directive [ ] does not allow the waiving of consent by proxies. another contention of the present paper is this [ ] : critical care physicians may still believe that rcts remain the best tool for improving knowledge and care, and in this case they must accept to use the means needed to achieve the end and therefore to insist on mandatory informed consent from the patient or proxy; or they may realize that the game is not worth the candle and they must then turn to other forms of research that are ranked less highly in the pyramid of evidence-based medicine [ ] . in so doing they give priority to the well-being of families and must seek the help of innovative methodologists who accept to deal with the real world. before discussing the problem of informed consent to research a critical appraisal of the scientific and ethical validity of rcts in critical care medicine is in order. the role of rcts was overemphasized from the outset many critical care physicians believe in the superiority of rcts over other types of clinical studies. this belief may not be supported by the evidence. before discussing objective data on the value of rcts in critically ill patients it is important to recall a number of historical facts. rcts emerged in the s as a tool for overcoming the drawbacks of anecdotal experience. although rcts initially met with considerable resistance, they undoubtedly contributed to making clinicians aware of the need for rigorous methods. however, the data obtained proved only that rcts are feasible, not that they are superior over other forms of well-conducted studies. this is best illustrated by the trials of streptomycin in tuberculosis in the united kingdom and of the salk polio vaccine in the united states. streptomycin was discovered in in the united states. the british government could not afford to purchase streptomycin for all patients with tuberculosis, and the effectiveness of the drug was not yet established. the words of d'arcy hart [ ] , a member of the medical research council scientific staff that conducted the streptomycin trial, are worth quoting: "the trial proceeded from . the small amount of streptomycin available made it ethically permissible for the control subjects to be untreated by the drug-a statistician's dream." this gives the two reasons for the trial: fair allocation of a limited resource and the determination of a brilliant statistician, sir austin bradford hill, to prove the validity of rcts. d'arcy hart [ ] further stated: "secondly, the trial heralded the general conversion of clinical scientists to randomisation." shortly after this brilliant demonstration of the feasibility of rcts the salk polio vaccine trial was interpreted as a triumph of the new rct method in the united states. the unique circumstances surrounding the vaccine trial deserve mention [ ] : there was a raging scientific controversy between salk and sabin regarding the type of vaccine that should be developed (salk was developing a killed virus vaccine, whereas sabin thought that only an attenuated living strain could be a suitable vaccine), and salk was under considerable pressure to perform the trial, both to stop the polio epidemic that was costing so many lives among children and to win the race against sabin. for the trial children in a number of states were randomly allocated to receive the vaccine or a placebo. the results proved the vaccine dramatically effective. this both improved medical care and propelled the rct to prominence as the gold standard design for clinical research. however, an observational study was conducted in states that refused the placebo-controlled trial [ ] . the results were similar to those of the rct, but only the results from the rct were considered statistically valid [ ] . interestingly, salk described the placebo-controlled study of his vaccine as "a beautiful... experiment over which the epidemiologist could become quite ecstatic but (which) would make the humanitarian shudder" [ ] . it seems that this rct was driven by statisticians rather than by clinicians, in order to counter sabin's criticism and gain the support of the leaders of the medical community [ ] . in brief, rcts were performed because they could be performed, not because they had been proved superior over other study designs. the same may still hold true in critical care research today. what is the socioeconomic role of rcts? henry k. beecher, a professor in anesthesia research at the massachusetts general hospital commenting years ago on the huge increase in funds for research, said "there is reason to fear that... these resources may be greater than the supply of responsible investigators," and "every young man knows that he will never be promoted to a tenure post, to a professorship in a major medical school, unless he has proved himself as an investigator" [ ] . these two sentences acknowledge an uncomfortable reality: money and careers are at the center of clinical research. the importance of rcts for drug companies large rcts are vital for drug companies. these trials are mandatory not only for proving the efficacy of new products but also for obtaining regulatory approval and marketing licenses for drugs. thus drug companies have good reason to argue that rcts offer unequalled methodological purity. however, drug company sponsored trials of both hematology treatments and nonsteroidal anti-inflammatory drugs usually showed superiority of the new drug [ ] . this finding obviously violates the principle of equipoise, under which one would expect only one-half of the studies to find better outcomes with the new drug [ ] . a similar bias has been reported with other studies funded by the pharmaceutical industry and ascribed to the use of an inappropriate comparator or to publication bias [ ] . the importance of rcts for medical journals although the editors of major medical journals have recently warned against the threat to objectivity posed by some forms of industry-sponsored research [ ] and stated that "the use of clinical trials primarily for marketing makes a mockery of clinical investigation," the potential for disseminating pharmacological breakthroughs makes them likely to accept most of the industry-sponsored trials. the recent controversy on the efficacy and safety of activated protein c in sepsis highlights the difficulties faced by editors in this area [ , ] . the ties that link drug companies to investigators and to prestigious academic centers pose a worrisome threat to academic medicine [ ] . money from advertising may also weigh on editorial policies. methodologists and statisticians are consulted by drug companies and by independent investigators at the studydesign stage and by editors at the peer-review stage. they developed the rules of "methodological validity," and these rules are likely to be the same at each step of the design and publication of a clinical trial. the grading system for the quality of evidence from clinical research, with rcts at the top, is akin to a self-fulfilling prophecy: young researchers and renowned experts alike comply with these "golden" rules to ensure publication of their findings in a prestigious journal. this compliance with artificial rules is taken as firm evidence of validity, thus spinning the wheel endlessly. as stated by knottnerus and dinant [ ] , "finally, in using strict criteria in reviewing manuscripts for publication, we should worry about risk avoidance by clinical researchers. they might focus their energies on topics where the methodological criteria of reviewers and editors can be most easily met, rather than studying real life clinical problems which present substantial methodological problems." the adage "publish or perish" still applies. all these interests shared by drug companies, investigators, methodologists, and journals concur both to overproduction of rcts and to overestimation of their contribution to medical progress. rcts tend to become an uncontrolled activity driven by forces foreign to scientific goals. this results in an inextricable tangle of so-called evidence. then, meta-analyses are performed, supposedly to clarify an issue that has been artificially obscured. they may merely add to the confusion, as discussed below. how useful are rcts in critically ill patients? the validity of conclusions of earlier studies on hepatitis and cirrhosis has been evaluated by poynard and coworkers [ ] under the provocative title of "truth survival in clinical research: an evidence-based requiem." in this study the -year survival rate of conclusions derived from meta-analyses was lower ( %) than that from nonrandomized studies ( %) or rcts ( %). more importantly, the truth survival rate was similar for studies of high and low methodological quality. examination of three important areas of critical care further indicates that challenging the usefulness of rcts is not necessarily sacrilegious. the acrimony of the debate on these three topics in medical journals and at international meetings is a strong indicator that rcts fail to provide the "definitive" answer expected from them. these three topics are mechanical ventilation in adult respiratory distress syndrome (ards), selective digestive decontamination (sdd), and prevention of gastrointestinal bleeding. controversy erupted after publication of the findings of the ards network study on tidal volume reduction during ards. although this remarkable work showed that a low tidal volume of ml/kg predicted body weight resulted in better outcomes than a higher tidal volume of ml/kg [ ] , it did not tell us how to ventilate these patients. decreases in mortality [ ] and tidal volume [ ] over time occurred well before the study was initiated. patients are usually ventilated with tidal volumes that are intermediate between the two arms of the ards network trial (and probably closer to the lower volume). there is no rct telling us whether ml/kg is better than the - ml/kg generally reported in international surveys [ , ] . the results of this trial will encourage clinicians to use smaller tidal volumes, a practice that has not yet gained sufficient acceptance [ , , ] . in that sense this rct will prove useful, but not more useful than the earlier experimental, physiological, and nonrandomized clinical trials that resulted in the use of gradually decreasing tidal volumes over time [ ] . it is merely one more brick in the wall, and not a gold one. similarly, the results of the recent alveoli randomized study that compared two peep levels will probably not change current practice [ ] . in this study peep levels higher than the moderate values used in most surveys [ , ] did not improve patient mortality. finally, will the lack of effect of prone positioning on mortality in rcts [ ] discourage clinicians from using this very inexpensive and effective maneuver to improve patient oxygenation, and will these clinicians continue to await an rct providing "proof" of efficacy [ ] ? ards mortality rates have decreased substantially over the years [ ] and are probably declining further still, yet this improvement is ascribable not to rcts but to patient-oriented research based on sound physiological thinking [ , , , ] . selective digestive decontamination meta-analyses of rcts indicate a clear survival advantage with sdd [ , ] . however, sdd is seldom used because of the legitimate fear that this practice may promote the emergence of resistant bacteria [ , ] . how many studies will be needed to convince clinicians to use a method they do not want to use? or shall we wait until the "final" meta-analysis is "negative" and "proves" that clinicians were right when they refused to use sdd despite the accumulation of so-called evidence? few fields in critical care have generated as many rcts and meta-analyses. a recent meta-analysis concluded that sucralfate and ranitidine failed to prevent gastrointestinal bleeding in critically ill patients [ ] . the authors noted that a previous meta-analysis [ ] found reduced bleeding rates with h antagonists but included several positive trials of cimetidine, which has since then been superseded by drugs with better safety profiles. in addition, the use of proton-pump inhibitors seems to be increasing in critically ill patients, although there is no proof that this practice is beneficial. finally, no one knows whether prophylaxis should be given, and this uncertainty has recently been exacerbated by an observational study in , patients showing no difference in bleeding rates between a cohort of patients given prophylaxis and a subsequent cohort not given prophylaxis at the same institution [ ] . additional rcts may be needed if the obsessive goal is to discover the illusive "truth," but their drawbacks should be weighed against their utility. this paper does not claim that all rcts are useless in critically ill patients. rcts may be helpful for evaluating a single and simple intervention (even if this intervention is technologically sophisticated) in patients with a welldefined disease. this is obviously the case for acute coronary syndromes. however, many conditions seen in icu patients stem from extraordinarily complex pathophysiological mechanisms that preclude simple trial designs and interpretations [ ] . a typical example is the patient with ards and septic shock, multiple indwelling catheters, and a high risk of nosocomial respiratory and systemic infections. it is difficult to conceive of a single therapeutic intervention capable of improving such a complex situation. a further obstacle to studies on such an intervention is the highly heterogeneous nature of the icu population. these issues relate to the internal and external validity of a trial. methodologists seek to maximize the internal validity of rcts to decrease the effects of confounders. however, as internal validity increases, external validity (i.e., generalizability) decreases [ ] . this problem may exist for the ards network trial on tidal volume reduction [ ] . indeed, because extremely stringent inclusion criteria were used, only % of ards patients admitted to the participating centers were included in the trial [ ] . what ethical problems do rcts raise in critically ill patients? some rcts may conflict with currently accepted principles of medical ethics [ ] : beneficence, nonmaleficence, autonomy, and justice [ , ] . in addition, rcts may conflict with the principle that what is not scientific is not ethical. it is of course difficult if not impossible to determine a priori that a research protocol has a favorable risk-benefit ratio. however, rcts should rest on a foundation of strong experimental or clinical concepts. this may not have been the case in all instances, most notably in studies of new treatments for sepsis [ ] . without seeking to fuel the debate on the failure of antimediator agents in sepsis, one cannot but wonder whether the huge financial and academic stakes were in part responsible for the apparent haste with which some trials were conducted. in addition, the quality of research oversight in several trials with high mortality rates has been challenged [ ] . it is difficult to ensure that the prerequisites for beneficence and nonmaleficence are met when there is a major influence of financial incentives and academic competition. thus we still encounter problems similar to those met by the salk vaccine trial: the process of virus inactivation was not fully mastered at all the vaccine production sites, and therefore active virus was inoculated into a number of children in whom poliomyelitis developed (http:// www.pbs.org/wgbh/aso/databank/entries/dm sa.html, accessed july ; http://www.polio-vaccine.com/fr/ histoire/vaccins_experience.html, accessed july ). another aspect of beneficence and nonmaleficence that does not seem well addressed during the conduct of rcts concerns proxies and are discussed below. most critically ill patients are too ill to deal with issues of consent. consent is therefore sought from a surrogate in the united states and most european countries. consent by a surrogate is usually considered the best means of protecting patients during research [ , ] , although studies have shown that the decisions made by surrogates do not always reflect the wishes of patients [ , ] . however, some states in the united states either do not accept surrogate consent for research or authorized this form of consent only after the end of the ards network study [ ] . in addition, mistrust is gaining ground in the public at large, and organizations such as the alliance for human research protection (ahrp), whose stated goal is to ensure that human rights are protected during research (http://www.researchprotection.org, accessed july ) , are opposed to surrogate consent. the ahrp contributed to drive the inquiry of the office for human research protection on the ards network trials [ ] . surrendering the principle of autonomy to the principle of beneficence is ethically acceptable only when there is a reasonable certainty of nonmaleficence. this degree of certainty is not usually obtainable, as discussed above. tremendous amounts of money are invested in clinical research, in principle in the best interest of patients. the above words of beecher [ ] on the discrepancy between the abundance of funds and the scarcity of responsible investigators deserve careful consideration. financial resources for healthcare and for research are limited [ ] , and their fair allocation is both a political and an ethical imperative. because rcts are far more expensive than observational studies [ ] , they should provide answers that cannot be given by observational studies. although the impressive work conducted by the ards network investigators is worthy of respect, its failure to achieve this goal must be acknowledged. millions of dollars were spent [ ] , but, as discussed above, this study [ ] is merely one among several (including physiological and observational studies [ ] ) showing that patients should be ventilated with less than ml/kg body weight. in addition, it failed to determine whether volumes smaller than the - ml/kg noted in observational studies should be used [ , ] . similarly, the alveoli trial [ ] randomized a large number of patients but simply ruled out a need for peep levels higher than those in observational studies [ , ] . what have these multimilliondollar trials contributed? in the words of the belmont report [ ] , "the term 'research' designates an activity designed to test a hypothesis, permit conclusions to be drawn, and thereby to develop or contribute to generalizable knowledge." it is usually assumed that a nonscientific trial is unethical. but how generalizable are the data generated by rcts with excellent internal validity but limited external validity? as mentioned above, what makes the greatest contribution to clinical practice regarding the prevention of gastrointestinal bleeding in icu patients: "discordant" metaanalyses of multiple rcts [ , ] or a well conducted cohort study [ ] ? the informed consent dilemma this is a major ethical issue. as brilliantly discussed by several authors, informed consent by a proxy is a key safeguard for patients eligible for clinical trial inclusion [ , , ] . however, proxies may not consistently make the same choices as the patients or correctly interpret their best interests. in addition, the consent process can impose considerable suffering on the proxy. however, investigators are encouraged to disseminate the results of their work to the study participants [ ] , which is laudable. after the death of a loved one, a proxy might learn, for instance, that he or she consented to a clinical trial in which mortality was higher in the treatment arm. in that sense informed consent probably protects the icu patient and the physician in charge of the research project but not the proxy who is asked to provide consent. anxiety in patients asked to give their "full informed consent" to a study of a life-threatening disease has received attention [ ] . it has been rightly pointed out that this form of consent may be needlessly cruel [ ] . the same may hold true of "full informed consent" given by a proxy for a loved one. everyday practice teaches that proxies are highly vulnerable to distress and guilt when they are asked to provide consent to care for a critically ill patient (e.g., to a high-risk therapeutic procedure in a desperately ill patient). families of icu patients often exhibit major signs of anxiety and depression [ ] . the information that they receive in the name of the principle of autonomy may conflict with the principles of beneficence and nonmaleficence when proxies are asked to consent to research rather than to care. the primary focus of informed consent is risk disclosure [ ] , and the informed consent dilemma can be summarized as follows: if the consent form is to be reassuring for the family, it must fall short of providing honest information, and if the consent form is to be honest, as commendably proposed by the ards network investigators [ ] , it must supply information that is distressing to the family. surprisingly, most physicians fail to recognize that evaluation of the riskbenefit ratio, considered a requisite for ethical research [ ] , should not focus solely on the patient. the emotional risk to proxies should be balanced against the putative benefit (and risk) for the patient included in the rct [ ] . otherwise, proxies may be subjected to stress whose harmful effects exceed by far the objective benefits the patient may derive from participating in the study. does it make ethical sense to distress family members by asking them to consent to yet another study on sdd or on the prevention of gastrointestinal bleeding? informed consent documents and information sites for families of critically ill patients are sometimes so frightening that perhaps the door to the icu should bear the words inscribed on the gate to hell in dante alighieri's inferno (la divina commedia): "lasciate ogni speranza, voi ch'entrate!" ("abandon all hope, ye who enter here!"). two categories of solutions may be considered. some require no important qualitative changes in the system that governs clinical research but simply a tightening of research oversight procedures [ , , ] . other solutions would require a reshaping of many parts of the system, including the financial and academic incentives to publication, as well as a number of methodological dogmas. these two categories of solutions are not mutually exclusive. they both seek the best compromise between conflicting principles to protect the rights of patients and proxies and to improve scientific knowledge and quality of care, but not at the expense of scientific or ethical distortions. solutions that do not require significant changes in the system the protection of subjects who are unable to give or refuse consent must receive particularly close attention. there is little doubt that waiving of the requirement for consent should be reserved for highly unusual situations [ ] , which are described in detail in the united states code of federal regulations for the protection of human subjects [ ] . this text allows some forms of research in emergency situations without consent from the patient or surrogate provided certain conditions are met, including: the disorder is immediately life-threatening, available treatments are unsatisfactory, obtaining consent is not feasible, the research could not be carried out without the waiver, participation in the study holds the prospect of direct benefit, and the waiver of consent is given by an institutional review board. the situation is less clear in europe where research in emergency medicine can be performed without consent in some countries (including france) but not in others [ ] . however, the european parliament and council have issued a new directive that forbids research without consent, even in emergencies [ ] . this directive has been criticized as a serious potential threat to research in emergency situations [ , , , ] . before examining the magnitude of this threat, one must acknowledge the risk of overuse or abuse of waiving consent for emergency research in critically ill patients [ , ] . as recently argued by john luce, "few patients face true emergencies.... for example... studies of new modes of mechanical ventilation, novel therapies for sepsis... have a relatively long therapeutic window during which obtaining consent from patients or surrogates may be possible" [ ] . for research on true emergencies (e.g., treatment of cardiac arrest, acute brain injury) in the european union the directive and/or the research modalities will have to be modified, as discussed below. a strong argument that waiving consent may be appropriate for some rcts was put forward by truog and coworkers [ ] . they base their position on the frequently poor comprehension of the rct process and of informed consent documents by patients or surrogates. a waiver of consent could be obtained from an institutional review board provided the treatments offered in the trial are available outside the trial without the need for consent, the study carries only minimal additional risk, genuine equipoise exists among the studied treatments, and no reasonable person should have a preference for one treatment over any other. in the letters published in response to this thoughtful paper, the risk of jeopardizing patient autonomy was the principal argument against the contention by truog et al. [ ] . interestingly neither truog et al. nor their detractors questioned the validity of the opinion that governed their debate, namely, that rcts are useful under these special conditions. this is discussed below. solutions that require significant changes in the system from the above it clearly appears that the debate on informed consent to research stems primarily from two axioms that can be questioned: care and research are two separate activities, and rcts are superior to other forms of clinical research. investigators should acknowledge that they cannot have their cake and eat it too: they cannot both enjoy the putative methodological advantage of an rct and carry out their study without obtaining informed consent. consent is inherent in the rct process because it is needed to ensure compliance with basic ethical principles, most notably the principle of autonomy. except in the rare cases of true emergencies (see above), there is no obvious ethical justification of waiving consent to rcts. therefore if we want critical care to continue its amazing progress, we must rethink our research policies. first, the limited role (if any) of rcts in this progress must be acknowledged. second, the formal distinction between care and research must be reappraised. third, current methodological dogma must be challenged. as stated by miller and rosenstein [ ] in an article focusing mainly on rcts, "medical care is characterized by a convergence of the doctor's interests and the patient's interests.... by contrast, in clinical trials, the principal interests of the investigator and the participating patient may diverge." as mentioned above, the distinction between care and research was first made in the belmont report [ ] and should not be dismissed except in specific circumstances. critical care may be one of these circumstances, given the consensus that "there is instead a spectrum that extends from established, evidence-based interventions through unproved therapeutic innovations to formal rcts", as underlined by truog and colleagues [ ] in their response to the abundant correspondence generated by their publication [ ] . it is important to bear in mind that most if not all of the debate on the therapeutic misconception concerns the distinction between care and rcts [ , ] . this distinction is obviously valid. however, clinical research can also have an integral role in clinical care [ ] , most notably when the interventions are not allocated at random. this offers an opportunity for reconciling the interests of the patients and those of the physicians, provided adequate methodological changes are implemented. vandenbroucke and de craen [ ] wrote that "sometimes we accept the evidence from the randomized trial and overturn a theory-however beautiful it was-but at other times we stick with the theory and dismiss the evidence." there is no inviolable scientific reason to prefer rcts and their mandatory informed consent procedure to the well-being of patients and their proxies. there is, however, a moral obligation to improve the quality of critical care. therefore alternative methodological approaches that protect both the welfare and the autonomy of patients should be given preference. these approaches should "find ways of accommodating clinical reality, not ignoring it" [ ] , and should require acknowledgement that rcts can produce inconsistent results and can have limited external validity [ , ] . investigators will have to stop their obsessive quest for the so-called "absolute truth that can be given only by rcts" and acknowledge the subjective element in the evaluation of science [ ] . as underlined by jerome cornfield (inventor of both the odds ratio and logistic regression; cited in [ ] ), "good scientific practice... places the emphasis on reasonable scientific judgement and the accumulation of evidence and not on dogmatic insistence of the unique validity of a certain procedure." it is astonishing that physicians pressure institutional review boards (irbs) to accept waivers of consent and zealously lobby for changes in regulations that they feel may "impede research" [ , ] without questioning the validity of the diktats issued by a number of methodologists and journal editors. the real problem is not to obtain a waiver of consent to rcts from patients or proxies: the consent that we need is that of methodologists, from whom we seek creative study designs, and of medical journal editors, from whom we ask for greater openness to contributions that are less highly ranked in the pyramid of evidence-based medicine [ ] . trials with prerandomization. this design was introduced by zelen [ ] . patients are randomly preallocated to the conventional or new treatment before they are asked to consent to the study. in the patients allocated to the control arm no specific consent to research need be obtained. in contrast, informed consent is sought from the patients (or proxies) in the group allocated to the new treatment; when consent is refused, the patient receives the conventional treatment. this appealing design requires unblinded treatment administration, which is the case in many trials in icu patients (most notably on procedures). it has been used in pediatric trials of extracorporeal membrane oxygenation [ , ] . prerandomization may increase patient inclusion rates. studies have calculated the "price of autonomy," that is, the number of lives that may be lost if the inclusion rate is slower because prerandomization is not used, resulting in delayed implementation of a life-saving treatment [ , ] . however, ethical objections to prerandomization [ ] include denial of information, using people, denial of choice, and "overselling" of allocated treatments [ ] . observational studies. well-conducted observational cohort and case-control studies can provide the same level of internal validity as rcts [ , , ] . they are particularly well suited to research in the icu. indeed, as mentioned above, blinding is neither necessary nor feasible for most therapeutic interventions in the icu. in fact most of the major recent rcts were unblinded [ , , , , , , ] . why not evaluate new therapeutic interventions sequentially under conditions of genuine equipoise? in such conditions, rather than a detailed explanation of the randomization process, "such an elegant, reliable, sophisticated concept to the research clinician, but so brutal and harsh from the patient's view point" [ ] , only consent to general care [ ] and to the use of data obtained during usual patient care for research purposes [ ] would need to be obtained. renunciation of rcts in favor of observational studies does not imply that physicians have a free hand on their patients. irb approval and close monitoring of data quality and patient safety would still clearly be needed. in addition, cohort studies are probably the only option left for emergency research when legislation prohibits the waiving of consent, as may unfortunately become the case in the european union within the next few years [ , ] . interestingly, this approach was used in a recent study of thrombolytic therapy after initially unsuccessful cardiopulmonary resuscitation [ ] . the cohort design deserves to be considered before the alarm is sounded for the questionable reason that research in emergencies may stop if rcts are no longer feasible [ , ] . these are among the most dangerous studies, as recently shown by the results published for several new drugs in icu patients [ ] . furthermore, by definition there is a risk of major conflicts of financial interest. it follows that patient protection should receive particularly careful attention, and that the studies must be completely free of potential methodological weaknesses and ethical flaws. clearly such studies must be randomized (which is mandatory anyway for obtaining regulatory approval). in addition to a thorough review of the research protocol by a completely independent irb, unblinded monitoring of adverse effects should be conducted, as recently suggested by freeman and coworkers [ ] . finally, the informed consent document should provide detailed information and receive careful scrutiny by the irb. for instance, a fair informed consent document on new therapies for sepsis should explain what physicians know, namely, that the pathophysiology of sepsis is incom-pletely understood, and that the animal models on which the clinical trial is based are not fully valid [ ] . in addition, if another new treatment was associated with increased mortality rates in other studies [ ] , this fact should be disclosed. finally, the amount of money received by the physicians or their institution per patient included should be indicated, as well as any financial interests linking the physicians to the drug company (e.g., shares owned or position as paid consultants), as suggested by organizations such as the alliance for human research protection (http://www.researchprotection.org/informedconsent/informedconsent.html, accessed july ). patient autonomy should not stop where potential financial profit begins. as pointed out by luce [ ] , stronger research oversight may be as important as informed consent in protecting patient welfare. since we contend that, provided investigators desist from performing rcts, formal consent to research need not be sought and consent to care and to the use of data is the rule, it also holds that research oversight must be reinforced [ , , , ] . research projects that do not require specific informed consent should be examined thoroughly by irbs, which should obtain the opinions of independent consultants if needed. the primary goal is not to make research easier for physicians but to increase the safety of patients and proxies. if greater ease of research occurs as a side effect, this will be welcome. in conclusion, rcts were born under a shroud of original sin consisting of financial, political, and academic pressure. this was summarized by yoshioka [ ] in a publication about the medical research council trial on streptomycin: "the innovation of centrally controlled randomisation can be attributed to a combination of scientific logic and political and social pressures on medical bureaucracy." this sin may have remained unredeemed, as suggested by the extraordinary controversy about the ards network study [ , ] . before deciding which clinical study design is best suited to critically ill patients, consideration should be given to several points: -many rcts in critical care generated heated pro-con debate during medical conventions yet failed to improve patient care. -respect for patients and their families requires that investigators refrain from using a plethora of informed consent documents which constitute a perversion of ethical principles but rather wield this two-edged sword with discernment. -clinical trials should be conducted not to achieve methodological purity but to improve patient management. -in clinical trials the best possible compromise should be sought between the ethical principle of beneficence to the patient and that of nonmaleficence to the proxy, who is asked to give consent while struggling with overwhelming distress. -clinical trials should be appraised according to the ethical principle of distributive justice: what cost for what result? -society is changing and may no longer be ready to accept what many persons may consider, rightly or wrongly, to be a manifestation of medical power. the effects of the aids epidemic on patient mentalities and the growing influence of organizations such as ahrp should be pondered. similarly, owing to widespread public concern about the adequacy of protection for human research subjects, litigation against investigators, irbs, and academic institutions is becoming increasingly common [ ] . then, nolens volens, clinicians may have to forbear conducting rcts for ethical, scientific, sociological, legal, and financial reasons. one solution may consist in giving preference to forms of clinical research that are tightly linked to care. clinicians should work closely with innovative methodologists to find new designs that are acceptable to all. clinicians serve patients. and methodologists serve clinicians, not the opposite. national commission for the protection of human subjects of biomedical and behavioral research ( ) belmont report: ethical principles and guidelines for the protection of human subjects of research. united states government printing office unraveling the tuskegee study of untreated syphilis the therapeutic orientation to clinical trials the integral role of clinical research in clinical care is informed consent always necessary for randomized, controlled trials? what makes clinical research ethical readability standards for informed-consent forms as compared with actual readability symptoms of anxiety and depression in family members of intensive care unit patients: ethical hypothesis regarding decisionmaking capacity is informed consent always necessary for randomized, controlled trials? non-therapeutic research in the eu in adults incapable of giving consent? a european directive for clinical research implications of the eu directive on clinical trials for emergency medicine directive / /ec of the european parliament and of the council to consent or not to consent, that is (not) the (sole) question evidence-based medicine. a new approach to teaching the practice of medicine a change in scientific approach: from alternation to randomised allocation in clinical trials in the s a calculated risk": the salk polio vaccine field trials of essai clinique. in: lecourt d (ed) dictionnaire de la pen-sØe mØdicale ethics and clinical research alternative medicine: a "mirror image" for scientific reasoning in conventional medicine pharmaceutical industry sponsorship and research outcome and quality: systematic review sponsorship, authorship, and accountability risks and benefits of activated protein c treatment for severe sepsis assessing the use of activated protein c in the treatment of severe sepsis is academic medicine for sale? medicine based evidence, a prerequisite for evidence based medicine truth survival in clinical research: an evidence-based requiem? ventilation with lower tidal volumes as compared with traditional tidal volumes for acute lung injury and the acute respiratory distress syndrome improved survival of patients with acute respiratory distress syndrome evidence-based medicine or fuzzy logic. what is best for ards management? characteristics and outcomes in adult patients receiving mechanical ventilation: a -day international study epidemiology and outcome of acute lung injury in european intensive care units. results from the alive study impact of randomized trial results on acute lung injury ventilator therapy in teaching hospitals are we really reducing tidal volume-and should we? barriers to providing lung-protective ventilation to patients with acute lung injury higher versus lower positive end-expiratory pressures in patients with the acute respiratory distress syndrome effect of prone positioning on the survival of patients with acute respiratory failure intermittent positive-pressure hyperventilation with high inflation pressures produces pulmonary microvascular injury in rats improved prognosis of acute respiratory distress syndrome years on searching for evidence: don't forget the foundations effectiveness of antibiotic prophylaxis in critically ill adult patients: systematic review of randomised controlled trials selective digestive decontamination: for everyone, everywhere? selective decontamination of the digestive tract and its effect on antimicrobial resistance prevention of hospital-associated pneumonia and ventilator-associated pneumonia bleeding and pneumonia in intensive care patients given ranitidine and sucralfate for prevention of stress ulcer: meta-analysis of randomised controlled trials stress ulcer prophylaxis in critically ill patients. resolving discordant meta-analyses clinically significant gastrointestinal bleeding in critically ill patients with and without stress-ulcer prophylaxis clinicians' approaches to mechanical ventilation in acute lung injury and ards the price of autonomy principles of medical ethics why immunomodulatory therapies have not worked in sepsis safeguarding patients in clinical trials with high mortality rates is the concept of informed consent applicable to clinical research involving critically ill patients? ethical considerations for research in critically ill patients do surrogate decision makers provide accurate consent for intensive care research? california's new law allowing surrogate consent for clinical research involving subjects with impaired decision-making capacity suspension of the nih ards network fluids and catheters treatment trial control group selection in critical care randomized controlled trials evaluating interventional strategies: an ethical assessment beyond randomised versus observational studies how best to ventilate? trial design and patient safety in studies of the acute respiratory distress syndrome informing clinical trial participants about study results bmj's present policy (sometimes approving research in which patients have not given fully informed consent) is wholly correct fully informed consent can be needlessly cruel influence of the law on risk and informed consent recommendations for informed consent forms for critical care trials. abstracts improving protection for research subjects protecting subjects with decisional impairment in research: the need for a multifaceted approach protection of human subjects: informed consent and waives of informed consent requirements in certain emergency research. final rules. title , code of federal regulations the european union directive and the protection of incapacitated subjects in research: an ethical analysis the effect of waiving consent on enrollment in a sepsis trial is informed consent always necessary for randomized controlled trials? a new design for randomized clinical trials extracorporeal circulation in neonatal respiratory failure: a prospective randomized study extracorporeal membrane oxygenation and conventional medical therapy in neonates with persistent pulmonary hypertension of the newborn: a prospective randomized study informed consent. numbers inform the debate zelen randomization: attitudes of parents participating in a neonatal clinical trial should zelen prerandomised consent designs be used in some neonatal trials a comparison of observational studies and randomized, controlled trials randomized, controlled trials, observational studies, and the hierarchy of research designs a multicenter, randomized, controlled clinical trial of transfusion requirements in critical care. transfusion requirements in critical care investigators, canadian critical care trials group a trial of goal-oriented hemodynamic therapy in critically ill patients. svo collaborative group early goal-directed therapy in the treatment of severe sepsis and septic shock intensive insulin therapy in the critically ill patients improving the process of informed consent in the critically ill patients' consent preferences for research uses of information in electronic medical records: interview and survey data efficacy and safety of thrombolytic therapy after initially unsuccessful cardiopulmonary resuscitation: a prospective clinical trial ethics, science, and oversight of critical care research: the office for human research protections use of randomisation in the medical research council's clinical trial of streptomycin in pulmonary tuberculosis in the s metaanalysis of acute lung injury and acute respiratory distress syndrome trials testing low tidal volumes the rise of litigation in human subjects research key: cord- -h ftu ax authors: macintyre, c. raina; chughtai, abrar ahmad title: a rapid systematic review of the efficacy of face masks and respirators against coronaviruses and other respiratory transmissible viruses for the community, healthcare workers and sick patients date: - - journal: int j nurs stud doi: . /j.ijnurstu. . sha: doc_id: cord_uid: h ftu ax background: the pandemic of covid- is growing, and a shortage of masks and respirators has been reported globally. policies of health organizations for healthcare workers are inconsistent, with a change in policy in the us for universal face mask use. the aim of this study was to review the evidence around the efficacy of masks and respirators for healthcare workers, sick patients and the general public. methods: a systematic review of randomized controlled clinical trials on use of respiratory protection by healthcare workers, sick patients and community members was conducted. articles were searched on medline and embase using key search terms. results: a total of randomised controlled trials were included in this study – in community settings, in healthcare settings and as source control. most of these randomised controlled trials used different interventions and outcome measures. in the community, masks appeared to be more effective than hand hygiene alone, and both together are more protective. randomised controlled trials in health care workers showed that respirators, if worn continually during a shift, were effective but not if worn intermittently. medical masks were not effective, and cloth masks even less effective. when used by sick patients randomised controlled trials suggested protection of well contacts. conclusion: the study suggests that community mask use by well people could be beneficial, particularly for covid- , where transmission may be pre-symptomatic. the studies of masks as source control also suggest a benefit, and may be important during the covid- pandemic in universal community face mask use as well as in health care settings. trials in healthcare workers support the use of respirators continuously during a shift. this may prevent health worker infections and deaths from covid- , as aerosolisation in the hospital setting has been documented. the pandemic of covid- is growing, and a shortage of masks and respirators has been reported globally. policies of health organizations for healthcare workers are inconsistent, with a change in policy in the us for universal face mask use. the aim of this study was to review the evidence around the efficacy of masks and respirators for healthcare workers, sick patients and the general public. a systematic review of randomized controlled clinical trials on use of respiratory protection by healthcare workers, sick patients and community members was conducted. articles were searched on medline and embase using key search terms. a total of randomised controlled trials were included in this study - in community settings, in healthcare settings and as source control. most of these randomised controlled trials used different interventions and outcome measures. in the community, masks appeared to be more effective than hand hygiene alone, and both together are more protective. randomised controlled trials in health care workers showed that respirators, if worn continually during a shift, were effective but not if worn intermittently. medical masks were not effective, and cloth masks even less effective. when used by sick patients randomised controlled trials suggested protection of well contacts. the study suggests that community mask use by well people could be beneficial, particularly for covid - , where transmission may be pre-symptomatic. the studies of masks as source control also suggest a benefit, and may be important during the covid- pandemic in universal community face mask use as well as in health care settings. trials in healthcare workers support the use of respirators continuously during a shift. the use of personal protective equipment for coronavirus disease (covid- ) has been controversial, with differing guidelines issued by different agencies ( ) . coronavirus disease is caused by severe acute respiratory syndrome coronavirus (sars-cov- ), a beta-coronavirus, similar to severe acute respiratory syndrome coronavirus (sars cov) ( ). seasonal alpha and beta coronaviruses cause common colds, croup and broncholitis. the transmission mode of coronaviruses in humans is similar, thought to be by droplet, contact and sometimes airborne routes ( ) ( ) ( ) . the world health organization recommends surgical mask for health workers providing routine care to a coronavirus disease patient ( ), whilst the us centers for disease control and prevention recommend a respirator ( ) . most authorities are recommending that community members not wear a mask, and that a mask should only be worn by a sick patient (also referred to as source control) ( ) . there are more randomised controlled trials of community use of masks in well people than studies of the use by sick people (source control). the aim of this study was to review the randomised controlled trials evidence for use of masks and respirators by the community, health care workers and sick patients for prevention of infection. we searched medline and embase for clinical trials on masks and respirators using the key words "mask", "respirator", and "personal protective equipment". the search was conducted between march to april and all randomised controlled trials published before the search date were included. two authors (crm and aac) reviewed the title and abstracts to identify randomised controlled trials on masks and respirators. we also searched relevant papers from the reference lists of previous clinical trials and systematic reviews. studies that were not randomised controlled trials, were about anaesthesia, or not about prevention of infection were excluded. animal studies, experimental and observational epidemiologic studies were also excluded. studies published in english language were included. we found papers on medline and on embase. papers were excluded by title and abstract review. full texts were reviewed for papers and were selected in this review. results were reported according to the preferred reporting items for systematic reviews and meta-analyses (prisma) criteria ( ) . in general, the results show protection for healthcare workers and community members, and likely benefit of masks used as source control. we found eight clinical trials ( - ) on the use of masks in the community (table ). in the community, masks appear to be more effective than hand hygiene alone, and both together are more protective ( , ) . however, the randomised controlled trials which measured both hand hygiene and masks measured the effect of hand hygiene alone, but not of masks alone ( , , ) . masks were only examined in combination with hand hygiene. therefore the protective effect of masks and hand hygiene combined could be due to both interventions together, or the effect of masks alone. the use of hand hygiene alone in these trials was not effective. in more than one trial, interventions had to be used within hours of exposure to be effective ( , , ) . to date, six randomised controlled trials ( ) ( ) ( ) ( ) ( ) ( ) has been conducted on the use of masks and/or respirators by healthcare workers in health care settings ( table ). the healthcare workers trials (table ) used different interventions and different outcome measures, and one was in the outpatient setting. a japanese study had only subjects, and likely was underpowered to find any difference between masks and control ( ) . two north american trials of masks and respirators against influenza infection found no difference between the arms, but neither had a control arm to differentiate equal efficacy from equal inefficacy ( , ) . without a control group to determine rates of influenza in unprotected healthcare workers, neither study is able to determine efficacy if no difference was observed between the two interventions. a serologic study showed that up to % of unprotected healthcare workers (a rate identical to that observed in loeb the trial, which also used serology) contract influenza during outbreaks ( ) , which suggests lack of efficacy. studies of nosocomial influenza generally find lower influenza attack rates in unprotected healthcare workers than observed in the loeb trial ( ) . further problems with this study are that the majority of subjects were defined as having influenza on the basis of serological positivity ( ) . the % seroconversion to pandemic h n (with no pandemic virus isolation or positive pcr) observed in the trial, suggests that pandemic h n was circulating in ontario before april , which is unlikely. the overall flu rate was %, higher than the expected attack rate in a pandemic ( ) . the majority of subjects defined as having influenza were by serology. a serological definition of influenza can be affected by vaccination. the authors claim they excluded influenza vaccinated subjects in the outcome, but according to figure , these subjects ( in total) are included in the analysis. if they had been excluded and even if no other subjects were excluded, the total analysed would be , which is lower than the subjects analysed ( ) . these vaccinated subjects should have been excluded entirely from the analysis. the vaccination status of subjects with seropositivity is not provided in the paper, but it appears people with positive serology due to vaccination may have wrongly been counted as influenza cases ( ) . in both the north american trials, the intervention comprised wearing the mask or respirator when in contact with recognized ili or when doing a high risk procedure, which is a targeted strategy ( , ) . one was in an outpatient setting. ( ) we conducted a randomised controlled trial comparing the targeted strategy tested in the two north american studies, with the wearing of respiratory protection during an entire shift, and showed efficacy for continual (but not targeted) use of a respirator ( ) . the study also did not show efficacy for a surgical mask worn continually, and therefore no difference between a surgical mask and targeted use of a respirator ( ) , which is consistent with the findings of the north american trials ( , ) . in summary, the evidence is consistent that a respirator must be worn throughout the shift to be protective. targeted use of respirators only when doing high risk procedures and medical mask use is not protective. another randomised controlled trial we conducted in china showed efficacy for continual use of a respirator, but not for a mask, and also found fit-testing of the respirator did not affect efficacy ( ) . however, this may be specific to the quality of the tested product, and is not generalisable to other respirators -fit testing is a necessary part of respirator use ( ) . for healthcare workers, there is evidence of efficacy of respirators if worn continually during a shift, but no evidence of efficacy of a mask ( , ) . for hospitals where covid- patients are being treated, there is growing evidence of widespread contamination of the ward environment, well beyond meters from the patient, as well as aerosol transmission ( , , ) . several studies have found sars-cov- on air vents and in air samples in intensive care units and covid- wards ( , , ) , and an experimental study showed the virus in air samples three hours after aerosolization ( ). the weight of this evidence and the precautionary principle( , ), favors respirators for healthcare workers. we showed lower rates of infection outcomes in the medical mask arm compared to control, but the difference was not significant ( ) . it could be that larger trials are needed to demonstrate efficacy of a mask, but any protection is far less than from a respirator. a trial we conducted in vietnam of -layered cotton cloth masks compared to medical masks showed a lower rate of infection in the medical mask group, and a times higher risk of infection in the cloth mask arm ( ) . the study suggests cloth masks may increase the risk of infection ( ), but may not be generalizable to all home-made masks. the material, design and adequacy of washing of cloth masks may have been a factor ( ) . there are no other randomised controlled trial of cloth masks published, but if any protection is offered by these it would be less than even a medical mask. table shows the trials of source control. there were five randomised controlled trials identified of masks used by sick patients ( ) ( ) ( ) ( ) . one was an experimental study of influenza patients, which did not measure clinical endpoints ( ) . participants with confirmed influenza coughed onto a petri dish wearing a n respirator or a mask. no influenza grew on the medium. a trial of sick patients wearing a mask (or no mask) in the household found no significant difference between arms ( ). however, the trial was terminated prematurely and did not meet recruitment targets, so was probably underpowered. one randomised controlled trial was conducted among hajj pilgrims, with both well and sick pilgrims wearing masks, and low rates of ili were reported among contact of mask pilgrims ( ) . our randomised controlled trial is the largest available, and studied patients randomised to mask or control ( ) . compliance was suboptimal in the mask group and some controls wore masks. the intention to treat analysis showed no difference, but when analysed by actual mask use, the rate of infection in household contacts was lower in those who wore masks ( ) . a trial with an experimental design was published in april , examining a range of viruses including seasonal human coronaviruses ( ) . this showed that coronaviruses are preferentially found in aerosolized particles compared to large droplets, and could be expelled by normal tidal breathing. wearing a surgical mask prevented virus from being exhaled. there are more randomised controlled trials of community use of masks in well people ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) than studies of the use by sick people (also referred to as "source control"), and these trials are larger than the few on source control ( ) ( ) ( ) . the evidence suggests protection of masks in high transmission settings such as household and college settings, especially if used early, if combined with hand hygiene and if wearers are compliant ( , ( ) ( ) ( ) ( ) ( ) . if masks protect in high transmission settings, they should also protect in crowded public spaces, including workplaces, buses, trains, planes and other closed settings. the trial which did not show efficacy used influenza as the outcome measure ( ) , which is a rare outcome, so requires a larger sample size for adequate power and may have been underpowered. for healthcare workers, the only trials to show a difference between respirators and masks demonstrated efficacy for continuous use of a respirator through a clinical shift, but not masks ( , ) . the two trials which showed no difference are widely cited as evidence that masks provide equal protection as respirators ( , ) . however, without a control arm, the absence of difference between arms could reflect equal efficacy or inefficacy, and it is not possible to draw any conclusions about efficacy. the high rates of influenza in the loeb trial suggest equal inefficacy, and further, there were likely misclassified outcomes in the trial by inclusion of seropositive, vaccinated healthcare workers, which would have biased the results ( ) . the outpatient setting in the us trial may have had lower exposure risk than the inpatient setting of other trials. ( ) in both the north american trials, the intervention comprised wearing the mask or respirator when in contact with recognized ili or when doing a high risk procedure ( , ) . the underlying assumption that the majority of infections in healthcare workers occur during self-identified high-risk exposures is not supported by any evidence. it assumes healthcare workers can accurately identify when they are risk in a busy, clinical setting, when the majority of infections may occur when healthcare workers are unaware of the risk (such as when walking through a busy emergency room or ward where aerosolized virus may be present). conversely, infections could occur outside the workplace. this could explain the lack of difference if there was no actual efficacy of either arm and if much of the infection occurs in unrecognised situations of risk either within or outside the workplace. in practice, hospital infection control divides infections into droplet or airborne spread, and recommends droplet (mask) or airborne (respirator) precautions accordingly ( ) . in a pooled analysis of both healthcare worker trials, we showed that continual use of a respirator is more efficacious in protecting healthcare workers even against infections assumed to be spread by the droplet route ( ) . medical masks did not significantly protect against viral, bacterial, droplet or other infection outcomes. however, the summary odds ratio for masks was less than one, which suggests a low level of protection. targeted use of respirator protected against bacterial and droplet infections, but not against viral infections, suggesting viral infections may be more likely to be airborne in the hospital setting ( ) . the five available studies of mask use by sick patients suggest a benefit, but are much smaller trials than the community trials, two without clinical endpoints, and with less certainty around the findings ( ) ( ) ( ) ( ) . only / trials examined clinical outcomes in close contacts ( ) ( ) ( ) . many systematic reviews have been conducted on masks, respirators and other ppe in past ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) . these reviews generally examined multiple interventions (e.g. masks and hand hygiene etc), often combined different outcome measures that were not directly comparable and were inconclusive. moreover, most of these reviews did not include more recent randomised controlled trials ( , ) . this systematic review only focuses on masks and respirators and contains all new studies. in summary, there is a growing body of evidence supporting all three indications for respiratory protectioncommunity, healthcare workers and sick patients (source control). the largest number of randomised controlled trials have been done for community use of masks by well people in high-transmission settings such as household or college settings. there is benefit in the community if used early, and if compliant. they also found no evidence of efficacy of hand hygiene or health education, suggesting mask use is more protective than hand hygiene. respirators protect healthcare workers if worn continually, but not if worn intermittently in self-identified situations of risk. this supports the suggestion that the health care environment is a risk to healthcare workers even when not doing aerosol generating procedures or caring for a known infectious patient. for covid- specifically, the growing body of evidence showing aerosolisation of the virus in the hospital ward highlights the risk of inadvertent exposure for healthcare workers and supports the use of airborne precautions at all times on the ward ( , , ) . further, the rule of - m of spatial separation is not based on good evidence, with most research showing that droplets can travel further than m, and that infections cannot be neatly separated into droplet and airborne ( , ) . in the uk, one healthcare trust found almost one in five healthcare workers to be infected with covid- ( ) . the deaths of healthcare workers from covid- reflect this risk ( ) . the use of masks by sick people, despite being the who's only recommendation for mask use by community members during covid- pandemic, is supported by the smallest body of evidence. source control is probably a sensible recommendation given the suggestion of protection and given specific data on coronaviruses showing protection ( ) . it may help if visitors and febrile patients wear a mask in the healthcare setting, whether in primary care or hospitals. universal face mask use is likely to have the most impact on epidemic growth in the community, given the high risk of asymptomatic and pre-symptomatic transmission ( ) . research fellowship) and sanofi currently. she has received funding from m more than years ago for face mask research. abrar ahmad chughtai had testing of filtration of masks by m for his phd more than years age. m products were not used in his research. he also has worked with cleanspace technology on research on fit testing of respirators (no funding was involved). epidemiological and clinical characteristics of cases of novel coronavirus pneumonia in wuhan, china: a descriptive study surface environmental, and personal protective equipment contamination by severe acute respiratory syndrome coronavirus (sars-cov- ) from a symptomatic patient molecular and serological investigation of -ncov infected patients: implication of multiple shedding routes infection prevention and control during health care when novel coronavirus (ncov) infection is suspected. interim guidance interim healthcare infection prevention and control recommendations for patients under investigation for novel coronavirus policies on the use of respiratory protection for hospital health workers to protect from coronavirus disease (covid- ) preferred reporting items for systematic review and meta-analysis protocols (prisma-p) facemasks, hand hygiene, and influenza among young adults: a randomized intervention trial findings from a household randomized controlled trial of hand washing and face masks to reduce influenza transmission in bangkok, thailand. influenza and other respiratory viruses impact of nonpharmaceutical interventions on uris and influenza in crowded, urban households mask use, hand hygiene, and seasonal influenza-like illness among young adults: a randomized intervention trial. the journal of infectious diseases face mask use and control of respiratory virus transmission in households. emerging infectious diseases preliminary findings of a randomized trial of non-pharmaceutical interventions to prevent influenza transmission in households the role of facemasks and hand hygiene in the prevention of influenza transmission in households: results from a cluster randomised trial facemasks and hand hygiene to prevent influenza transmission in households: a cluster randomized trial n respirators vs medical masks for preventing influenza among health care personnel: a randomized clinical trial use of surgical face masks to reduce the incidence of the common cold among health care workers in japan: a randomized controlled trial a randomized clinical trial of three options for n respirators and medical masks in health workers a cluster randomized clinical trial comparing fit-tested and non-fit-tested n respirators to medical masks to prevent respiratory virus infection in health care workers a cluster randomised trial of cloth masks compared with medical masks in healthcare workers surgical mask vs n respirator for preventing influenza among health care workers: a randomized trial incidence and recall of influenza in a cohort of glasgow healthcare workers during the - epidemic: results of serum testing and questionnaire influenza in the acute hospital setting. the lancet infectious diseases examining the policies and guidelines around the use of masks and respirators by healthcare workers in china, pakistan and vietnam transmission potential of sars-cov- in viral shedding observed at the university of nebraska medical center aerosol and surface distribution of severe acute respiratory syndrome coronavirus in hospital wards detection of air and surface contamination by severe acute respiratory syndrome coronavirus (sars-cov- ) in hospital rooms of infected patients aerodynamic characteristics and rna concentration of sars-cov- aerosol in wuhan hospitals during covid- outbreak respiratory protection for healthcare workers treating ebola virus disease (evd): are facemasks sufficient to meet occupational health and safety obligations? uncertainty, risk analysis and change for ebola personal protective equipment guidelines a quantitative assessment of the efficacy of surgical and n masks to filter influenza virus in patients with acute influenza infection cluster randomised controlled trial to examine medical mask use as source control for people with respiratory illness surgical mask to prevent influenza transmission in households: a cluster randomized trial pilot randomised controlled trial to test effectiveness of facemasks in preventing influenza-like illness transmission among australian hajj pilgrims in respiratory virus shedding in exhaled breath and efficacy of face masks the efficacy of medical masks and respirators against respiratory infection in healthcare workers. influenza and other respiratory viruses the use of masks and respirators to prevent transmission of influenza: a systematic review of the scientific evidence. influenza and other respiratory viruses protecting healthcare workers from pandemic influenza: n or surgical masks? critical care medicine protecting health care workers from sars and other respiratory pathogens: a review of the infection control literature physical interventions to interrupt or reduce the spread of respiratory viruses: systematic review physical interventions to interrupt or reduce the spread of respiratory viruses. the cochrane database of systematic reviews physical interventions to interrupt or reduce the spread of respiratory viruses: systematic review non-pharmaceutical public health interventions for pandemic influenza: an evaluation of the evidence base physical interventions to interrupt or reduce the spread of respiratory viruses -resource use implications: a systematic review personal protective equipment for preventing highly infectious diseases due to exposure to contaminated body fluids in healthcare staff airborne or droplet precautions for health workers treating covid- ? the journal of infectious diseases roll-out of sars-cov- testing for healthcare workers at a large nhs foundation trust in the united kingdom death from covid- of health care workers in china temporal dynamics in viral shedding and transmissibility of covid- . medrxiv key: cord- -sls bsm authors: dean, natalie e.; pastore y piontti, ana; madewell, zachary j.; cummings, derek a.t; hitchings, matthew d.t.; joshi, keya; kahn, rebecca; vespignani, alessandro; elizabeth halloran, m.; longini, ira m. title: ensemble forecast modeling for the design of covid- vaccine efficacy trials date: - - journal: vaccine doi: . /j.vaccine. . . sha: doc_id: cord_uid: sls bsm to rapidly evaluate the safety and efficacy of covid- vaccine candidates, prioritizing vaccine trial sites in areas with high expected disease incidence can speed endpoint accrual and shorten trial duration. mathematical and statistical forecast models can inform the process of site selection, integrating available data sources and facilitating comparisons across locations. we recommend the use of ensemble forecast modeling – combining projections from independent modeling groups – to guide investigators identifying suitable sites for covid- vaccine efficacy trials. we describe an appropriate structure for this process, including minimum requirements, suggested output, and a user-friendly tool for displaying results. importantly, we advise that this process be repeated regularly throughout the trial, to inform decisions about enrolling new participants at existing sites with waning incidence versus adding entirely new sites. these types of data-driven models can support the implementation of flexible efficacy trials tailored to the outbreak setting. the covid- pandemic is a public health emergency, and there is an urgent need for effective vaccines to limit morbidity and mortality. efforts are underway to accelerate all steps in the vaccine development pathway ( ) . large randomized field trials are crucial for determining the safety and efficacy of candidates to inform regulatory decisions ( ) . in these trials, many thousands of eligible and consenting participants across multiple sites are enrolled and individually randomized to vaccine or control. these trials are event driven, where an expected primary endpoint is laboratory-confirmed symptomatic disease ( ) , with infection regardless of symptoms as a valuable secondary endpoint ( ) . selecting vaccine trial sites where disease incidence is highest during the study period can accelerate the accrual of endpoints. mathematical and statistical models are recognized as valuable tools for planning infectious disease clinical trials ( ) . they can be used to optimize design features such as cluster size or to examine the validity of the trial's statistical analysis ( ) . the use of spatially explicit forecast models to select vaccine trial sites was first explored during the - zika epidemic ( ) . these forecast models synthesize available data to make projections about which sites might have the highest future disease incidence. an important value of models is that they standardize projections across locations. trends in raw reported numbers of cases depend heavily on the sensitivity of the underlying surveillance system. case definitions and access to care and testing may vary over time and space. models that integrate many data sources, such as reported cases, test positivity, hospitalizations and deaths, can facilitate more meaningful comparisons across locations. forecasts provide estimates along with the uncertainty associated with those estimates to make best use of the available information. models can incorporate many features to capture the complex dynamics of infectious diseases. incidence is expected to vary widely over time and between locations, as a function of control measures in place, patterns of introduction, seasonality, and other sources of variability. mathematical models naturally account for prior circulation of the virus and any buildup of population-level immunity. areas that have already experienced substantial outbreaks may be less suitable for inclusion, and this would be reflected in projections. models can explicitly capture correlation due to movement between nearby sites or between sites and a common hub ( ) . models can also reflect relevant population-level features associated with expected incidence, such as density, race/ethnicity, age distribution, and educational status. we recommend the use of ensemble modeling, whereby multiple modeling groups prepare independent projections and these are combined to guide decision-making. individual models can be agent-based, compartmental, or statistical, can use different assumptions and data sources, but are all tasked with the same question of which sites are likely to have the highest disease incidence over a moderate time horizon. ensemble modeling has been shown to be more robust for complex systems than specialist models and better able to capture the complete range of possible outcomes ( ) . the strength of ensemble modeling has been shown for diseases like influenza ( ), dengue ( ) and ebola ( ) . ensemble modeling for covid- , like the covid- forecast hub, is similarly more robust ( ) . in addition to using forecast modeling for initial site selection, we propose that modeling be repeated at regular intervals throughout the trial. in the context of outbreaks, trials should be flexible to allow new sites to be added in response to evolving epidemiology ( ) . some sites will have lower than projected incidence during the trial period. for example, local policies or voluntary changes in behavior could effectively reduce transmission, meaning that the site is no longer a "hotspot." the modeling results can guide investigators deciding whether to continue to enroll new participants from existing sites, or to enroll new sites in emerging hotspots. in this paper, we describe a simplified framework for the use of ensemble modeling to guide the selection and continued evaluation of sites for a vaccine efficacy trial, with a focus on the covid- pandemic. individual modeling teams are welcome to contribute to the consensus model. we assume that models would already be built for general public health planning, so they would not be constructed only for this effort, though they may need to be modified. investigators can leverage existing groups or form new groups of modelers. participating modeling teams would be provided with a list of all candidate sites being explored. this list of sites may be based on previous engagement between the trial investigators and potential research partners. for a multi-country trial, this may include several sites per country from multiple countries. participating models must meet a minimum set of requirements. suggested guidelines are the ability to: (i) capture all geographic areas in the candidate list of sites, (ii) disaggregate to at least the first administrative level (e.g. state, province), though finer levels may be preferred for certain planning activities, (iii) project the covid- symptomatic cumulative incidence, i.e. the number of new symptomatic infections of any severity divided by the total population size during a pre-specified period (three months suggested), and (iv) produce a minimum of simulated epidemics. models must also be screened for internal consistency and basic plausibility when compared to historical trends. for each site, each model must generate a probabilistic predictive distribution for quantities of interest, such as the symptomatic cumulative incidence. these are bins of % width centered around whole numbers [ . , . %), [ . , . %), [ . , . %) and so on. the bin that includes % is narrower [ , . %). for each site, probabilistic predictive distributions are aggregated across models using stacking. figure describes a hypothetical model stacking procedure for a target site, per ray and reich ( ) . for simplicity and transparency, each model is assigned an equal weight, which is one over the number of models as done by the covid- forecast hub ( ) . if a participating team has developed more than one model, they must specify which model is primary and will contribute to the aggregate. more complex weighting schemes exist that preferentially weight models that performed best in previous rounds after an appropriate burn-in period ( ) . for each site, we can use the combined predictive distribution to produce summary statistics. suggested summary statistics are: (i) median incidence value, (ii) th percentile incidence value, (iii) th percentile incidence value, and (iv) probability incidence value is [ , . %) (probability of a very small or no outbreak). to present this information in a way that is easy for trial investigators to explore, we recommend reporting stacked projections, summary statistics, and basic information about the sites in an interactive tool, like the r shiny platform ( ) . this allows the end user to sort the table or select rows for closer examination. in this way, they could select a subset of "best rows" and view these together to approximate the formation of a trial. figure is a sample screenshot from such a program (code provided in supplemental materials). by generating a range of possible outcomes, models can capture the stochasticity of future transmission, including scenarios where incidence is much lower or much higher than the median projection. where incidence is highly variable with the potential to be very low, it may be preferable to include a larger number of sites to guard against the chance of accruing no efficacy data. the goal of ensemble modeling is to provide a simple and informative resource rather than a definitive recommendation. investigators will simultaneously consider many operational, political, and scientific factors. to provide context, we describe several other key considerations. to ensure a high-quality trial, sites should have adequate capacity for testing, safety monitoring, active surveillance, and high participant retention. nonetheless, sites with projected high incidence but poorer capacity should not be excluded if there is a potential role for mobile trial teams, as was used in the ebola ring vaccination trial in guinea ( ) . approval for the trial may be at the national or sub-national level, with flexibility to identify the particular target population when investigators are ready to start enrollment. for multi-country trials, investigators must weigh including multiple sites per country against including more countries. on one hand, given the complexity of country-specific procedures for approving clinical research, it may be easier to include multiple sites per country from fewer countries overall. on the other hand, the global community must ensure equitable access to potentially effective vaccines. broad representation also increases generalizability of the trial results, as it can best capture the effectiveness of vaccine candidates in diverse settings. these include variations in population age profile, race/ethnicities, climate, background presence of non-pharmaceutical interventions, and co-circulation of other coronaviruses. including many different geographic locations makes trials more robust to changes in the epidemic. while china was once the center of the covid- epidemic, several treatment trials initiated there were underpowered due to waning transmission ( ) . as other countries adopt more effective control strategies, incidence would likely decline, but it is less likely to wane in all areas, and new sites can also be added. experience with zika in the americas provides a useful counter-example, though, where trials were not possible because incidence dramatically declined everywhere ( ) . if that were to occur, the ensemble modeling process would be useful for assessing trial feasibility. finally, the ensemble modeling process should be evaluated by comparing model projections to subsequently observed data. an evaluation procedure could be conducted prior to each new round of modeling, before investigators want to make decisions about adding new trial sites. this process could assess how well model-projected rankings corresponded to observed rankings of hardest hit sites. where there is a lot of uncertainty in which sites will have highest incidence, as reflected in low correlation, investigators may feel more comfortable making future decisions based on logistical or political considerations rather than purely on model rankings. this evaluation procedure could also be conducted formally after the trial ends to compare model-projected and observed cumulative incidence and observed incidence during the target time periods. these types of reports are very useful for understanding the role of modeling as a tool for real-time decision-making in outbreaks ( , ) . we describe an ensemble modeling procedure to inform site selection for a vaccine efficacy trial planned during an ongoing epidemic. by prioritizing sites with highest projected disease incidence, investigators can accelerate the pace of endpoint accrual. mathematical and statistical models synthesize the best available evidence to guide this planning. we focus on covid- as a motivating example, but the general principles apply to other emerging infectious diseases. we present a highly simplified procedure to reduce the burden on modeling groups to prepare results and potentially enable more groups to participate. for example, models could, but not be required to, explicitly account for the impact of vaccination on transmission dynamics. the assumption is that population vaccine coverage will be relatively low even in large trials, and that the rank ordering of sites is similar in less complex models. this is a recommended minimum structure, but other relevant practical questions will likely emerge that can be explored as add-ons. for example, the modeling results can be used to answer questions about expected duration of the trial as a function of enrollment rates and expected incidence. nonetheless, it is important to remember that projections can be very uncertain, particularly as they depend upon rapidly changing policies and human behavior. thus, we focus on simple output for the purposes of prioritization, acknowledging that other important questions may be difficult to answer precisely. in addition to identifying geographic locations, models could also be used to explore targeted enrollment in sub-populations defined by age, occupation, or other covariates. models could also guide the design of post-licensure observational studies for continued evaluation of vaccine effectiveness. it is a top priority to rapidly evaluate the safety and efficacy of candidate covid- vaccines. data-driven models can help to optimize site selection and contribute to accelerating trials in a setting where every day counts. developing covid- vaccines at pandemic speed design of vaccine efficacy trials during public health emergencies world health organization. an international randomised trial of candidate vaccines against covid- antibody testing will enhance the power and accuracy of covid- -prevention trials simulations for designing and interpreting intervention trials in infectious diseases statistical power and validity of ebola vaccine trials in sierra leone: a simulation study of trial design and analysis preliminary results of models to predict areas in the americas with increased likelihood of zika virus transmission in spread of zika virus in the americas prediction of infectious disease epidemics via weighted density ensembles accuracy of real-time multi-model ensemble forecasts for seasonal influenza in the u.s an open challenge to advance probabilistic forecasting for dengue epidemics the rapidd ebola forecasting challenge: synthesis and lessons learnt ensemble forecasts of coronavirus disease (covid- ) in the u creating a framework for conducting randomized clinical trials during disease outbreaks ring vaccination with rvsv-zebov under expanded access in response to an outbreak of ebola virus disease in guinea, : an operational and vaccine safety report a trial of lopinavir-ritonavir in adults hospitalized with severe covid- demonstrating vaccine effectiveness during a waning epidemic: a who / nih meeting report on approaches to development and licensure of zika vaccine candidates mathematical modeling of the west africa ebola epidemic funding: this work was supported by the national institutes of health r -ai (ned, meh, iml, conceptualization, writing -review and editing. ☒ the authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.☐the authors declare the following financial interests/personal relationships which may be considered as potential competing interests: key: cord- -gmitz gg authors: clemens, john d.; ochiai, r. leon title: sequential stages of clinical trials and overview of issues to be considered date: journal: the grand challenge for the future doi: . / - - - _ sha: doc_id: cord_uid: gmitz gg nan we live in an exciting era in which the fruits of molecular biology and biotechnology are yielding a profusion of new and improved candidate vaccines, delivery systems, and adjuvants that have the potential to control many of the infectious disease scourges of the developing world. indeed, it has been estimated recently that there are over vaccine candidates currently under development against nearly infectious diseases [ ] . discovery of a new vaccine candidate and demonstration of its safety, immunogenicity, and protectivity in animal models are, however, only the first steps toward licensure and introduction of the vaccine into public health practice. the longest phase of development for most vaccines before licensure, and arguably the most uncertain, laborious, and expensive phase, comprises clinical testing of the vaccine in humans [ ] . at a minimum, this clinical testing must demonstrate the vaccine to be acceptably safe and suitably protective in the population that will ultimately be targeted for the vaccine in public health practice [ ] . over the years, regulatory agencies have adopted a standard paradigm for the manner in which new vaccines are tested in humans. a key feature of this paradigm is the phased fashion in which the testing occurs. this chapter describes the phases of vaccine evaluations in humans, including several considerations for the phased testing of vaccines in developing countries. vaccines have had a remarkable track record of safety. nevertheless, even when manufactured flawlessly, some vaccines have caused serious sideeffects [ ] . the mechanisms for these side-effects are diverse and are sometimes related to such problems as vaccine-induced immunopathology, as occurred with early-generation measles and respiratory syncytial virus vaccines [ , ] . at times the mechanism may be obscure, as was the case for sequential stages of clinical trials and overview of issues to be considered intussusception induced by quadrivalent rhesus reassortant vaccine against rotavirus diarrhea [ , ] . whatever the mechanism, the point to be noted is that such severe side-effects are not always predictable. to minimize potential injuries to subjects caused by vaccine side-effects during re-licensure trials, vaccines are tested in a phased manner. in this phasing, early evaluations are conducted in small numbers of subjects, so that if reactions are observed, they will affect a minimum number of volunteers. and early evaluations are typically conducted in the least vulnerable subjects, often healthy adults, so that if the reactions occur, their severity will be minimized. successive evaluations of a particular vaccine are then conducted in progressively larger numbers of volunteers, and, as confidence in the safety-profile of the vaccine increases, in subjects in the ultimate target group, including persons who are more vulnerable (e.g., infants). with these successive studies, the complete ensemble of information about the vaccine's safety, immunogenicity, protectivity, and sometimes additional characteristics (e.g., transmissibility) is accrued. moreover, the large number of subjects ultimately studied ensures estimation of these features in a statistically precise fashion, and, for sideeffects, in a way that enables detection of relatively rare events. clinical trials for licensing a new vaccine candidate are generally planned in three phases. in the code of federal regulations (usa), the phases of clinical trials are described by using arabic numerals (phase , phase , phase ), while in world health organization publications, roman numerals are generally used (phase i, phase ii, phase iii). in this paper, we use roman numerals to describe the phases of clinical trials. phase i trials of experimental vaccines are the first human studies to be conducted after preclinical studies have demonstrated suitable safety, potency, immunogenicity, and, when possible, protectivity. the primary purpose of a phase i trial is to rule out the possibility of frequent vaccine sideeffects. additional goals include preliminary assessment of vaccine immunogenicity, determination of an appropriate dose and regimen, and, for live vaccines, measurement of vaccine shedding. phase i trials are typically small, often on the order of - subjects, usually enroll healthy adults, and may be done with preliminary formulations of the vaccine. depending on whether there are concerns about potential severe sideeffects and the need for biological containment of excreted vaccines, such studies may be done on an inpatient or an outpatient basis. for example, a genetically attenuated, live oral vaccine candidate might well be tested initially on an inpatient basis with containment if there is concern about transmission of fecally excreted vaccine organisms, or about the genetic stability of the candidate during the course of fecal shedding [ ] . often, phase i studies are designed in an uncontrolled fashion. for vaccine candidates that are found to yield promising findings in phase i trials, phase ii trials may be undertaken. the primary goals of phase ii trials are to evaluate vaccine safety and immunogenicity in larger numbers of subjects, and ultimately in the target population for whom the vaccine is intended. for live vaccines, phase ii trials may also be designed to evaluate vaccine shedding and transmissibility. phase ii trials may also further evaluate different vaccine doses and regimens. a frequently used strategy for phase ii trials of vaccine candidates intended for infants is to initiate the studies in an older age group, often adults, and to conduct successive studies in progressively younger age groups, with the transition to each younger age group contingent on satisfactory results from the study of the previous age group. in contrast to phase i trials, which are often done with preliminary formulations, phase ii trials typically evaluate the final formulation of the vaccine, since data from studies of preliminary formulations are not usually eligible for consideration by regulatory authorities in their deliberations about vaccine licensure. phase ii trials are typically larger than phase i trials, sometimes enrolling or more subjects. by the time that a vaccine candidate has reached phase ii, concerns that would mandate testing it on an inpatient basis have typically been resolved, and outpatient studies are the norm. in contrast to phase i trials, phase ii trials are conventionally designed as randomized, controlled trials, and control groups typically receive a placebo or an active agent to permit blinding. for live vaccine candidates, there may be a concern about unintended transmission of the vaccine strain from vaccinees to non-vaccinees with whom they are in contact. special phase ii trials are sometimes conducted to assess the transmissibility of such vaccine candidates. for example, a phase ii study of the live oral cholera vaccine, cvd -hgr, in jakarta, indonesia, randomly allocated pairs of sibling children within households to either vaccine or placebo, and judged transmissibility of the vaccine candidate by the rate of fecal excretion of the vaccine strain and seroconversion to the vaccine in placebo recipients [ ] . phase ii trials may also be done to address environmental concerns about genetically engineered, live vaccine candidates. for cvd -hgr, for example, open sewers outside the jakarta homes of children who had received the vaccine were sampled to evaluate whether there was any detectable persistence of the vaccine strain during the - hours after vaccination [ ] . a recent trend in vaccine evaluation is to use phase ii trials to obtain initial data on the level of vaccine protection against targeted, naturally occurring infections. what distinguishes these phase ii trials from conventional phase iii trials (vide infra) is that they are smaller in size (although usually larger than phase ii trials geared only to the assessment of safety and immunogenicity) and less able to evaluate vaccine protection with suitable statistical precision than well-designed phase iii trials. also in contrast with phase iii trials, such phase ii trials may be used to evaluate vaccine prototypes that will be later modified or augmented into final vaccines for licensure (e.g., a vaccine against a single serotype, when a multi-serotype vaccine will be required for the final vaccine), and may attempt to obtain estimates of vaccine protection based on prevention of a surrogate endpoint (e.g., the use of hiv viral load as a surrogate for the rapidity of hiv disease progression). for certain vaccines, studies are done in volunteers to evaluate the clinical protection against an intentional challenge with the target pathogen. such studies are sometimes termed phase iib trials. in these studies volunteers are typically allocated at random to receive the vaccine or a comparison agent, usually a placebo, and are then challenged at a defined interval after vaccination with an inoculum of the pathogen predicted to cause the target disease in nearly % of the control group. the comparative attack rate of the target disease in vaccinees versus controls provides an estimate of the conventional measure of vaccine protective efficacy (pe) = ( minus the relative risk of the disease in vaccinees versus controls) × %. estimates of vaccine protection in phase iib studies, which are typically small, often conducted in ca. - subjects, may be helpful in triaging vaccines that are deserving of study in larger and more expensive phase iii trials [ , ] . in addition to providing estimates of vaccine protection, phase iib trials can serve to provide data on vaccine safety, vaccine immunogenicity, vaccine shedding and transmissibility (for live vaccines), and preliminary assessments of immunological responses that correlate with protection. because such studies entail intentional challenge with pathogens, subjects for the studies should always be healthy adults. the intentional challenge also limits phase iib trials to infections for which there is no risk of severe acute complications, significant sequelae, or chronic infection if appropriate therapy is administered promptly upon recognition that the challenge has resulted in infection. examples of pathogens for which phase iib studies have been successfully carried out include cholera, diarrheagenic escherichia coli, shigella, rocky mountain spotted fever, malaria, and influenza [ ] . it is imperative that such studies be carried out by staff who are highly skilled in the diagnosis and treatment of the infections under study, and it is usually desirable to conduct the challenge phase of these studies under inpatient conditions. for vaccine candidates found to be suitably safe and immunogenic in phase ii trials, phase iii trials may be done to provide rigorous evidence about vaccine protection against naturally occurring infections, and to provide additional data on vaccine safety in larger numbers of vaccinees. phase iii studies are designed as randomized, controlled trials with clear hypotheses, and are conducted in the target group for whom vaccine licensure is desired and in a population that normally experiences the target infection. these studies thus constitute pivotal evaluations that provide the basis for decisions about whether to license a vaccine for use in public health practice [ ] . figure diagrammatically depicts the design of a phase iii trial comparing two groups. in such a trial participants are recruited from a target population and are enrolled for the study after acquisition of informed consent and ascertainment of eligibility. prior to the trial, a formal randomization scheme is developed to allocate subjects to an experimental vaccine group or a control group, and this randomization scheme is used to allocate the compared agents to consenting subjects who are eligible for participation. to ensure blinding of investigators and subjects to the identities of the compared agents, controls may receive an inert placebo or an active vaccine. if the latter is chosen, it is typically an agent that is identical in appearance to and given with the same regimen as the experimental vaccine, but does not elicit immune responses that are known to protect against the pathogen targeted by the experimental vaccine. after randomization, subjects in the compared groups are followed concurrently with uniform surveillance procedures to detect target infections and adverse events, and the target infections adverse events immune responses figure . a simplified schematic of a phase iii vaccine trial designed as a two-group, randomized, controlled trial. in such a trial, the study population is assembled from a target population and is then randomized to receive an experimental vaccine or a control agent. the experimental and control groups are followed longitudinally and concurrently to detect the comparative occurrence of target infections, adverse events, and immune responses in the two groups to assess vaccine protection, vaccine safety, and vaccine immunogenicity, respectively. comparative rates of these events in the two groups form the basis for the assessment of vaccine pe and safety. similarly, participants, or a subsample of participants, are assessed immunologically at baseline and at a defined interval after dosing. because phase iii trials must provide statistically meaningful estimates of vaccine pe (calculated in the same way as described above for phase iib trials) and because the target disease outcomes to be prevented by vaccination in the trials are typically rare in occurrence, phase iii trials are often quite large, sometimes enrolling tens of thousands of subjects. in addition, because it may be necessary to measure vaccine pe over several years following vaccination in order to provide information necessary to convince regulatory and public health authorities, phase iii trials may entail maintenance and follow-up of study populations for long durations. the large size, lengthy duration, prospective conduct, and extensive quality control and quality assurance procedures needed for phase iii trials make them extremely expensive, often costing millions of dollars. measurement of immune responses in phase iii trials is done for two purposes. first, it is necessary to document that the vaccine tested in a phase iii trial elicited the level of immune response expected on the basis of earlier studies. comparison of immune responses in the vaccine and control groups permits estimation of the proportion of immune responders among the experimental vaccine group that can be attributed to receipt of the vaccine. if vaccine pe proves unexpectedly low in the trial, this assessment will be crucial in helping to explain these results. assessments to assess whether immune responses were as robust as expected usually require evaluation only of a small subsample of subjects participating in the trial. second, it is often desirable that a trial define an immunological correlate of vaccine protection -a level of short-term immune response to vaccination that demarcates vaccinees who are protected against the target infection. immunological correlates of protection are best defined by comparing immune responses to the vaccine in vaccinees who develop the target infection (breakthroughs) versus vaccinees who do not. definition of immunological correlates of vaccine protection is very important because such correlates permit assessments of the protection of the tested vaccine and ones suitably similar to it in small, short-term studies with immunological endpoints, without resort to full-scale, phase iii efficacy trials with clinical infection endpoints. phase iii trials usually provide the only opportunity before licensure for determining an immunological correlate of protection. unfortunately, determination of immunological correlates within phase iii trials can be logistically demanding and costly. these problems result from four considerations: ) it is necessary to contrast short-term immune responses in vaccinees who develop the target infection versus those who do not in order to determine an immunological correlate of protection; ) at the time of vaccination and collection of specimens for immunological assessments it is unpredictable which vaccinees will acquire the target infection ("breakthrough events"); ) most phase iii trials have very few breakthrough events; and ) blinding of phase iii trials prevents knowledge of who received the vaccine and who received the control agent. the implication of these considerations is that it may be necessary to obtain suitable specimens from virtually every participant in the trial to ensure that immunological assessments will be available for a large enough number of vaccine breakthroughs to enable statistically meaningful assessments. moreover, even when arrangements are made to collect the necessary specimens from the required number of subjects, measurement of immune responses in these specimens may not necessarily yield an immunological correlate of the vaccine protection observed in the trial, sometimes for obscure reasons [ ] . experience has demonstrated that the performance of a vaccine in affluent populations in industrialized countries cannot be assumed to generalize to persons living in developing-country settings. examples of vaccines that performed less well in developing-country than in industrialized-country populations have included oral polio vaccine, certain live oral rotavirus and cholera vaccines, and parenteral, polysaccharide-diphtheria protein conjugate haemophilus influenzae type b (hib) vaccine [ ] [ ] [ ] [ ] . it is therefore important that new vaccines be tested in developing countries prior to their licensure in these settings. this raises the question, for vaccine candidates developed in the industrialized world, of when in the phased sequence the candidate should be tested in the developing world. traditionally, it has been argued, partly over concerns about using poor populations in the developing world as "human guinea pigs", that trials in the developing world should commence only when a vaccine has been fully evaluated and licensed in the industrialized world. recent changes in thinking have challenged this view. it is now recognized that deferring trials in developing countries to this extent carries the unwanted consequence of significantly delaying the availability of vaccines to populations in developing countries. for example, polysaccharideprotein vaccines against hib, which have been licensed for over a decade in the united states and other industrialized countries and have nearly eliminated invasive hib disease in children in many of these settings, are used only to a limited extent today in the developing world [ ] . this inequity in use of new-generation vaccines has elicited calls for parallel testing, rather than sequential testing, of new vaccines in industrialized-and developing-world populations. parallel-testing of new vaccines in the industrialized and developing worlds is now being pursued for both experimental hiv and rotavirus vaccines [ , ] . another important trend affecting the testing of vaccines in developingcountry populations is the emergence of highly qualified vaccine producers in the developing world, some capable of developing innovative new vaccines. clearly, it is appropriate that the initial groups for testing of an experimental vaccine developed by such producers be selected from the population of the country in which the vaccine is developed. conventional descriptions of phased trials often portray a seamless progression of trials from phase i to phase ii, and from phase ii to phase iii. in reality, the vast majority of vaccine candidates that are tested in phase i studies do not progress all the way through phase iii testing, because of disappointing findings or because of a lack of sufficient resources to support the increasingly expensive studies in successive phases [ ] . while unwelcome findings at any stage may lead to termination of the clinical development of a vaccine, this need not always be the case. for example, the live-attenuated, oral cholera vaccine, cvd -hgr, was shown to be safe and highly immunogenic when given as a single dose of × organisms in an extensive series of studies in north american volunteers. however, when the vaccine was subsequently tested in thai adult volunteers, it proved erratically immunogenic [ ] . to address this problem, the vaccine developer raised the dose by one log, to × organisms, and found the vaccine to be suitably immunogenic but still safe in a variety of settings in developing countries [ ] . the cvd -hgr experience illustrates that successful vaccine development may require detours in the phased sequence of clinical trials in order to accommodate necessary adjustments in vaccine dose, schedule, or even formulation in response to findings that emerge during the course of clinical studies. however, because vaccines that appear promising in developed countries not infrequently fail to perform as well in developing countries, and because such detours can greatly increase the expense of clinical development, completion of clinical testing of vaccines for developing countries may be exceptionally challenging. the conventional sequence of phases usually culminates in one or more phase iii trials that are designed to provide pivotal evidence on the clinical protection conferred by the vaccine against the targeted, naturally occur-ring, infectious disease. sometimes, however, proof of clinical efficacy in a phase iii trial may not be necessary for licensure of a vaccine. for example, cvd -hgr vaccine against cholera was licensed in europe for use in travelers on the basis of phase i-ii studies, together with phase iib trials showing a high level of protection against experimental cholera in north american volunteers [ ] . however, it is unlikely that phase iib evidence from industrialized countries will be taken as a basis for licensure of vaccines in developing countries, in part because of the discrepancies between vaccine performance in developed versus less-developed settings and in part because of the uncertainties about whether protection in phase iib studies predicts the performance of a vaccine in populations experiencing endemic disease. in addition, if there is an accepted correlation between a certain immune response (e.g., a serological antibody titer) to vaccination and clinical protection, this immune response may sometimes be used as prima facie evidence for judging whether the vaccine is likely to be sufficiently protective to warrant licensure. for example, prp-crm diptheria toxin and prp-neisseria meningitidis outer membrane protein (omp) conjugate vaccines against hib were licensed for infants on the basis of phase iii evidence of clinical protection against invasive hib disease. subsequently, prp-tetanus toxoid conjugate vaccine was licensed for infants in the united states largely on the basis of proven safety and the attainment of protective serum antibody titers to prp, without corresponding evidence of clinical protection from a phase iii trial [ , ] . serological endpoints are also commonly used as a basis for licensure of combination vaccines for which the component vaccines have already been licensed and seroresponse criteria for these components have been established [ ] . some vaccines of public health importance are directed to diseases that cannot be reliably predicted in populations at risk and that cannot be ethically studied in phase iib trials. examples include vaccines against some bioterrorism agents such as anthrax and plague, and vaccines against certain epidemic diseases such as sars and ebola. in these situations, decisions about licensure will have to depend on pre-clinical data, demonstration of safety in humans, and assessments of immune responses in humans (even if such immune responses are not known with certainty to be correlated with clinical protection of humans against the target disease). in recent years, considerable attention has been devoted to the ethics, scientific quality, conduct, documentation, and reporting of clinical trials. several organizations have produced guidelines for these activities, which are subsumed under the rubric of "good clinical practice (gcp)". guidelines for gcp have been published by the world health organization, the united states food and drug administration, and the international conference on harmonization (ich), among others [ ] . table presents the essential elements of gcp in guidelines issued by the international conference on harmonization [ ] . the promulgation of gcp has had several implications for vaccine trials in developing countries. increasingly, the world of vaccines is becoming globalized. vaccines targeted for a global market may be tested in developing countries to provide initial licensure. for example, a live oral human rotavirus vaccine developed by a major multinational vaccine producer has recently received its initial licensure in mexico. moreover, vaccines produced by manufacturers in developing countries are now being used throughout the world. indeed, the majority of all doses of measles vaccines now administered to children in the world are produced in india. with the trend toward globalization of vaccines, increasing emphasis has been placed on conducting clinical trials in developing countries in a manner that conforms to international gcp standards. while the benefits of gcp trials are unquestionable, rigid adherence to gcp standards can be a double-edged sword. the expense of clinical trials has risen rapidly in recent years [ ] . a portion of this increased expense arises from the extensive documentation and auditing requirements demanded by regulatory agencies as a component of gcp, together with the expense of proprietary products developed to ensure compliance with regulatory requirements, such as proprietary computer software for data management and electronic reporting of trials. while this increased expense can be borne by producers anticipating major markets for their vaccines in affluent markets, it constitutes a disincentive to the clinical testing of vaccines against "orphan diseases" affecting developing countries, for which lucrative markets are not foreseen. the ethical guidelines for clinical trials in developing countries have been the subject of recent controversy. the ethical principles of respect for persons, beneficence, and justice, which underlie modern clinical trials, are well accepted [ ] . nevertheless, several issues continue to be debated, of which three are touched on here [ ] [ ] [ ] . one issue concerns the threshold for the risk/benefit ratio for a vaccine to be tested in developing countries. for example, a live oral, quadrivalent rhesus rotavirus-reassortant vaccine was licensed and then withdrawn from the united states market by its manufacturer because of an association between vaccination and the rare occurrence of intussusception [ , ] . understandably, this action had a chilling effect on the testing of this vaccine in developing countries. a meeting of experts organized by the world health organization acknowledged the rationale for the company's withdrawal of this vaccine in the united states, where rotavirus diarrhea is a cause of morbidity but not of significant mortality. nevertheless, there was agreement among participants in the meeting that the burden of mortality of rotavirus in developing countries provided an ethical justification for continued evaluation of this vaccine in these settings [ ] . a second controversy over the ethics of trials in developing countries has focused on the use of placebos. as stated in the declaration of helsinki, "the benefits, risk, burdens, and effectiveness of a new method should be tested against the best current prophylactic, diagnostic, and therapeutic methods" [ ] . there is agreement that a placebo would be inappropriate as a control agent for a trial if a locally licensed standard alternative vaccine exists; there is also general agreement that if such an alternative does not exist, it would be preferable to use for the control group an active vaccine against a different target infection, provided the active control does not cross-protect against the target pathogen and that use of the active vaccine maintains blinding in the trial [ ] . however, it remains controversial whether a placebo can be ethically administered to the control groups in trials for which there is no suitable alternative active control vaccine and in which a licensed alternative vaccine against the target infection exists, but the alternative is not licensed in the country for the trial [ ] . yet a third controversy concerns what is owed to the participants at the conclusion of a vaccine trial. the declaration of helskinki states that: "at the conclusion of a study, every patient entered into the study should be table . principles of good clinical practice (gcp). from [ ] . clinical trials should be conducted in accordance with the ethical principles that have their origin in the declaration of helsinki and that are consistent with gcp and the applicable regulatory requirement(s). . before a trial is initiated, foreseeable risks and inconveniences should be weighed against the anticipated benefit for the individual trial subject and society. a trial should be initiated and continued only if the anticipated benefits justify the risks. . the rights, safety, and well-being of the trial subjects are the most important considerations and should prevail over interests of science and society. . the available nonclinical and clinical information on an investigational product should be adequate to support the proposed clinical trial. . clinical trials should be scientifically sound and described in a clear, detailed protocol. . a trial should be conducted in compliance with the protocol that has received prior institutional review board (irb)-independent ethics committee (iec) approval-favorable opinion. . the medical care given to and medical decisions made on behalf of subjects should always be the responsibility of a qualified physician or when appropriate, a qualified dentist. . each individual involved in conducting a trial should be qualified by education, training, and experience to perform his or her respective task(s). . freely given informed consent should be obtained from every subject prior to clinical trial participation. . all clinical trial information should be recorded, handled, and stored in a way that allows its accurate reporting, interpretation, and verification. . the confidentiality of records that could identify subjects should be protected, respecting privacy and confidentiality rules in accordance with the applicable regulatory requirement(s). . investigational products should be manufactured, handled, and stored in accordance with applicable good manufacturing practice (gmp). they should be used in accordance with the approved protocol. . systems with procedures that assure the quality of every aspect of the trial should be implemented. assured of access to the best proven prophylactic, diagnostic, and therapeutic methods identified by the study" [ ] . few would disagree that if the vaccine tested in a trial is found to be beneficial, it should be provided to all participants in the trial. however, controversy surrounds the boundaries for the group of persons entitled to receive the investigational vaccine at the conclusion of the study -only participants in the trial, all persons living in the same region, or all persons living in the same country? and for how long are these beneficiaries entitled to receive the vaccine? increasing expectations for demonstration of vaccine safety before licensure because vaccines are administered in public health practice to healthy individuals, most commonly children, there is a higher expectation of safety for vaccines than for many therapeutic agents. in the past, pre-licensure phase iii trials were often designed merely to evaluate the safety profile of a vaccine observed in previous studies. accordingly, the focus of such trials was on evaluating events that were expected on the basis of earlier studies, that occurred during an interval immediately following vaccination, and that were seen with appreciable frequency. indeed, a common tactic in older phase iii vaccine trials was to evaluate adverse events only in a small subsample of the total trial population, obviating the possibility of detecting rare but potentially significant side-effects. modern regulatory agencies now usually require that surveillance for adverse events be undertaken for the total study population rather than for a subsample, at least for events that can be detected passively (e.g., among persons presenting to health care centers for treatment), and that this surveillance be maintained for the duration of the trial. increasing concern about vaccine safety has also led to other changes in the design of phase iii trials. formerly, calculation of sample sizes for such trials was usually geared to enable detection of a certain level of vaccine protection against the target illness, with an acceptable level of statistical power. detection of rare side-effects was not a typical goal of such calculations. the recent experience with the live oral, quadrivalent rhesus rotavirus reassortant vaccine, which was withdrawn from the united states market because of an association with a small but statistically significant risk of intussusception [ , ] , has led to more stringent regulatory requirements for pre-licensure trials of newer-generation, live oral rotavirus vaccines. the united states food and drug administration, for example, has required that these newer-generation vaccines be tested in numbers of infants sufficiently large to be able to detect whether intussusception is a vaccine side-effect. current trials of these newer vaccines have therefore enrolled samples of infants many times the number enrolled in pre-licensure trials of the live oral, quadrivalent rhesus rotavirus reassortant vac-cine, resulting in greatly increased clinical development costs. whether the experience with rotavirus vaccines portends regulatory requirements for substantially larger phase iii trials for other new-generation vaccines remains to be seen. the experience in the united states with the live oral, quadrivalent rhesus rotavirus reassortant vaccine illustrates yet another issue confronting pre-licensure trials. data from pre-licensure trials of this vaccine showed a suggestive but non-significant elevation of the risk of intussusception [ ] . the united states food and drug administration licensed the vaccine with the proviso that the occurrence of this potential side-effect be scrupulously monitored post-licensure. this was possible because of the well-developed systems for post-licensure surveillance for vaccine side-effects established in the united states [ , ] . unfortunately, post-licensure surveillance systems capable of detecting and evaluating rare but serious vaccine adverse reactions are absent in most developing countries. the weaknesses of post-licensure surveillance systems in developing countries place an even greater onus upon pre-licensure trials in these settings to provide reassurance that a vaccine will not cause rare but serious side-effects. however, the large size and great expense of trials capable of detecting rare side-effects present a major dilemma, especially for vaccines against diseases that are primarily limited to the developing world, for which resources for clinical trials are scarce. clearly, there is an urgent need to develop improved systems of post-licensure surveillance capable of detecting rare but significant vaccine adverse reactions in developing countries. recently work has begun to evaluate the methodological challenges and practical feasibility of establishing large population-based, dynamic computerized databases for evaluating potential vaccine side-effects in developing-country settings. such databases link vaccination histories to severe disease outcomes in defined cohorts, and have been used successfully in industrialized countries for rapid and credible evaluations of putative vaccine adverse reactions [ ] . one such database has been successfully established on a pilot scale in vietnam [ ] . overcoming the challenges of establishing such databases in the diverse settings of the developing world remains a significant but important research agenda. the basic phases of testing of vaccine candidates are relatively straightforward and widely accepted. the successive phases of clinical evaluation of vaccine candidates allow for acquisition of critical information about vaccine safety, immunogenicity, excretion, transmission, and protection in an incremental fashion, while minimizing the risks to subjects who volunteer to participate in these studies. in recent years we have seen a burgeoning of vaccine candidates against diseases of developing countries, creating breathtaking possibilities for disease prevention in these settings. at the same time, this profusion of vaccine candidates for the developing world has added a layer of complexity to the seemingly straightforward phased sequence of trials. the factors underlying this complexity are multiple, and include considerations about when in the phased testing sequence a vaccine developed in an industrialized country should begin testing in developing countries; about whether and how to pursue the clinical development of a vaccine when disappointing clinical results are observed in developing countries; about how to meet the scientific, financial, logistical, and ethical challenges of conducting clinical trials in developing countries that conform to contemporary international standards of good clinical practice; and about how to ensure that a vaccine is acceptably safe before and after licensure. the manner in which these issues are addressed in the future will have a great bearing on the success of efforts to accelerate the introduction of new-generation vaccines into programs for the poor in developing countries. the jordan report: accelerated development of vaccines. the institute vaccine economics: from candidates to commercialized products in the developing world long-term evaluation of vaccine protection:methodological issues for phase iii and iv studies the hazards of immunization an epidemiological study of altered clinical reactivity to respiratory syncytial (rs) virus infection of children previously vaccinated with an rs virus vaccine measles immunization with killed virus vaccine, serum antibody titers and experience with exposure to measles epidemic intussusception among infants given an oral rotavirus vaccine population-based study of rotavirus vaccination and intussusception safety, immunogenicity, and efficacy of recombinant live oral cholera vaccines, cvd and cvd -hgr the safety, immunogenicity, and transmissibility of single-dose live oral cholera vaccine cvd -hgr in - month old indonesian children the ethical challenge of infection-inducing challenge experiments infectious disease experimentation involving human volunteers initial clinical evaluation of new vaccine candidates: investigators' perspective of phase i and phase ii clinical trials of safety, immunogenicity, and preliminary efficacy evaluating new vaccines for developing countries: efficacy or effectiveness? protective effect of two acellular pertussis vaccines in a double-blind, placebo-controlled trial factors affecting the immunogenicity of oral polio vaccine in developing countries: a review trial of an attenuated bovine rotavirus vaccine (rit ) in gambian infants limited efficacy of a haemophilus influenzae type b conjugate vaccine in alaskan native infants. the alaska h. influenzae vaccine study group efficacy of a single dose of live oral cholera vaccine cvd -hgr in north jakarta, indonesia, a cholera-endemicarea global reduction of hib disease: what are the next steps? designing hiv vaccines for developing countries the future of rotavirus vaccines: a major setback leads to new opportunities safety and immunogenicity of different immunization regimens of cvd -hgr live oral choleravaccine in soldiers and civilians in thailand randomized, double-blind, placebo-controlled, multicentered trial of the efficacy of a single dose of live oral cholera vaccine cvd -hg-r in preventing cholera following challenge with vibrio cholerae el tor inaba three months after vaccination the efficacy in navajo infants of a conjugate vaccine consisting of haemophilus influenzae type b polysaccharide and neisseriae meningitidis outer-membrane protein complex efficacy in infancy of oligosaccharide conjugate haemophilus influenzae type b (hiboc) vaccine in a united states population of , children. the northern california kaiser permanente vaccine study center pediatrics group prelicensure evaluation of combination vaccines international conference on harmonisation expert working group ( ) international conference on harmonisation guideline for good clinical practice. e . the conference the future of vaccines: an industrial perspective the national commission for the protection of human subjects of biomedical and behavioral research ( ) the belmont report: ethical principles and guidelines for the protection of human subjects of research. the commission the ethics of clinical research in the third world ethical issues in the design and conduct of clinical trials in developing countries ethical complexities of conducting research in developing countries the future of research into rotavirus vaccine declaration of helsinki: ethical principles for medical research involving human subjects. the association joint united nations programme on hiv/aids ( ) ethical considerations in hiv preventive vaccine research when are placebo-controlled trials ethically acceptable? public health considerations for the introduction of new vaccines against rotavirus: a case study of the rhesus rotavirus-reassortant vaccine the vaccine adverse event reporting system (vaers) vaccine safety datalink project: a new tool for improving vaccine safety monitoring in the united states the vaccine data link in nha trang, vietnam: a progress report on the implementation of a database to detect adverse events related to vaccinations key: cord- -iy h pyf authors: nasrallah, ali a.; farran, sarah h.; nasrallah, zainab a.; chahrour, mohamad a.; salhab, hamza a.; fares, mohammad y.; khachfe, hussein h.; akl, elie a. title: a large number of covid- interventional clinical trials were registered soon after the pandemic onset: a descriptive analysis date: - - journal: j clin epidemiol doi: . /j.jclinepi. . . sha: doc_id: cord_uid: iy h pyf abstract background there is a pressing need for evidence-based interventions to address the devastating clinical and public health effects of the coronavirus disease (covid- ) pandemic. the number of registered trials related to covid- is increasing by the day. objectives to describe the characteristics of the currently registered clinical trials related to covid- . methods we searched the world health organization (who)’s international clinical trials registry platform (ictrp) on may , . we included any entry that is related to covid- . we abstracted then descriptively analyzed the following characteristics of the registered trials: study design, status, phase, primary endpoints, experimental interventions, and geographic location among other qualifiers. results we identified , eligible registered trials. the majority of trials were initially registered with clinicaltrials.gov (n= ; . %) and the chinese clinical trial registry (chictr) (n= ; . %). the number of participants to be enrolled across these trials was , , with a median of participants per trial. the most-commonly studied intervention category was pharmacologic (n= ; . %), with antiparasitic medications being the most common subcategory. while over half of trials were already recruiting, we identified published peer-reviewed results for only of those trials. conclusion there is a relatively large number of registered trials but very few results published so far. while our findings suggest an appropriate initial response by the research community, the real challenge will be to get these trials completed, published, and translated into practice and policy. in december , the chinese city of wuhan witnessed the outbreak of a pneumonia of unknown origin [ ] . the outbreak was traced back to wuhan's seafood market [ ] , and characterized by a strong person-to-person transmission [ ] . subsequently, scientists identified a new strain of coronavirus, the severe acute respiratory syndrome coronavirus (sars-cov- ), as the source of the outbreak [ ] . sars-cov- is a novel member of the beta coronavirus family, which includes sars-cov, source of an outbreak in , and mers-cov, the origin of an outbreak in saudi arabia in [ ] . on th of february , the world health organization (who) announced coronavirus disease (covid- ) as the name of this new disease [ ] . within three months, covid- outbreak had already affected six continents [ ] , and the who upgraded its status from epidemic to pandemic on march , [ ] . as of may th , there have been , , confirmed cases and , deaths [ ] . patients infected with covid- present with a wide spectrum of clinical presentations [ ] , ranging from no symptoms to acute respiratory distress syndrome (ards) and death [ ] . in parallel, with the high infectivity rates of the virus have led to the overstraining of healthcare systems [ ] [ ] [ ] . supportive management remain the pivot of treatment protocols in the absence of evidence on efficacious antiviral or anti-inflammatory medications [ ] . to date, most recommendations on preventing disease transmission and treating infected patients are based on anecdotal evidence and experts' opinions. randomized control trials (rcts) are needed to provide unbiased evidence to guide the clinical care and public health practices aimed to control covid- outbreak [ ] . analyzing the status of clinical trials is a way of describing the current status of research in a scientific field, assessing the direction and magnitude of progress, and identifying potential gaps in interventional research [ ] . thus, this study aimed to describe the characteristics of the currently registered clinical trials related to covid- . we used the who international clinical trials registry platform (ictrp) database [ ] to identify on all covid- clinical trials and retrieve related information. the ictrp is a network of international clinical trial registers which ensures single-point access and unambiguous identification of trials [ ] we included all records retrieved on may th , from the ictrp that were labeled as one of the following study types: interventional, screening, prevention, treatment. we excluded the following: trials not directly related to treating or preventing covid- disease, noninterventional trials, and the following types of trials; basic science, diagnostic test, epidemiological research, expanded access, health-services, observational, and prognosis. we exported for each record all the variables reported by ictrp (see appendix a). we included the following variables for our analysis: study id, source register unique identifier, original registry, public title, primary sponsor, location (country and region), recruitment status, age range, gender, target size, study design, phase, publication (yes/no, count, and url), intervention (category, subcategory, and name), primary outcomes, registration date, enrollment date, retrospective label, and trial url. using the source register unique identifier numbers, we verified the data exported from the ictrp datafile and collected any missing data. then, two investigators (aan and hhk) categorized in duplicate and independently the intervention variables into detailed subcategories, as shown in appendix b. similarly, they categorized outcomes into the following types: mortality, morbidity, patient-reported, surrogate, composite, and other. in addition, we searched for publications related to the eligible trials. we used the source register unique identifier to search for peer-reviewed publications related to the eligible trials (on pubmed, medline, embase, and scopus), and for pre-print articles (on medrxiv and osf) [ , ] . two investigators (hhk and zan) reviewed potentially relevant peer-reviewed publications and pre-print articles independently to confirm their relatedness to the eligible trials. the complete covid- file retrieved from ictrp database included a total of , records. we excluded , records for the following reasons: non-interventional trials (n= , ); cancelled/withdrawn/suspended/terminated/retracted trials (n= ), not directly related to covid- (n= ), duplicate records (n= ), not found in source registry (n= ). as a result, , records met our eligibility criteria. figure shows the time distributions of the cumulative number of registered trials with the cumulative number of confirmed cases of covid- [ ] . while the former follows an exponential growth pattern, the latter follows an arithmetic growth pattern. case [ ] . it is worth mentioning that seven trials started between the years of and , respectively, and adjusted their protocols and eligibility criteria to include covid- patients. table : characteristics of registered trials stratfitied by phase, and across phases (n= ) table : characteristics of assessed interventions stratified across phases (n= ) table provide a re-categorization of the types of primay outcomes assessed in the eligible trials. overall, the majority of trials planned to include morbidity outcomes as primary outcomes (n= ; . %). the next types by order of frequency were surrogate outcomes (n= ; . %), mortality (n= ; . %), and composite outcomes (n= ; . %). of these trials, ( . %) had surrogate only outcomes, ( . %) had morbidity-only outcomes, while trials ( . %) had both. only one study did not report any primary outcomes. table : types of primay outcomes in the eligible trials stratified by phase male < female < we have described the characteristics of currently registered clinical trials related to covid- . the trials were planned in countries, with the majority in china. the median number of participants per trial was , with only % of trials being phase . % of the trials were recruiting or ongoing, and . % were completed. we only found peer-reviewed original articles reporting results for the eligible trials. we also found preprints of trial results. pharmacologic interventions were the most-studied category, with antiparasitic drugs, alone or in combination, being the most studied subcategory. the majority of trials included morbidity outcomes as primary outcomes. our study was based on trials registered in the who international clinical trials registry platform (ictrp), which allows single-point access and unambiguous identification of trials [ ] . also, we used duplicate and independently approach to categorizing interventions and outcomes. a major limitation of our study is that the pool of covid- registered trials is rapidly growing, hence the data would need to be periodically updated. moreover, some registered trials may have incorrect, missing or outdated information [ ] . this shift in geographical location reflects the change of the geographical focus of the pandemic itself to europe and united states. the number of trials also remarkably increased in iran, which is now the third largest country of origin for registered trials ( . %). the previous focus of trials was on traditional chinese medicine, which is somehow expected given that the pandemic started in china. the current focus is on antiparasitic drugs, immunomodulators, and antivirals. during the covid- pandemic, breakthrough news from chinese hospitals regarding the effectiveness of hydroxychloroquine in covid- patients drew significant attention to the drug [ ] . it was especially made the subject of public attention after a cohort study in france found the use of hydroxychloroquine with azithromycin effective and free of side-effects in patients if used early after diagnosis [ ] . chloroquine and its reportedly less toxic derivative, hydroxychloroquine [ ] , used alone or in combination with azithromycin, are now part of clinical practice for the management of covid- in more than countries including china, iran, and italy [ ]. on march , , the us food and drug administration also approved hydroxychloroquine for emergency use authorization in treating covid- patients [ ] . the increasingly widespread use of the drug brings important considerations regarding evidence supporting its use. as of may , , the results of four clinical trials and one prospective observational study on the use of hydroxychloroquine in covid- were published. three found it superior to conventional treatment [ , , ] , while the others did not observe significant difference between groups [ , ] . one target trial emulation of patients did not find evidence supporting the use of hydroxychloroquine for covid- related hypoxic pneumonia [ ] . with no long-term follow-up, small sample sizes, multiple methodological flaws, and conflicting results, these published trials do not offer enough high-quality evidence to adequately support guideline recommendatios [ ] . we found registered trials investigating the use of hydroxychloroquine in covid- patients for either treatment or prophylaxis. the majority are either in phase (n= ), or phase (n= ), and most are conducted in europe (n= ). only three of these trials had published results [ , , ] . with the known side effect profile of the drug, including cardiomyopathy and arrhythmias [ ] and its suggested ability to induce renal and liver impairment [ ] , the inclusion of hydroxychloroquine in clinical practice remains questionable until strong and convincing evidence can be generated. other therapeutic agents under study included anti-virals, immunomodulators, and biological agents. these include many drugs previoulsy used for the treatment of other infectious pathogens. one example is umifenovir (brand name arbidol), an antiviral agent used in russia and china for treating influenza infection, but not approved by the u.s. food and drug administration (fda) [ ] . another example is oseltamivir, an fda-approved drug for the treatment of influenza a and b [ ] . remdesivir is an anti-viral agent that has recently received considerable attention. the adaptive covid- treatment trial (actt), a phase trial involving participants lead by the national institute of allergy and infectious diseases , found that patients treated with remdesivir had significantly faster recovery and lower mortality compared to placebo [ ] . in light of the optimistic results, remdesivir was announced as the new standard of care for covid- patients in the us on april , [ ] . available results by another, although smaller, clinical trial lead by gilead sciences did not a significant difference in outcomes between and days of treatment with remdesivir [ ] . ongoing trials investigating vaccines for covid- are very important, as vaccination is the sustainable solution for counteracting this public health threat [ ] . traditionally, vaccine development is a lengthy process faced by multiple challenges including unknown virus immunogenic profile, vaccine safety, and participants recruitment/adherence [ ] . in times of pandemics, additional challenges appear, such as difficulty randomizing populations in high mortality situations, overburdening ethics and regulatory authorities, as well as the absence of large-scale manufacturing for any novel platform technology [ ] . although promising results are available for one trial investigating the adenovirus type (ad ) vectored covid- vaccine [ ] , and most of the identified trials on covid- vaccines are in phase and/or , the much-awaited vaccine is not expected to be available before at least year to months [ ] . the selection of primary outcomes reflects how researchers define meaningful evidence for the success of an intervention. however, the selection of outcomes has to insure adequate validity of their measurments and their generalizability for translation in clinical practice or health policies [ ] . we found that pharmacologic and biological interventional trials addressed mainly morbidity and surrogate outcomes more frequently than composite, mortality or patient-reported outcomes. trials targeting psychological interventions or physical therapy measured patientrelated outcomes, an expected finding. we found that % of registered trials addressed surrogate outcomes exclusively. in general, it is unclear to what degree surrogate outcome correlates with clinically meaningful effects, like those targeted by clinical outcomes [ ] . however, surrogate outcomes are used in accelerated approval pathways during epidemics or increases in life-threatening diseases, as they allow measurement of intervention effects with smaller sample sizes and shorter trial durations [ ] . we identified no completed trials, and only peer-reviewed articles were available. the most likely cause is the recency of the events that triggered the trials of interest. while peer review can be a lengthy process, peer-review platforms are making changes to optimize their assessments, for example by directly posting their reviews to preprint servers [ ] . there is no available evidence to date that appropriately guides recommendations for the prevention and treatment of covid- . those guiding health policies and clinical practice may therefore have to rely on a limited number of trial results, or indirect evidence derived from other diseases. there is a need for evidence-based interventions to mitigate the global humanitarian and economic sequelae of the pandemic. registration of ongoing trials is essential for researchers to coordinate efforts, but more importantly to optimize the methods of those trials and ensure the transparency of their methods. the case of hydroxychloroquine illustrates very well how the medical community adopted a promising but still unproven intervention, in spite of the supporting evidence being inadequate, and that trials are still ongoing. this could be explained by the lack of effective medications in the face of high mortality rate. still, the medication comes with a significant side effects and harms. therefore, it is imperative to have a living process to keep the evidence uptodate as the results of trials start coming out [ ] , and then feeding them into living recommendations [ ] . tracking registered trials would improve the efficiency of those living processes. future research can focus on improving trial recruitment process, and generating results in the most expeditious way possible. in the face of ongoing challenges, collaborative international efforts may be the key to success. there is also need to explore how to make trial outcomes available to decision makers, including guideline developers in a timely fashion. future research on trial registration in this field will need to explore its dynamics over the time given the large number of trials being registered. for example, it would be interesting to explore whether and how the country where the trial is being conducted changes with the change of the countries most badly affected. similarly, it will be important to explore how the categories of interventions and outcomes change over time. none to declare. epidemiological and clinical characteristics of cases of novel coronavirus pneumonia in wuhan, china: a descriptive study a new emerging zoonotic virus of concern: the novel coronavirus (covid- ) early transmission dynamics in wuhan, china, of novel coronavirus-infected pneumonia <em>severe acute respiratory syndrome-related coronavirus</em>: the species and its viruses -a statement of the coronavirus study group journal of experimental stroke & translational medicine naming the coronavirus disease (covid- ) and the virus that causes it a bibliometric analysis of covid- research activity: a call for increased output coronavirus disease (covid- ) situation report - , in coronavirus disease (covid- ) situation reports. , world health organization: world health organization an interactive web-based dashboard to track covid- in real time clinical characteristics of coronavirus disease in china clinical course and outcomes of critically ill patients with sars-cov- pneumonia in wuhan, china: a single-centered, retrospective, observational study. the lancet respiratory medicine the response of milan's emergency medical system to the covid- outbreak in italy critical care utilization for the covid- outbreak in lombardy, italy: early experience and forecast during an emergency response how will country-based mitigation measures influence the course of the covid- epidemic? the lancet the origin, transmission and clinical therapies on coronavirus disease (covid- ) outbreak -an update on the status randomized controlled trials: part of a series on evaluation of scientific publications status of the pediatric clinical trials enterprise: an analysis of the us clinicaltrials.gov registry international clinical trials registry platform (ictrp) our world in data exploring qualitative methods reported in registered trials and their yields (equity): systematic review traditional chinese medicine in the treatment of patients infected with -new coronavirus (sars-cov- ): a review and perspective breakthrough: chloroquine phosphate has shown apparent efficacy in treatment of covid- associated pneumonia in clinical studies hydroxychloroquine and azithromycin as a treatment of covid- : results of an open-label non-randomized clinical trial hydroxychloroquine, a less toxic derivative of chloroquine, is effective in inhibiting sars-cov- infection in vitro coronavirus (covid- ) update: daily roundup efficacy of hydroxychloroquine in patients with covid- : results of a randomized clinical trial. medrxiv preliminary evidence from a multicenter prospective observational study of the safety and efficacy of chloroquine for the treatment of covid- a pilot study of hydroxychloroquine in treatment of patients with common coronavirus disease- (covid- ) hydroxychloroquine in patients mainly with mild to moderate covid- : an open-label, randomized no evidence of clinical efficacy of hydroxychloroquine in patients hospitalized for covid- infection with oxygen requirement: results of a study using routinely collected data to emulate a target trial should chloroquine and hydroxychloroquine be used to treat covid- ? a rapid review liver and kidney injuries in covid- and their effects on drug therapy; a letter to editor. archives of academic emergency medicine arbidol as a broad-spectrum antiviral: an update oseltamivir for influenza infection in children: risks and benefits remdesivir for the treatment of covid- -preliminary report emory helps lead research on drug to treat covid- patients remdesivir for or days in patients with severe covid- the effect of control strategies to reduce social mixing on outcomes of the covid- epidemic in wuhan, china: a modelling study the current challenges for vaccine development developing covid- vaccines at pandemic speed safety, tolerability, and immunogenicity of a recombinant adenovirus type- vectored covid- vaccine: a dose-escalation, open-label, non-randomised, first-in-human trial. the lancet how will country-based mitigation measures influence the course of the covid- epidemic? choosing primary endpoints for clinical trials of health care interventions time to review the role of surrogate end points in health policy: state of the art and the way forward living systematic review: . introduction-the why, what, when, and how living systematic reviews: . living guideline recommendations ( ) ( key findings:• , eligible trials related to covid- were registered up to may th , • trials were planned in countries, with the majority in europe• while more than half of the trials were already recruiting, eight had peer-reviewed publications what this adds to what is known • the research community has shown a good response to the pandemic in terms of initiating trials. • the ultimate test will be whether the research community will be able to generate the needed evidence to guide the management of the pandemic.• efforts should focus on completing the trials and publishing them in a timely fashion. none to declare. key: cord- -i ynjp authors: milner, ross; donington, jessica; matthews, jeffrey; posner, mitchell; turaga, kiran; angelos, peter title: is it ethically appropriate to continue surgical clinical trials during the covid- pandemic? edited by dr sarr date: - - journal: surgery doi: . /j.surg. . . sha: doc_id: cord_uid: i ynjp covid- has greatly impacted surgical care and decision-making. the status of surgical clinical trials during this pandemic has not been addressed. we provide a framework and recommendations for the management of patients involved in surgical clinical trials. the covid- pandemic has greatly impacted both clinical care and its underlying ethical basis, shifting from the traditional patient-centered approach to a public health strategy. as surgeons, we have been challenged to balance the risk of proceeding with planned operations ith the cncerns to the patient and to the health care system against the risk of delay. for the first time in the united states, we have found it necessary to consider the availability of appropriate hospital resources, the potential for increased adverse outcomes in occult covid- -positive patients, and limitations in pre-operative testing, as well as potential health dangers posed not onoy to the patients but also to hospital staff including the entire operating room staff-nurses, anesthesiologists, physicians, etc. at the university of chicago, we introduced a scoring system to assist in the ethical triage of medically-necessary, timesensitive (ments) procedures given that the imperative to halt all "elective" surgery does not adequately capture the nature or define the types of non-emergency procedures that may still need to proceed. we have found this approach to be particularly useful with respect to the constantly changing pressures on peri-operative resources (anesthesiologists, nursing, personal protective equipment, blood, ventilators) and hospital inpatient capacity otherwise diverted to dedicated covid units and icus during different phases of the pandemic. the timing of surgery for cancer patients has been particularly thorny, and a number of groups have proposed thoughtful frameworks for these clinical decisions. , , so far unaddressed during this pandemic has been the status of surgical clinical trials. specifically, is it ethically appropriate to continue enrollment and treatment of patients in clinical trials that include surgical intervention? if we continue such clinical trials, are we putting patients at greater risk of covid -related complications? could the pandemic inappropriately skew the results? if we expedite and prioritize surgical treatment for those patients who are part of a trial, are we unduly pressuring them to participate (a form of coercion)? in contrast,, is it ethical to suspend future enrollment or active participation if it would potentially deny patients the benefit of novel therapeutic interventions or improved outcomes associated with the clinical trial? although the ethical debates might apply across a broad range of clinical studies during the pandemic, we sought to focus specifically on surgical trials. we discuss here the ethics of clinical trial care within the surgical specialties and the the pros and cons of participation in clinical trial during the covid- pandemic, with a specific focus on surgical oncology and vascular surgery. the clinical trial portfolio is the core of any comprehensive cancer center. oncologic therapeutic clinical trials offer patients access to exciting new treatments. these trials are designed typically to answer specific questions regarding treatment and outcomes and not the timing of surgical procedures or the frequency of visits and invasive procedures (at least not as primary objectives). the current need for social distancing and limitations of health care resources has shifted priorities appropriately, but completely halting clinical trials would hinder dramatically the delopment of novel treatment sand leave patients currently enrolled in these trials without access to potentially life-saving medications. the cancer therapy evaluation program (ctep) at the national cancer institute (nci) recognizes these issues and encourage sponsors, investigators, and institutional review boards to revise existing policies and procedures to mitigate risk and to protect trial participants while keeping clinical trials open. this program includes steps to alter the informed consent process, study visits and procedures, data collection, and the reporting of adverse eventz. for example, local treating physicians can perform a majority of study-related activities and administer all medications except investigational agents. the issues related to surgery in a clinical trial during the covid pandemic are more challenging. many of the proedyres ypically cannot be performed by the surgeon at the local hospital, have strict time constraints dictated by inclusion/exclusion criteria in the trial protocol, and carry the potential to place an undue strain on limited inpatient resources and the health care workforce. futhermore, cancer patients undergoing complex surgical procedures are inherently immunosuppressed and are at risk for superinfection by the covid- virusn and the associated increase in morbidity and mortality. therapeutic clinical trials that involve oprative intervention tend to focus on locally advanced tumors and often involve induction therapies (chemotherapy, radiation therapy or both) that increase the potential for post-induction fibrosis, which may increase the technical difficulty of an operative procedure and prolong the postoperative hospital stay due to peri-operative complications. in the end, the decision to proceed with a planned cancer resection for a patient enrolled in a clinical trial is highly dependent on the institution's phase in the pandemic and availability of peri-operative resources. the decision to proceed with an operationthat is governed by stict time-dependent guidelines is often somewhat arbitrary and that time interval can likely be prolonged without adversely affecting the there are cities and hospitals where only urgent surgery is possible and others where resources still allow for high priority, oncologic resections. at the university of chicago, we have attempted to maintain patients already enrolled in clinical trials, butwehave curtailed new enrollment dramatically due to the uncertainty of available resources and have focused our research efforts on covid-centric issues. only low risk trials with novel agents and unique therapies remain open to enrollment, but, as expected, accrual has slowed tremendously. trials that include surgery do not fall into this category. surgical trials typically require multiple different therapies on rigid time schedules and are not deemed prudent in our current environment. it is unclear how these decisions will impact cancer care for individual patients and the advancement of our science, but in a time of limited resources, we do not want to embark on a treatment strategy that we cannot potentially safely complete. but, it is not lost on us that clinical cancer trials that do or do not involve surgery are invaluable in defining new treatment paradigms that lead to improved outcomes for cancer patients, and it is known that just by participating in a clinical trial, derivative benefits in patient outcomes are the rule. therefore, striking the correct balance between these two conflicting concepts is the essence of the ethical dilemma we are facing. the majority of clinical trials in vascular surgery involve the use of medical devices or a surgical procedure to treat peripheral artery disease, carotid artery disease, or aortic aneurysms/dissections. at the university of chicago, we participate in a number of such trials. many of these large, multiinstitutional device trials are sponsored by industry. in light of the covid- pandemic, all vascular clinical trials have been halted. one of the large device trials that was scheduled to start in march has delayed its start during the covid- pandemic due to the need to eliminate "elective" cases for the well-being of individual health care systems. the decision on the part of industry to suspend such clinical trials has been met largely with support from vascular surgeons. what is the ethical basis for supporting this decision even though it means that some patients are not getting the latest and potentially more effective devices to treat their vascular disease? the care and focus of linical trials in vascular surgery are both resource-intensive and time-sensitive. using our current framework of ments procedure prioritization, we have focused attention on patient care issues related to critical limb ischemia, symptomatic carotid artery disease, and symptomatic or ruptured aortic aneurysms. as such, vascular surgeons should not utilize scarce operating room time or clinic time to investigate novel devices with unknown outcomes. it is our belief that this same prohibition on vascular clinical trials should also apply to fda-approved devices that are being investigated under a registry designation. by halting the nenrollment of patients into such trials, we relieve the pressure placed on surgeons to enroll patients in a clinical trial that could negatively impact the timely care of other patients whose needed procedures have greater medical urgency. most importantly, we believe that in the present environment, surgeons should provide care that is the best known "standard of care" and that can provide the most benefit to our patients and minimize the impact on our hospital and health care systems with unknown outcomes from devices. finally, patients should not feel pressured to participate in clinical trials in the hope that by enrolling they may be given an advantage to have surgery sooner. the covid- pandemic has created both clinical and ethical dilemmas for surgeons. we believe that continuing surgical clinical trials at the present time poses unique ethical concerns. before continuing to enroll patients in surgical trials, we believe that surgeons must carefully consider the type of trial, the institutional status with respect to scarce resources, and the potential risk/benefit ratio to patients and health care workers involved. we have decreasedour clinical trial efforts mzarkedly during the pandemic to minimize patient coercion and to maximize the use and avialablity of patient care resources for evidence-based procedures. . surgeons, ethics, and covid- : early lessons learned medically-necessary, time-sensitive procedures: a scoring system to ethically and efficiently manage resource scarcity and provider risk during the covid- pandemic management of cancer surgery cases during the covid- pandemic: considerations a multidisciplinary team approach for triage of elective cancer surgery at the massachusetts general hospital during the novel coronavirus covid- outbreak are we harming cancer patients by delaying their cancer surgery during the covid- pandemic? preserving clinical trial integrity during the coronavirus pandemic safety considerations in the laboratory testing of specimens suspected or known to contain the severe acute respiratory syndrome coronavirus (sars-cov- ). lab medicine cancer patients in sars-cov- infection: a nationwide analysis in china covid- has greatly impacted surgical care and decision-making. the status of surgical clinical trials during this pandemic has not been addressed. we provide a framework and recommendations for the management of patients involved in surgical clinical trials key: cord- -l nlrv h authors: chauvenet, alienor; buckley, ralf; hague, leah; fleming, chris; brough, paula title: panel sampling in health research date: - - journal: lancet psychiatry doi: . /s - ( ) - sha: doc_id: cord_uid: l nlrv h nan carsten hjorthøj and colleagues question the extent to which the effects of cannabidiol as a pharmacological treatment for cannabis use disorder might be clinically meaningful. as they pointed out, and as discussed in our article, the phase a trial was not designed to estimate the magnitude of efficacy. however, phase a trials can be valuable when testing a novel indication with no previous evidence on what doses might be efficacious or safe. we found that cannabidiol mg and cannabidiol mg were more efficacious than placebo according to both primary endpoints (reduced urinary -nor- -carboxy-δ- tetrahydrocannabinol:creatinine ratio and increased days with abstinence from cannabis during treatment) based on a priori bayesian criteria. we did not make inferences about clinical relevance in our article and it would be premature to do so because our phase a trial was not intended to address this question. larger phase b or phase trials are needed to determine how efficacious and clinically meaningful the effects of cannabidiol are at the doses we identified in our trial. we used a -week treatment design, similar to the first randomised clinical trial of cannabidiol for the treatment of psychosis. more research is needed to test different dosing durations and formulations. three randomised clinical trials have investigated nabiximols (low dose cannabidiol and tetrahydrocannabinol). [ ] [ ] [ ] only one trial found a reduction in cannabis use compared with placebo, and none of the trials reported increases in sustained abstinence compared with placebo. hjorthøj and colleagues believe that a change in paradigm is needed in the treatment of cannabis use disorder, away from a focus on reduction in use and towards complete abstinence. their views contrast with expert consensus on clinical outcomes for cannabis use disorder trials, published in : the primary recommendation is that sustained abstinence from cannabis should not be considered the primary outcome for all cannabis use disorder clinical trials because it has multiple limitations. furthermore, given the absence of any recommended pharmacotherapies at present, a treatment that consistently reduces cannabis use would represent a major achievement towards decreasing the global burden of cannabis use disorders. we declare no competing interests. the lancet psychiatry, matthias pierce and colleagues , identify the importance of sampling in studying mental health effects of covid- . we found that a mental health survey using a com mercial panel (of approximately people) overrepresented mentally unhealthy respondents by approximately · times. this over representation occurred despite multiple measures to ensure representativeness: pre specified demographic and geographical sampling quotas; post-collection checks on the distribution of socioeconomic parameters; and adjustments for mismatches between clinical psychological scores and use of health-care services. further random subsampling, before ana lysis, was required to correct for this sampling bias. it seems that self-selected commercial survey panels in general might be biased towards mentally unhealthy or unhappy individuals. commercial survey organisations operate through networks of subcontractors who hold customer contact lists. individuals self-select to take part, for a small financial incentive. this might create bias towards people who are in difficult financial circumstances, and hence are under mental stress. the turnover in these self-selected panels is high. it is now easy to target precise population segments using social media, but difficult to obtain random representative population samples. political and personality representativeness have been tested. surveys measuring mental health specifically can correct for bias during analysis. however, commercial surveys are also widely adopted in physical and social health research, and these might risk invalid results if they omit mental health measures. we declare no competing interests. school of environment and science (ac, rb), school of applied psychology (lh, pb), and school of business (cf), griffith university, gold coast campus, qld , australia characteristics of participants who successfully adhere to the treatment, those who start medication again without relapse, and those who have a severe relapse and irreversible consequences such as treatment resistance and functional decline. third, observational data such as nationwide population-based registers could be used to emulate a hypothetical target trial if randomisation is not feasible. a target protocol describes the ideal, but unachievable randomised clinical trial. this trial can be emulated by exploiting the natural variation in observational data, which would allow causal inference by adjustment for confounders and selection bias. the concept has been increasingly applied in pharmaco-epidemiology and provides reliable answers in the comparative effectiveness of research. fourth, n-of- trials should be used to develop personalised decision making. these recommendations are proposed to avoid pitfalls of the current approach to precision medicine. a common pitfall is to split variance around an estimate, in the so-called responders and non-responders, using arbitrary definitions on a continuous outcome. using these arbitrary categories as true, and looking for prognostic factors predicting the response, is a simplistic and often misleading way to develop personalised risk models because all control conditions are completely ignored. by using the n-of- design, the same individual is acting as their own control by comparing periods when on medication with periods when not on medication. in conclusion, we know from cohort studies that a substantial proportion of individuals can manage without antipsychotic medication, and will not relapse. therefore understandably, many try to stop medication at some point to find out if they belong to this group. the duty of clinicians is to provide knowledge about the risks the pressure to find answers has been felt worldwide and three large randomised clinical trials (eudractct - - , e u d r a c t c t - - , actrn ) have been initiated by the authors of this correspondence. however, none of these trials are progressing as expected. the first problem is insufficient recruitment. despite great interest in the discontinuation of antipsychotic medication, few individuals can equally accept either treatment group in a randomised discontinuation trial, because the decision to maintain or discontinue is too important to be left to randomisation. low recruitment leads to small sample sizes with a high risk of type errors and excludes the possibility of developing personalised risk profiles. the second problem is poor adherence to the treatment arm. despite agreeing to participate, participants' strong personal preferences lead to high rates of crossover between the treatment groups. poor adherence to the allocated treatment arm leads to data with less clinical use because describing differences in outcomes between similar treatment arms has no real value to the patient. in fact, weak adherence to treatment might create data that are approaching observational, where confounding is a major limitation for causal inference. we suggest four recommendations using alternative designs for future research that could shed light on the questions about maintenance treatment with antipsychotic medication. first, to reach a sufficient number of participants in randomised clinical trials, international consortia should be established to enable recruitment within a reasonable timeframe. second, clinical cohort studies including individuals who discontinue antipsychotic medication should be done to generate precise knowledge about the proportion and there is a pressing need for knowledge about the effects of discontinuing antipsychotic medication in patients with remitted psychosis. patients usually ask how long they will have to continue, and many stop taking medication, hoping that they can manage without it. as health-care professionals, we are responsible for providing evidence-based coun selling for the initiation and discontinuation of medication, to help patients make informed choices. however, the two randomised trials , that have compared a maintenance strategy with an early dose reduction strategy after remission in patients with first-episode psychosis, have reported contradictory results. the dutch mesifos study found that more patients achieved long term functional remission in the group who were assigned to early discontinuation of antipsychotic medication after months of remission, compared with those who were assigned to maintenance treatment. however, a recent study from hong kong did not replicate this finding in a larger sample. says who? the significance of sampling in mental health surveys during covid- mental health before and during the covid- pandemic: a longitudinal probability sample survey of the uk population economic value of protected areas via visitor mental health recruiting large online samples in the united states and india: facebook, mechanical turk, and qualtrics personality biases in different types of internet samples can influence research outcomes key: cord- - ggxzxu authors: husebo, bettina sandgathe; allore, heather; achterberg, wilco; angeles, renira corinne; ballard, clive; bruvik, frøydis kristine; fæø, stein erik; gedde, marie hidle; hillestad, eirin; jacobsen, frode fadnes; kirkevold, Øyvind; kjerstad, egil; kjome, reidun lisbeth skeide; mannseth, janne; naik, mala; nouchi, rui; puaschitz, nathalie; samdal, rune; tranvåg, oscar; tzoulis, charalampos; vahia, ipsit vihang; vislapuu, maarja; berge, line iden title: live@home.path—innovating the clinical pathway for home-dwelling people with dementia and their caregivers: study protocol for a mixed-method, stepped-wedge, randomized controlled trial date: - - journal: trials doi: . /s - - -y sha: doc_id: cord_uid: ggxzxu background: the global health challenge of dementia is exceptional in size, cost and impact. it is the only top ten cause of death that cannot be prevented, cured or substantially slowed, leaving disease management, caregiver support and service innovation as the main targets for reduction of disease burden. institutionalization of persons with dementia is common in western countries, despite patients preferring to live longer at home, supported by caregivers. such complex health challenges warrant multicomponent interventions thoroughly implemented in daily clinical practice. this article describes the rationale, development, feasibility testing and implementation process of the live@home.path trial. methods: the live@home.path trial is a -year, multicenter, mixed-method, stepped-wedge randomized controlled trial, aiming to include dyads of home-dwelling people with dementia and their caregivers, recruited from municipalities in norway. the stepped-wedge randomization implies that all dyads receive the intervention, but the timing is determined by randomization. the control group constitutes the dyads waiting for the intervention. the multicomponent intervention was developed in collaboration with user-representatives, researchers and stakeholders to meet the requirements from the national dementia plan . during the -month intervention period, the participants will be allocated to a municipal coordinator, the core feature of the intervention, responsible for regular contact with the dyads to facilitate l: learning, i: innovation, v: volunteering and e: empowerment (live). the primary outcome is resource utilization. this is measured by the resource utilization in dementia (rud) instrument and the relative stress scale (rss), reflecting that resource utilization is more than the actual time required for caring but also how burdensome the task is experienced by the caregiver. discussion: we expect the implementation of live to lead to a pathway for dementia treatment and care which is cost-effective, compared to treatment as usual, and will support high-quality independent living, at home. trial registration: clinicaltrials.gov: nct . registered on march . the world's population is rapidly aging as a result of fewer births and declining mortality rates [ ] . the global health challenge of dementia is exceptional in size, cost and impact [ ] . according to the world health organization, the number of people living with dementia is estimated to be million worldwide, expected to almost triple by [ ] . despite most people, also from a caregiver perspective, preferring to live longer at home, and to die there, if possible [ , ] , about , of the estimated , - , persons with dementia (pwds) in norway reside in nursing homes [ ] . the urbanization of our societies, in particular younger persons moving toward central areas and leaving their older relatives behind, underlines the need for cost-effective service collaboration to provide adequate treatment and care for the aging home-dwelling population. among the top ten causes of death globally, dementia is the only one that cannot be prevented, cured or substantially slowed [ ] , leaving disease management, caregiver support and service innovation as the top priority for health policy-makers in the reduction of disease burden. due to expected positive interactions within the family, interventions supporting them as caregivers not only potentially lessen the caregivers' burden [ ] , but could also be beneficial for the pwd (e.g. reducing neuropsychiatric symptoms and delaying nursing home admission) [ , ] . as such, interventions supporting caregivers hold the potential for better overall resource allocation and utilization [ ] . caring for a pwd comes at a high cost, both individually and at societal level. caregivers to pwds have lower perceived health and higher rates of mortality relative to their noncaregiver counterparts [ ] . the effect of practical assistance and psychoeducational programs have been evaluated, but most single initiatives have fallen short in reducing the caregivers' burden [ ] . the maximizing independence (mind) at home study undertaken in baltimore, usa, during - included approximately home-dwelling persons with cognitive impairment or dementia in a parallel randomized multicomponent trial [ , ] . this study showed that months of care coordination through individualized care planning, implementation of a care plan, monitoring and reassessment had beneficial effects on the time to transition from home, number of dementiarelated unmet needs, quality of life (qol) and, importantly, a potentially clinically relevant reduction in selfreported number of hours spent on caregiving tasks, as a measure of caregiver burden [ , ] . developing this model further, the mind at home-plus study included an additional persons to evaluate the effect on longterm care placement, hospitalization and health-care expenditures of a -month homecare coordination program for pwd [ ] . the mind at home-streamlined trial is now refining the intervention to investigate its impact on time to long-term care placement, needs, burdens and qol in pwds and their caregivers, as well as cost utilization [ ] . results of the latter study are highly anticipated due to the potential for effective system-level approaches to dementia care [ ] . yet, due to fairly large regional and cultural differences in care organization, there is a need for implementation studies in other countries to explore the generalizability of the program. a multicomponent intervention is not merely a discrete package of separate components, but a process of changing what complex systems do [ ] . intervening within a complex system involves disrupting prior ways of working while introducing new ones [ ] . the degree of complexity can be understood as a relative construct, defined by the number of components, diversity of the intended outcome, number of targeted organizational levels and level of skill required to deliver the intervention [ ] , while additionally considering the interplay between context, setting and the implementation process [ ] . in the cosmos trial, a randomized implementation hybrid trial carried out in norwegian nursing homes during - , our group successfully developed, implemented and effect evaluated a multicomponent intervention addressing communication, systematic assessment and treatment of pain, medication review, organization of activities and safety [ ] . overall, the intervention resulted in improved qol and activities of daily living (adl), in addition to a decrease in neuropsychiatric symptoms such as agitation and depression as well as a reduction in the number of medications used among nursing home residents [ ] [ ] [ ] [ ] [ ] . to provide cost-effective care while securing the needs of pwds and caregivers represents a complex health challenge warranting multicomponent interventions implemented in daily clinical practice. aiming at systemlevel change, such interventions require stakeholder involvement as well as collaboration within and between different levels of primary and specialist health-care services, nongovernmental institutions, users and researchers, addressing the need for appropriate and coordinated cross-sector action. the live@home.path trial aims to develop, adapt, implement and effect-evaluate a multicomponent intervention for home-dwelling dyads of pwds and their caregivers, aiding them to stay safer, longer and more independently at home with cost-effectiveness. in this study, caregivers are defined as family or close friends, equaling informal caregivers. live@home.path is an acronym referring to each component of the complex intervention: learning, innovation, volunteer support and empowerment-at home pathway. the primary outcome is resource utilization. this is measured by the resource utilization in dementia (rud) instrument and the relative stress scale (rss), reflecting that resource utilization is more than the actual time required for caregiving tasks, but also how burdensome the task is experienced by the caregiver. importantly, the caregiver burden is individual, and may be related to economic hardship, anxiety, depression, hopelessness, impaired qol or lack of sleep and time for recreation. this individual perspective underlines the significance of user involvement, reflected in the trial's slogan: what matters to you? secondary outcomes include neuropsychiatric symptoms, number of adverse events, use of assistive technology, involvement of volunteers, qol and clinical global impression of change for the pwd as well as caregivers' depression, qol and work performance. the live intervention will reduce time and resources that caregivers spend in organizing and supporting pwds' daily activities, thereby reducing the caregiver burden. the live@home.path trial is a -year, multicenter, mixed-method, stepped-wedge randomized controlled trial (rct). we aim to recruit dyads of home-dwelling pwds and their caregivers from the municipalities of bergen, baerum and kristiansand. based on experiences with two pre-projects-research council of norway sponsor code (uib) and (haraldsplass deaconess hospital)-the intervention was developed in collaboration with userrepresentatives, stakeholders and scientific partners from the scientific advisory board. to meet the requirements from the dementia plan by the ministry of health and care services [ ] , we identified the "big issues" expected to facilitate support for home-dwelling pwds and their caregivers. as such, we combined and adapted existing knowledge rather than designing new components, contributing to service innovation in the health-care systems. the process was tailored to meet the standards of "development-evaluation-implementation", an internationally agreed approach for complex interventions launched by the uk medical research council [ ] . at the start of the -month intervention period, the dyads will be allocated to a municipal coordinator, offering regular contact to assist in finding a pathway throughout the administrative trajectory of dementia care. the coordinator should hold a bachelor degree in health-related science (e.g. nursing, ergo or physiotherapy), and will make a minimum of two home visits, one immediately after the intervention start and the second after approximately months. supplementary visits will be offered if needed, in addition to monthly telephone calls. during the intervention, the coordinator will introduce the dyads to the different stages of the live intervention: learning, innovation, volunteer support and empowerment (table , fig. ). all components will be carefully adapted to local conditions. learning a fruitful learning process is characterized by relevance, timing, confidentiality and reflection as well as fulfilment of expectations regarding content. the dementia plan [ ] underlines increased knowledge at all societal levels as crucial for improvements in dementia care. a meta-analysis on the effectiveness of educational interventions supporting caregivers of communitydwelling pwds found a moderate impact on the caregiver burden, a small effect on depression, but no effect on transition to long-term care [ ] . a norwegian multicenter randomized controlled trial found no reduction in depressive symptoms for pwds and caregivers after a -month psychosocial support program including formal education seminars [ ] . yet coping had a positive impact on the caregiver burden in the latter study, possibly reflecting improved understanding of the caregiver situation [ ] . in practice in the live@home.path: the coordinator will encourage and facilitate that both the pwd and the caregiver participate in local educational programs arranged by the municipality or the specialist health services several times yearly. as an example, the nationally established educational program for relatives of pwds is developed by the norwegian advisory unit on ageing and health [ ] , and implemented for use in bergen, baerum and kristiansand. innovation innovation is understood as the application of better and more original solutions to meet new requirements, unarticulated needs or existing market needs, or employing established solutions in new areas, both technological, such as information and communication technology (ict), and organizational. crucially, the process will result in more effective products, processes, services, technologies or business models being made available for all, including markets, government and society [ ] . as such, the live@home.path can be viewed as a service innovation, aiming at the development of a clinical pathway for dementia care. ict approaches in elderly care are broadly categorized as technical aids, cognitive intervention devices, and sensor and assistive living systems [ ] . ict in dementia care holds potential for optimizing safety at home, reducing caregiver burden and, although the findings are not conclusive [ ] , possibly also improving cost-effectiveness. yet we have limited knowledge about which type of devices are used, regarded as useful and requested by caregivers and pwds at different stages of dementia [ ] . most important, this field requires a careful, individual risk-benefit assessment, as ict might negatively impact autonomy and privacy, and provide a false sense of safety. in practice in the live@home.path: the coordinator will assess and evaluate the usefulness of ict solutions already in use for pwds and caregivers and inform about additional relevant welfare technology available in the municipality. the participants will receive information about a newly launched online communication platform tailored to meet the needs of families organizing dementia care (jodacare©) [ ] , and be informed about a web page with scheduled activities of relevance (fris-kus©) [ ] . in bergen, the participants will be invited to test the prototype alight©, an application for tablets providing a "digital memory book" developed by soundio as and nks olaviken gerontopsychiatric hospital [ ] . additionally, up to ten participants in bergen will be invited to test a prototype of the adapted communication platform in collaboration with the western norway university of applied sciences. underlining the aspects of service innovation, all data will be collected on tablets owned by the project group via the software surveyjs [ ] . the live@home.path trial was selected as a pilot for the development and evaluation of this software, providing secure data transfer and storage on the safe server at the university of bergen for research project with sensitive data. after approval from the principal investigator, researchers affiliated with the project will be given access to the server, avoiding export of data and maintaining high levels of security [ ] . volunteer support volunteer support is understood as any activities that involves someone spending time, unpaid and of one's own will, doing something that aims to benefit someone else outside their own families and households [ ] . being important suppliers of unpaid support, it is estimated that volunteers contributed , full-time equivalents (ftes) in norway in [ ] . however, the majority are engaged in sports and culture, and representation in the elderly care sector is sparse [ ] . volunteering among older adults reduces their depressive symptoms, improves self-reported health and functional performance, and increases survival [ , ] . the volunteers additionally report better health through their own engagement [ , ] . volunteerism has contributed to the development of the norwegian welfare system through identifying and providing solutions to societal challenges [ ] , being formally integrated into core strategic plans in the health-care sector and being launched as a prioritized political strategy in elderly and dementia care in norway [ ] . yet we have sparse knowledge about volunteer support schemes for homedwelling pwds. to provide better services, understanding of the dynamics, motivations and interactions in volunteerism in dementia care is required. in practice in the live@home.path: the coordinator will investigate pwd and caregiver attitudes toward volunteer support, and inform about volunteer services. if this is of interest, the coordinator will contact local volunteer coordinators for nonprofit organizations (the red cross [ ] and the norwegian association for public health [ ] ), aiming at the best possible match of volunteers based on assessment of preferences and wishes. empowerment empowerment in dementia care can be defined as "a confidence building process whereby pwd are respected, have a voice and are heard, are involved in making decisions about their lives and have the opportunity to create change through access to appropriate resources" [ ] . the process of advanced care planning (acp) can increase empowerment for pwds and their caregivers [ , ] , underlined by the norwegian policy guidance by the directorate of health on diagnosis, treatment and care for pwds [ ] . pwds do not necessarily die from dementia, they die with it, and the life expectancy after onset of symptoms ranges from to years, depending on age and the presence of comorbidities [ ] . the continuing process of communication should be initiated as early as possible in collaboration with the general practitioner as a comprehensive medical examination including revision of medications, enabling the pwd to clarify individual values and wishes for domestic and institutionalized treatment and care (i.e. "what matters to you?"). in practice in the live@home.path: the coordinator will schedule a minimum of one appointment at the general practitioner's office for empowering acp, including the issues of formal next of kin and guardianship. in addition, a systematic medication review will be undertaken to ensure use of medications in line with diagnoses and symptoms, utilizing recommended guidelines [ ] . to evaluate the feasibility and the implementation strategy of the coordinators of the live intervention, a feasibility study was conducted during - . sixteen dyads in bergen were assigned a coordinator for months, participating in a minimum of two home visits and providing monthly follow up by telephone. one dyad dropped out after a few weeks of participation due to permanent placement in a nursing home, leaving dyads followed by coordinators for assessment. qualitative individual and focus group interviews utilizing a hermeneutic approach were performed with six dyads, three caregivers and the two coordinators as well as the coordinators' leader, exploring the usefulness of the coordinator function. this process revealed that the core feature of the coordinator was to support the caregivers in finding, applying and organizing support, and to provide emotional care, support and guidance. the objective of empowering the pwd in the decision-making processes was nonetheless particularly difficult to achieve. this finding was further incorporated into the live intervention for the stepped-wedge rct, with increased focus on the acp process and follow up of the gp [ ] . implementation research is defined as the scientific investigation concerning the act of carrying an intervention into effect in the real-world setting [ , ] . even a superbly designed intervention will fail to change practice if the process of implementation is futile. in the live@home.path trial, the implementation can be viewed as a two-stage process: first, from the research team to the coordinators; and, second, from the coordinators to the dyads. the first part encompasses all activities arranged by the research team empowering the coordinators to standardize the implementation of the intervention, such as seminars, development of written material and follow-up of coordinators during the intervention period. six months prior to the intervention start, kick-off workshops for all involved collaborators in the municipalities, including coordinators and affiliated specialized health services, will be arranged at all study sites, facilitating enthusiasm, collaboration and recruitment of participants. two weeks before the intervention start, a -day implementation seminar for the coordinators will be delivered by the research team at all study sites, training the coordinators through lectures, roleplay and discussions (see additional file ). halfway through the -month intervention period, a -day midway evaluation workshop for the coordinators will be arranged, allowing for discussion of obstacles and pitfalls, which acts as a source for facilitating a more effective and standardized implementation. as a part of the intervention, the research team will contact each coordinator by telephone every days to keep track of the process, discuss potential challenges and follow-up use of the checklist for implementation of the intervention. this ten-page pocket manual will contain a simplified howto-do description of the intervention components. it will be filled out for each dyad by the coordinator, registering time use and whether each of the distinct live components has been addressed during the intervention period. additionally, a -page tutorial will be developed as a comprehensible introduction to the rationale, method and practical aspects of the conduction of the trial, aimed for an audience not skilled in the research method. the second part of the implementation process encompasses the coordinator-dyad relationship. the coordinators are obliged to arrange a minimum of two home visits during the intervention period, and provide monthly contact by telephone. the checklist for implementation of the intervention will be used at every contact, and collected by the research team at the end of the intervention, providing documentation for the implementation process. in addition to the midway evaluation, a live conference will be organized for all coordinators at the end of the third intervention period, collecting data on their experiences of the suitability of the single components and the implementation process. additionally, at data collection after the intervention period, the participants will be asked if and to what extent they were offered the live components, and how often they were contacted by their coordinator. as such, if the live intervention fails to prove an effect on resource utilization, it will be possible to examine whether this is a result of the live components not being tailored to produce such an effect (i.e. that our main hypothesis was wrong) or whether it was caused by a lack of proper implementation. evaluation of the implementation process will further be investigated by conducting qualitative interviews with the coordinators as part of the mixed-method design. the required sample size was calculated to detect a difference of h/week for the primary outcome rud. based on the literature, we assumed that the mean number of hours of informal care is h/week with a standard deviation (sd) of h/week [ ] . with % power and a significance level of %, the required sample size was estimated to be dyads. to allow for % loss to follow-up, a total of dyads, equaling per municipality, must be included. participants will be recruited from memory clinics at local hospitals, from municipal memory teams and after advertisements in general media such as newspapers, radio and tv in bergen, baerum and kristiansand. bergen is the second largest municipality of norway with approximately , inhabitants in , baerum is ranked the fifth largest with , inhabitants, while the , inhabitants of kristiansand constitute norway's sixth largest municipality [ ] . pwds are eligible for inclusion if they: are aged ≥ years; are home-dwelling; have a minimum h/week regular face-to-face contact with the caregiver; are diagnosed with dementia according to standardized protocol [ ] ; have mini-mental state examination (mmse) score of - ; have a functional assessment staging test (fast) score of - ; and provide written informed consent. exclusion criteria are: participation in another ongoing intervention trial; or expected survival < weeks. pwds are eligible for inclusion regardless of etiology of the dementia and presence of other disorders. caregivers are eligible for inclusion if they have a minimum of h/week regular face-to-face contact with the pmd and provide written informed consent. as such, both the pwd and the caregiver will be included in the trial, representing a dyad. the mixed-method, stepped-wedge randomized control design data from all dyads will be assessed every months from baseline to the end of study period after months, death or permanent residency in a nursing home-in total, five waves of data collection. the stepped-wedge randomized control design [ ] implies that all participants will receive the -month intervention program during the study period, for which the timing of the intervention is determined by the randomization (fig. ) . the control group constitutes the dyads waiting for the intervention at a given time during the study; this group will have access to health care and receive treatment as usual. criteria for discontinuing the intervention or participation are requested from participants to withdraw from the trial. the trial's user-oriented approach, aiming at minimizing the participant burden associated with follow-up visits, in addition to flexibility in scheduling of the visits are sought to promote retention and prevent loss to follow-up over the trial. no distinct adverse events are expected before the start of the trial or during the trial, while possible adverse events related to the change in prescribed medication during the general practitioner's medication review might occur. if so, they will be reported by the coordinators to the researchers, either immediately or at their regular follow-up every weeks (physical meeting, by phone or by e-mail), in addition to feedback from the coordinator to the general practitioner. a statistician will randomly allocate the order of the intervention using block randomization; the dyads are randomized in clusters within each geographical location. the random sequence will be generated using a computerized random number generator undertaken for all three municipalities after the inclusion and baseline assessments are completed for all participants. research assistants, researchers conducting the analyses and other study personal conducting data collection will be blind to the randomization order and to the implementation process of the intervention. participants will not be informed of the intervention and implementation strategy to secure blinding until they are allocated to their coordinator during the intervention period. from this point of time, they become unblinded. given the practice change of the intervention, the municipality homecare services will be aware when their cluster enters the intervention period. when developing a pathway for dementia care, incorporating experiences and perspectives from the pwds and their caregivers is fundamental. in line with the involve framework [ ], this trial is developed through user involvement from the conception of the idea, via design through the implementation phase. at the structural level, user involvement is secured via collaboration with the head of research at the norwegian health associations [ ] , participating in the steering committee, and locally grounded by dementia coordinators in the municipalities. at the individual level, the centre for elderly and nursing home studies (sefas), responsible for conducting the trial, employs a user-representative as a co-researcher in a % position, who participates in the study's advisory board and working group. the mixed-method design [ ] encompasses the integration of data from quantitative assessment of validated outcomes with material from qualitative interviews and participant observation. utilizing an exploratory hermeneutic design [ ] , in-depth and focus group interviews with pwds (n = ), caregivers (n = ), municipality health-care staff (n = ), general practitioners (n = ), volunteers (n = ) and volunteer coordinators (n = ) will be conducted. to evaluate the acceptability and feasibility of the communication platform, interviews with caregivers and care staff will be made, as well as real-life observations form use among pwds and caregivers. table presents the primary and secondary outcomes according to domain, specific measurement, metric, method of aggregation and time points. the primary outcome of the live@home.path trial is formal and informal resource utilization, measured by the rud instrument [ , ] and the rss [ ] ( table ). as such, we consider overall resource utilization as more than the time required to care for the pwd; it also encompasses how burdensome the task is experienced by the caregiver. the informal care time use is measured in hours/ month [ , ] , in addition to numbers of contacts with the health-care system and use of medications. the rud is a standardized and widely used instrument assessing dementia care, proven useful across different care systems and countries and in both clinical trials and observational studies [ , ] . caregivers stress will be assessed by the rss, a self-report instrument covering three dimensions of "emotional distress", "social distress" and "negative feelings". it is regarded as a useful instrument to stratify careers according to the risk of psychiatric morbidity [ , ] . the secondary outcomes presented in table include measures of qol, psychiatric symptom load, adl, comorbidity and pain as well as measure of goal achievements. the qol for both the pwd and the caregiver will be measured by self-report using the quality of life in alzheimer's disease scale (qol-ad) [ ] and the generic quality of life measure eq- d- l [ ] , including the eq- d-vas scale [ ] . additionally, qol for the pwd will be assessed by proxy by the caregiver with the qol-ad [ ] . psychiatric symptoms for the pwd will be proxy rated by the caregiver using the neuropsychiatric inventory questionnaire (npi- ) [ ] , the cohen-mansfield agitation inventory (cmai) [ , ] and the cornell scale for depression in dementia (csdd) [ ] , fig. a stepped-wedge randomized control design. the randomization in time takes place at month . first group (red) is in the intervention period from month to , second group (yellow) from month to and third group (green) from month to . implementation seminars will be held at months , and , and midway evaluation at months , and . data will be collected at baseline (month ), after the first intervention period (month - ), after the second intervention period (month - ), after the third intervention period (month [ ] [ ] and at the end of the study at months. b schedule of enrollment, interventions and assessments over the study period mean mean difference in score over the -month intervention period summarized for the three while caregiver psychiatric symptoms will be selfreported using the geriatric depression scale (gds) [ ] in addition to the rss [ ] . data on adl for the pwd will be proxy rated by the caregiver utilizing instrumental (i-adl) and personal (p-adl) measures [ ] . data on pain will be obtained by self-report from the pwd using the mobid- pain scale [ ] [ ] [ ] [ ] [ ] and the level of comorbidity will be evaluated by the interviewer according to the general medical health rating scale (gmrh) [ ] . the clinical global impression of change scale (cgic) will be assessed after the intervention to quantify and track patient progress and treatment response [ ] . in addition to the instruments presented in table , other outcome measures include the number of adverse events (falls, disappearances outdoors, fire hazard), use of assistive technology (number of technical aids, cognitive intervention devices and assisted living systems), involvement of volunteers (number of participants with contact with a volunteer, number of hours spent with a volunteer), number of medications used (both regular and on demand) and participation in educational programs for the pwd and the caregiver. these outcome measures will be described as the mean change in sum of events (number devices, hours, medications, educational programs) over the intervention period compared to controls (as outlined in table ). prior to inclusion and baseline data collection, a -day seminar will be arranged for the study personal to secure training in the use of tablets and scoring of relevant psychometric scales. a study manual has been developed to guide data collectors during their visits to secure standardized reporting. close to h/day, telephone and mail support will be offered by the research team during times of data collection. researchers and municipal study personal will collect data at baseline as well as , , and -month follow-up. the municipalities will receive nok per enrolled dyad to compensate for extra administrative work. at baseline, demographic data such as year of birth, gender, marital status, housing characteristics, education and employment will be collected, as well as data on the dementia syndrome, including the current score on the mini-mental state examination, norwegian version (mmse-nr ) [ , ] , mean difference in score over the -month intervention period summarized for the three intervention groups compared to mean difference in score summarized for the control groups a mean difference in score over the follow-up period in -month intervals stratified by time from end of intervention b all assessment will be made by research personal or affiliated staff in the municipalities during home visits with the person with disability (pwd) and the caregiver a intervention groups: group (red), t -t ; group (yellow), t -t ; group (green), t -t . control groups: (t -t + t -t ) (see fig. a ) b group (red): three -month periods, t -t , t -t and t -t . group (yellow): two -month periods, t -t and t -t . group (green): one -month period, t -t (see fig. a ) functional assessment staging test (fast) [ ] and the informant questionnaire on cognitive decline in the elderly (iqcode) [ , ] . the mmse-nr [ ] will be assessed every months during the trial. intention-to-treat analyses will be performed accounting for municipality as a random effect in mixed-effect models and the generalized estimating equation (gee) with nonlinear effect comparing the intervention groups to controls. repeated observations within persons will be accounted for with a correlation matrix. all secondary outcomes will be adjusted for multiple comparisons using the hochberg method [ ] . given the potentially informative censoring due to dropout, institutionalization and death, we will jointly model the primary outcome and attrition through a shared person-specific random intercept. missing data will be handled using multiple imputations by chained equations (mice). the study was approved in may by the regional committee for medical and health research ethics, north norway ( / ) and west norway ( / ) (the pilot), and registered at clinicaltrials.gov (nct ). assessment and utilization of personal data from the dyads as well as from volunteers and volunteer coordinators from nonprofit organizations are approved by the norwegian centre for research data (nsd) (ref. ). after verbal and written information, spoken and written informed consent was obtained in direct conversation with the caregiver and the pwd, if capable of providing consent for participation. if not, the next of kin or a legal advocate provided consent based on their determination on whether the pwd, when they were able, would have agreed to participate in the trial. compared to care as usual, we expect the live@home.-path trial to innovate the clinical pathway in dementia care, facilitating cost-effective, feasible and independent living at home through learning, innovation, volunteering and empowerment. participation in research is based on affirmative, unambiguous, informed and specific consent [ ] . persons with cognitive impairment will often not be able to provide such a comprehensive consent or understand the scope and consequences of data assessment. local legislation for obtaining ethical permission in studies varies substantially between european countries [ ] . in norway, the next of kin or a legal advocate can provide consent based on their determination of whether the person, when they were able, would have agreed to participate in the trial [ ] . these principles for obtaining informed consent were applied in the live@home.path trial. from , the european union-wide law on data protection, the general data protection regulation (gdpr), represents a significant step toward protection of participants in research [ ] . in particular, article protects pwds and their relatives from being coerced to consent without awareness of how their data will be used [ , ] . when assessing sensitive data such as mental health, article requires a data protection impact assessment (dpia), a formal process systematically analyzing, identifying and minimizing the data protection risks of a project. we developed a dpia (ephorte uib: / ) for the live@home.path trial in collaboration with the data protection official at the university of bergen, encouraging us to again evaluate which data to assess, as well as focus on safe data management. nonetheless, we anticipate the participation in the live@home.path trial to be less burdensome relative to, for example, rcts on effect of medications, due to the user-oriented approach emphasizing the investigation of the perspective "what matters to you?" stakeholders and research funders increasingly require patient and public involvement (ppi) at all stages of research from design, implementation and dissemination of results, shifting focus from research "about" or "for" to research "with" or "by" someone [ , ] . our userrepresentative has provided feedback on a close to weekly basis through participation in the working group and advisory board of the trial. a related principle, responsible research and innovation (rri), is defined as a transparent, interactive process making societal actors and innovators mutually responsive to each other, and encouraging them to set up a critical perspective when evaluating the innovation and marketability of products [ , ] . taken together, these components constitute a framework for sustainable ethic innovation in dementia research (fig. ) , a model that easily can be applied when designing and conducting research on other vulnerable patient groups. a stepped-wedge randomized controlled trial design is recommended for evaluation of a multicomponent intervention in health-care services as it provides a number of practical and scientific benefits compared to an ordinary rct [ ] . it is increasingly used in effectiveness studies in the geriatric field [ , ] . most importantly, the design allows for providing the intervention to all participants, overcoming ethical and logistical challenges arising from withholding the intervention. this design is, however, more vulnerable to temporal external changes, as more participants are exposed to the intervention toward the end of the study than in earlier stages. if the live intervention fails to prove an effect on resource utilization, we will examine whether this is due to a lack of proper implementation. thus, if the implementation process is satisfactory, it may suggest that the live components were not tailored to be sufficiently cost-effective if no effects on primary outcome measures are found. an alternative interpretation is that the intervention may not be cost-effective even if primary outcomes change significantly, as resource use by the intervention is more time consuming and/or expensive than the alternative. some challenges have emerged during the start of the trial. first, it is demanding to include the estimated number of participants, and, additionally, to keep the number of dropouts low due to the progression of the disease. we should have established closer collaborations with the geriatric specialist health-care services, as we experienced that patients recruited from geriatric outpatient clinics were in the most optimal disease stage for this trial. to increase recruitment, we prolonged the inclusion period to december and expanded the inclusion criteria to age ≥ years and mmse range - , while the sefas researchers, journalist and co-researcher with user experience continuously work on positive media coverage. second, data collection from home-dwelling persons in three distinct municipalities is resource and logistically demanding. third, being selected as a pilot for the data collection software has been challenging, as the file format initially generated handled missing data in a way that was not compatible with our statistical programs. finally, the participants have so far been recruited in various ways, from home-care services in the municipality and memory clinics at hospitals, to self-referrals after advertisements in the general media. this implies that the dyads included in our trial represent a heterogeneic group of home-dwelling people with dementia. in conclusion, we expect the implementation of live to lead to a pathway for dementia treatment and care that is cost-effective, feasible and supports independent living, at home. a total of dyads had been screened for participation from may , of which were included in the trial. by january , when recruitment ended, dyads had dropped out. mainly due to a more rapid inclusion process than anticipated, this protocol was submitted after the end of the recruitment period but in due time before the last visit for data collection. at the time of resubmission in may , the covid- pandemic had profoundly impacted the norwegian healthcare system, including services in the municipal sector, challenging the implementation of the intervention in group . newsletters with status, possible modifications and upcoming events will be sent by e-mail to the site leaders and coordinators every - months. final protocol version number will be prepared by june . plan for dissemination apart from the usual academic publications from the live trial in terms of papers and conference presentations, the authors will ensure maximum publicity through the collaborating centers' popular blogs, media work and scientific network. the latter includes most of the world's leading experts on pain, bpsd, palliative care, and wearable and sensing technology for people with dementia. we will exploit the technology network, cost-action td group, and conduct research visits to three of the overseas associated centers of excellence (harvard university, yale university and tohoku university) that are part of our management group; host at least four visits by overseas members of the network; host two major -day international workshops (years and ); and host nine seminars for formal caregivers in homecare services. the live website will also provide a forum for outreach for the public, including research participants, continuously updated with results from the trial. researchers will attend two international conferences per year, while we expect each researcher to attend a conference every other year to achieve coverage and exposure of the trial. conception or design of the work: bsh is the principal investigator of the trial, lib is the site lead for the trial. all coauthors have contributed substantially to the conception of the idea and at the different stages of development of the trial and/or toward the different components of the intervention and/or practical conduction of the trial. drafting the article: bsh and lib drafted the manuscript. critical revision of the article: all coauthors contributed significantly to the critical revision of the drafts, improving the method and its content. final approval of the version to be published: all coauthors approved the final submitted version of the manuscript. the international committee of medical journal editors criteria for authorship will be applied to evaluate whether contributors fulfill the criteria for authorship on future publications with data from the trial. no professional writers will be involved in manuscripts with data from the trial. the trial is funded by the research council of norway (www.forskningsradet. no) (sponsor's protocol code ), the research council of norway (sponsor's protocol code-pre-project (uib) and (haraldsplass deaconess hospital)), including two phd positions (mv and mhg) and three postdoctoral positions (rca, np and lib). the dignity centre funds one additional phd position (eh). the sponsors will have no role in planning the design, collection, management, analysis, interpretation of data and writing of reports and will have no decision on where to submit the report for publication. data sharing is not applicable to this article as no datasets were generated or analyzed during the current study. the public will not receive full access to the complete protocol, dataset and statistical procedures; however, this information can be made available to other researchers upon request. the study was approved in may by the regional committee for medical and health research ethics, north norway ( / ) and west norway ( / ) (the pilot), and registered at clinicaltrials.gov (nct ). assessment and utilization of personal data on the dyads, volunteers and volunteer coordinators from nonprofit organizations are approved by the norwegian centre for research data (nds) (ref. ). after verbal and written information, spoken and written informed consent was obtained in direct conversation with the caregiver and the pdw, if capable of providing consent for participation. if not, the next of kin or a legal advocate provided consent based on their determination on whether the pwd, when they were able, would have agreed to participate in the trial. not applicable. the world health organization, dementia a public health priority dementia: a global health priority-highlights from an adi and world health organization report the world health organization, facts on dementia heterogeneity and changes in preferences for dying at home: a systematic review we live as good a life as we can, in the situation we're in"-the significance of the home as perceived by persons with dementia the norwegian insitute of public health. public health report dementia. oslo: the norwegian institute of public health alzheimer's disease facts and figures. chichago: the alzheimers association effect of a training programme to reduce stress in carers of patients with dementia the going to stay at home program: combining dementia caregiver training and residential respite care time until institutionalization and death in patients with dementia. role of caregiver training and risk factors cost effectiveness of a training program for dementia carers caregiver burden among dementia patient caregivers: a review of the literature evidence of what works to support and sustain care at home for people with dementia: a literature review with a systematic approach a randomized controlled trial of a community-based dementia care coordination intervention: effects of mind at home on caregiver outcomes a multidimensional home-based care coordination intervention for elders with memory disorders: the maximizing independence at home (mind) pilot randomized trial comprehensive home-based care coordination for vulnerable elders with dementia: maximizing independence at home-plus-study protocol mind at home-streamlined: study protocol for a randomized trial of home-based care coordination for persons with dementia and their caregivers all interventions are complex, but some are more complex than others: using icat_sr to assess complexity theoretical foundations guiding culture change: the work of the partnerships in dementia care alliance assessing the complexity of interventions within systematic reviews: development, content and use of a new tool (icat_sr) making sense of complexity in context and implementation: the context and implementation of complex interventions (cici) framework cosmos-improving the quality of life in nursing home patients: protocol for an effectiveness-implementation cluster randomized clinical hybrid trial the effect of a multicomponent intervention on quality of life in residents of nursing homes: a randomized controlled trial (cosmos) deprescribing antihypertensive treatment in nursing home patients and the effect on blood pressure implementing a novel strategy for interprofessional medication review using collegial mentoring and systematic clinical evaluation in nursing homes (cosmos) description of an advance care planning intervention in nursing homes: outcomes of the process evaluation advance care planning in nursing homes-improving the communication among patient, family, and staff: results from a cluster randomized controlled trial (cosmos) the ministry of health and care services. the dementia plan . oslo: the ministry of health and care services developing and evaluating complex interventions: the new medical research council guidance effectiveness of educational interventions for informal caregivers of individuals with dementia residing in the community: systematic review and meta-analysis of randomised controlled trials the effect of psychosocial support intervention on depression in patients with dementia and their family caregivers: an assessor-blinded randomized controlled trial the norwegian advisory unit on ageing and health. educational program for relatives a review of the role of assistive technology for people with dementia in the hours of darkness telehealth and telecare need a different approach sensing technology to facilitate behavioral and psychological symptoms and to monitor treatment responses in people with dementia. a systematic review security approaches in using tablet computers for primary data collection in clinical research volunteerism research: a review essay updated numbers on volunteering in norway satelite acount for non profit organisations the bravo team. the benefits associated with volunteering among seniors: a critical review and recommendations for future research volunteering and subjective wellbeing in midlife and older adults: the role of supportive social networks meeting needs in a welfare state: relations between government and voluntary organizations in norway the norwegian government. the voluntary sector the red cross the norwegian association for public health co-producing a shared understanding and definition of empowerment with people with dementia the national ministry of health, national guideline for diagnosis and treatment of dementia. oslo: the national ministry of health survival times in people with dementia: analysis from population based cohort study with year follow-up the compound role of a coordinator for home-dwelling persons with dementia and their informal caregivers: a qualitative feasibility study of the live@home.path trial submitted implementation research: what it is and how to do it republished research: implementation research: what it is and how to do it end-of-life care and the effects of bereavement on family caregivers of persons with dementia statistics norway, population statistics of norway the norwegian national advisory unit of ageing and health, diagnostic criteria for dementia the stepped wedge cluster randomised trial: rationale, design, analysis, and reporting designing and conducting mixed methods research conducting hermeneutic research application of resource utilization in dementia (rud) instrument in a global setting the resource utilization in dementia (rud) instrument is valid for assessing informal care time in community-living patients with dementia measuring behavioural disturbance of elderly demented patients in the community and its effects on relatives: a factor analytic study recording care time in nursing homes: development and validation of the "rud-foca" (resource utilization in dementia-formal care) the relative stress scale, a useful instrument to identify various aspects of carer burden in dementia? high score on the relative stress scale, a marker of possible psychiatric disorder in family carers of patients with dementia use of the qol-ad for measuring quality of life in people with severe dementia-the laser-ad study utility-based quality of life measures in alzheimer's disease correspondence between eq- d health state classifications and eq vas scores the neuropsychiatric inventory: comprehensive assessment of psychopathology in dementia factor analysis of the cohen-mansfield agitation inventory in three large samples of nursing home patients with dementia and behavioral disturbance assessment of agitation in elderly patients with dementia: correlations between informant rating and direct observation cornell scale for depression in dementia development and validation of a geriatric depression screening scale: a preliminary report aging and performance of home tasks the mobid- pain scale: reliability and responsiveness to pain in patients with dementia who suffers most? dementia and pain in nursing home patients: a cross-sectional study pain behaviour and pain intensity in older persons with severe dementia: reliability of the mobid pain scale by video uptake pain in older persons with severe dementia. psychometric properties of the mobilization-observation-behaviour-intensity-dementia (mobid- ) pain scale in a clinical setting mobilization-observation-behavior-intensity-dementia pain scale (mobid): development and validation of a nurse-administered pain assessment tool for use in dementia the general medical health rating: a bedside global rating of medical comorbidity in patients with dementia ecdeu assessment manual for psychopharmacology mini-mental state". a practical method for grading the cognitive state of patients for the clinician the norwegian national advisory unit on ageing and health. mmse-nr functional assessment staging (fast) the informant questionnaire on cognitive decline in the elderly (iqcode): a review the informant questionnaire on cognitive decline in the elderly (iqcode): socio-demographic correlates, reliability, validity and some norms controlling the false discovery rate: a practical and powerful approach to multiple testing the european union general data protection regulation (eu / ) and the australian my health record scheme-a comparative study of consent to data processing provisions huge variation in obtaining ethical permission for a non-interventional observational study in europe the eu's general data protection regulation (gdpr) in a research context impossible, unknowable, accountable: dramas and dilemmas of data law why and how we should care about the general data protection regulation alzheimer europe's position on involving people with dementia in research through ppi (patient and public involvement) public involvement in health and social sciences research: a concept analysis the assisted living project: a process evaluation of implementation of sensor technology in community assisted living. a feasibility study when robots care: public deliberations on how technology and humans may support independent living for older adults nurse-led medicines' monitoring for patients with dementia in care homes: a pragmatic cohort stepped wedge cluster randomised trial alleviating staff stress in care homes for people with dementia: protocol for stepped-wedge cluster randomised trial to evaluate a web-based mindfulness-stress reduction course publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations the live@home.path trial is funded by the research council of norway with two phd grants and three postdoctorate grants. the centre for elderly and nursing home medicine at the university of bergen responsible for conducting the trial is funded by gc rieber foundations and the norwegian government. the authors acknowledge valuable support from the collaborating municipalities and their main contact person anne marie hanson (baerum), beate sørensen (kristiansand) and anita krokeide (bergen), as well as from the dignity centre, the dam foundation western norway university of applied sciences and the norwegian national advisory unit on women's health, oslo university hospital. supplementary information accompanies this paper at https://doi.org/ . /s - - -y.additional file . implementation seminar for the live@home.path trial. key: cord- -nrnc u authors: zhao, yang; wei, yongyue; shen, sipeng; zhang, mingzhi; chen, feng title: appealing for efficient, well organized clinical trials on covid- date: - - journal: nan doi: . / . . . sha: doc_id: cord_uid: nrnc u the rapid emergence of clinical trials on covid- stimulated a wave of discussion in scientific community. we reviewed the characteristics of interventional trials from chinese clinical trial registration (chictr) and clinicaltrials.gov. a total of covid- -related interventional trials were identified on feb nd, . these trials are classified into categories based on treatment modalities, including chemical drugs, biological therapies, traditional chinese medicine treatments and other therapies. our analysis focused on the issues of stage, design, randomization, blinding, primary endpoints definition and sample size of these trials. we found some studies with potential defects including unreasonable design, inappropriate primary endpoint definition, insufficient sample size and ethical issue. clinical trials on covid- should be designed based on scientific rules, ethics and benefits for patients. as of february , , approximately , cases of coronavirus disease (covid- ) have been confirmed worldwide, with nearly , deaths occured. recently, many covid- -related interventional clinical trials have emerged in china, the original and high-incidence area of covid- . the rapid emergence of these trials stimulated a wave of discussion. herein, we reviewed the characteristics of interventional trials from chinese clinical trial registration (chictr) and clinicaltrials.gov. , the data of interventional trials from chictr and clinicaltrials.gov were retrieved updated on february , . two authors (zhao and shen) were independently responsible for collecting the relevant information, including clinical phase, study design, presence or absence of randomization, blinding, sample size, severity of disease and source of samples. a total of covid- -related interventional trials were identified ( from chictr and from clinicaltrials.gov). the registration date distribution of these trials was shown in figure . of the trials registered at chictr, were approved by the institutional review boards. the trials are classified into categories based on treatment modalities, including chemical drugs (cds), biological therapies (bts), traditional chinese medicine (tcm) treatments and other therapies. the characteristics of these trials were summarized in table more trials on tcm treatment use or more primary endpoints (pes) than those on cds or bts ( . %, . % and . %, respectively). some trials even have or more pes. the most common used pes are associated with clinical symptoms( . %), pathogen( . %) and lung function( . %). trials on cds have greater proportions of pathogen ( . %) or fatality ( . %) related pes than the others. it is worth noting that these trials have registered over distinct pes totally. during the outbreak, we are in urgent need of an effective treatment strategies. through analyzing currently-registered interventional trials, we found some studies with potential defects including unreasonable design, inappropriate pe definition, small sample size and ethical issue. determining the appropriate pes is crucial to address the primary scientific question of the study. the pes should be defined clearly and specifically to improve the operation quality. objective endpoints, such as pathogen or imaging results, are preferred to avoid the bias during evaluation. it is possible that some trials lack statistical power. assuming a two-side significance level of . and power of %, a trial on mild pneumonia patients should recruit nearly , patients if the effective rate increases from % to % for the new treatment. more patients are needed for the trials planning to decrease the fatality of critically ill patients. some drugs and therapies under study are short of previous theoretical support for . these drugs and therapies may be not beneficial to the patients, leading to non-compliance to ethical standards. these unnecessary trials may also waste medical resources, as well as diverting patients resources. who has suggested an establishment of a centralized research program to ensure the most promising researches . . cc-by-nc . international license it is made available under a author/funder, who has granted medrxiv a license to display the preprint in perpetuity. is the (which was not peer-reviewed) the copyright holder for this preprint . https://doi.org/ . / . . . doi: medrxiv preprint as appealed by some chinese scientists recently, clinical trials on covid- should be designed based on scientific rules (appropriate controls, randomization, blinding, and sufficient sample size)，ethics and patients' benefits cc-by-nc . international license it is made available under a author/funder, who has granted medrxiv a license to display the preprint in perpetuity. is the (which was not peer-reviewed) the copyright holder for this preprint **three trials are registered as "factorial design", one for tcm, one for psychological intervention to doctors and nurses, and one for probiotics. however, unlike the general purporse of factorial design, none of the trials aim to find the best "combination" of the treatments. ***trials registered at clinicaltrials do not provide information about the collected samples. #percentages for pe type are calculated by the number of trials using this pe divided by the number of corresponding type of trials. percentages of severity and samples are calculated in the same manner. . cc-by-nc . international license it is made available under a author/funder, who has granted medrxiv a license to display the preprint in perpetuity. is the (which was not peer-reviewed) the copyright holder for this preprint . https://doi.org/ . / . . . doi: medrxiv preprint more than clinical trials launch to test coronavirus treatments an urgent call for raising the scientific rigorousness of clinical trials on covid- key: cord- -m suxh authors: rodgers, f.; pepperrell, t.; keestra, s.; pilkington, v. title: missing clinical trial data: the knowledge gap in the safety of potential covid- drugs date: - - journal: nan doi: . / . . . sha: doc_id: cord_uid: m suxh abstract ( / words) objectives: several drugs are being repurposed for the treatment of the coronavirus disease (covid- ) pandemic based on in vitro or early clinical findings. as these drugs are being used in varied regimens and dosages, it is important to enable synthesis of existing safety data from clinical trials. however, availability of safety information is limited by a lack of timely reporting of clinical trial results on public registries or through academic publication. we aimed to analyse the knowledge gap in safety data by quantifying the number of missing clinical trial results for drugs potentially being repurposed for covid- . design: clinicaltrials.gov was searched for drugs that have been identified as potential treatments for covid- . relevant clinical trials for any prior indication were listed by identifier (nct number) and checked for timely result reporting (within days of the primary completion date). additionally, pubmed and google scholar were searched using the nct number to identify publications of results not listed on the registry. a second, blinded search of % of trials was conducted to assess reviewer concordance. results: of completed trials, ( . %) did not post results on clinicaltrials.gov or in the academic literature. ( . %) completed trials had tabular results on clinicaltrials.gov. a further ( . %) completed trials had results from the literature search, but did not report results on clinicaltrials.gov. key drugs missing clinical trial results include hydroxychloroquine ( . % completed trials unreported), favipiravir ( . %) and lopinavir ( . %). conclusion: there is an important evidence gap for the safety of drugs being repurposed for covid- . this uncertainty could cause a large burden of additional morbidity and mortality during the pandemic. we recommend caution in experimental drug use for non-severe disease and urge clinical trial sponsors to report missing results retrospectively. coronavirus disease (covid- ) is a pandemic infection caused by severe acute respiratory syndrome coronavirus (sars-cov- ). its global spread has been rapid and unprecedented, at the time of writing , , confirmed cases have been reported and , deaths across countries ( ) . currently, treatment options for covid- are limited. however, several drugs developed for other indications have shown promising results against sars-cov- in vitro, in animal models, or in compassionate use trials ( , ) . many of these drugs are now being experimentally repurposed for covid- or are undergoing clinical trials in humans ( ) . such candidates include nitazoxanide, remdesivir, favipiravir, lopinavir, darunavir, hydroxychloroquine, chloroquine, and ivermectin amongst others ( ) . investigations into the efficacy of these experimental treatments for covid- are ongoing. meanwhile, in response to positive media coverage, some speculative rather than evidence-based, governments are stockpiling vast supplies of these treatments in anticipation of their licensing for covid- . furthermore, national regulatory institutions, meant to safeguard against the unsafe use of drugs, are under increasing pressure to relax approval standards to accelerate market-entry for covid- treatments. for example, on april th the u.s. food and drug administration (fda) approved the antimalarial drug hydroxychloroquine for emergency treatment of covid- with unknown optimal dosage and duration of treatment ( ) . however, the efficacy of hydroxychloroquine is still under question ( ) . furthermore, it is cardiotoxic at the higher doses which may be indicated for covid- , causing qt prolongation leading to ventricular tachycardia and death ( ) . care must be taken not to lose the rigorous safety standards usually stipulated for pharmaceuticals, even during a pandemic, to avoid unnecessary morbidity and mortality worldwide. given the pandemic status of covid- , it is more important than ever that safety information for drugs potentially being repurposed is publicly available. pharmacokinetics for many drugs differ by phenotype. examples include ace inhibitors, which are cautioned in people of african descent, and dolutegravir, which causes greater weight gain in women than men ( , ) . as pharmaceutical companies often run trials in high income countries, treatment cohorts are often largely white ( ) . furthermore, regulatory authorities are often conservative about including pregnant women in trials, discouraging female recruitment ( ) . as experimental treatments for covid- may be distributed to millions of people worldwide, maximising phenotypic variety as well as the total number of person-years-of-follow-up available in the public realm is vital for ensuring that rare adverse events and contraindications can be anticipated. as many of these drugs have been used widely for years, there should be substantive information on safety, tolerability and pharmacokinetics available in the public domain, including public trial registries. yet, whilst the international committee of medical journal editors (icmje) policy requires prospective registration of interventional studies on a who primary registry or on clinicaltrials.gov, the icmje does not currently require researchers to report summary results on these registries ( ) . however, according to the fda amendment act , the responsible party for applicable clinical trials that are registered on clinicaltrials.gov must report results to a public register within twelve months of the primary completion . cc-by . international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. (which was not certified by peer review) the copyright holder for this preprint this version posted june , . . date, or in some cases risk a fine of $ , for every day results are delayed ( , ) . failure to share clinical trial results publicly can additionally have far-reaching consequences for health and public expenditure, as illustrated by the widespread stockpiling and prescription of oseltamivir (tamiflu) during the swine flu outbreak in , despite a lack of evidence on safety and efficacy ( ). clinical trial transparency is therefore vital in order to maximise and unify the sharing of safety information and data on efficacy, during this pandemic and beyond. public clinical trial registries are an important tool for transparent collaborative research. on these registries, safety and efficacy data can be uploaded freely, shortly after completion of the study, and protocol and data collection methods are still quality assessed ( , ) . in contrast, academic publication may stall for significant periods of time as can be costly and selective, with time-intensive writing and review processes. furthermore, clinical trial registry data is available free of charge and can be pooled without concern for silent outcome switching or publication bias ( ) ( ) ( ) . this is especially important during the covid- pandemic, as interest in potential treatments for coronavirus generates even greater incentive than normal for the publication of studies with positive results only. without rapid sharing of datasets for drugs that may be repurposed for covid- , secondary analyses of safety data will be arduous and often incomplete ( , ) . this may slow down the biomedical innovation process and could lead to preventable side effects occurring in vulnerable patients if safety information remains missing. as the pharmaceutical pipeline is accelerated to address the covid- pandemic, enhancing clinical trial transparency is more important than ever. in this study, we aimed to determine the scale of unpublished clinical trial results which may hinder safety reviews of repurposed drugs for covid- . we reviewed the number of completed or terminated trials that have not reported results for an extensive, list of medications being repurposed for covid- , looking at all previous indications for these drugs. specifically, we searched for any trial results that have not been made available to the public, with no results published on either the clinicaltrial.gov registry or in the academic literature. the drugs assessed were pirfenidone, hydroxychloroquine, azithromycin, favipiravir, oseltamivir, sarilumab, tocilizumab, remdesivir, leflunomide, interferon-alpha, lopinavir-ritonavir, darunavir-ritonavir, baloxivir marboxil, umifenovir, interferon-beta, sofosbuvir, nitazoxanide, apn and ivermectin (table ) . these drugs were selected based on information found in potential covid- treatment reviews ( , , ) . synonyms and chemical names for these drugs were taken from pubchem.ncbi.nlm.nih.gov (appendix ) ( ). the u.s. clinical trials registry (clinicaltrials.gov) was searched for all trials that listed these drugs as an intervention, results of the search were downloaded on th april ( ). numbers of trials with and without results on the registry were recorded. for trials without results, trial status was determined (completed, ongoing, suspended, terminated or withdrawn). trials listing 'primary completion date' in the future were counted as ongoing, if no primary completion date was available then the study completion date was used. listed trial status was used to identify ongoing, terminated, suspended and withdrawn trials. for all trials without results on clinicaltrials.gov, a three-step process was followed between th - th april to determine whether results were reported elsewhere through academic publication ( figure ). included based on criteria below. if multiple publications were listed, the earliest dated publication was selected. ) if results were not available on the registry, the clinical trial identifier (nct number) was used to search and screen academic publications in pubmed. ) if the pubmed search did not retrieve an academic publication, an additional search was conducted in google scholar using the following search terms in succession: clinical trial identifier, listed title, intervention name with primary investigator's name. for each search, the first twenty results were screened. . cc-by . international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted june , . . if a publication did not include the clinical trial identifier, it was cross-referenced with the primary investigator, study design, intervention and outcomes listed on clinicaltrials.gov to assess relevance. we excluded publications that had fewer than words, as well as conference abstracts, posters, presentations and non-english texts. for ry e , . cc-by . international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted june , . . publications of results were recorded by pmid, doi and publication date. trials with results on clinicaltrials.gov combined with those with a journal publication gave a total number of trials with results where results were found in the public domain. this allowed approximation of registered trials without results. additionally, overdue trials were calculated as any completed trial with no result on the registry and a primary completion date before th april , days prior to final analysis ( year + -day grace period). this is the standard outlined in the fdaaa and used as a reference throughout this study despite not all included trials being covered by the law ( ). this is also consistent with international ethical standards for timely results dissemination ( ) . a second review was conducted by a different researcher on % of trials for each drug to check concordance between reviewers. the protocol during the second review remained unchanged and researchers were blinded to the results of the first review. a random number generator was used to select trials for second review. concordance was calculated by simple percent agreement on a results judgement; above % was deemed acceptable from consulting experts and in line with recommendations in the literature ( ). results published between the dates of first and second review ( th april - th may ) were not counted in the concordance. patients and the public were not involved in any way in this research. . cc-by . international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted june , . . nineteen drugs were screened, encompassing a total of clinical trials registered on clinicaltrials.gov (table ) . we excluded , with ongoing (primary completion date not passed) and with a trial status of suspended or withdrawn. figure shows the number of trials found on clinicaltrials.gov, those excluded from this analysis, and the final results status of all included trials (n= ). all recorded percentages in text are in relation to the completed trials, seen in table . is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted june , . in the blinded second review of ( %) trials, the same results status was found . % of the time meeting our - % threshold for acceptable concordance in searches. . cc-by . international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted june , . . cc-by . international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted june , . . cc-by . international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted june , . . https://doi.org/ . / . . . doi: medrxiv preprint discussion . % of the completed clinical trials for drugs that may be repurposed for covid- were not found to report results on either clinicaltrials.gov or through academic publication (table ). this shows a large gap in the evidence base for these drugs, limiting attempts to comprehensively review their safety before potential global distribution for the covid- pandemic. the ( . %) completed studies with available results were comprised of ( . %) with results on the registry and ( . %) without results on the registry, but with results from a standardised search of the literature (table ) . furthermore, ( . %) studies without registry results were outside of the day timeframe for results publication as mandated by the fdaaa ( ), although of these had results published in academic literature ( . %) . not all trials include in this study are applicable under the fdaaa , but this remains a benchmark for good scientific practice ( ) . with . % of clinical trial results unavailable for potential covid- treatments, the data for clinical decision making regarding the safety of these therapeutics are limited. if any drug with an incomplete evidence base is used during the pandemic, even in compassionate use programmes, there is a risk of avoidable harm being done because of missing adverse safety data. a knowledge gap was revealed for drugs which have had extensive media coverage such as hydroxychloroquine ( . % without results), favipiravir ( . %) and lopinavir ( . %) ( ) ( ) ( ) . these drugs are currently being used in covid- patients or trials across the globe, sometimes in novel regimens and doses ( , , ) . clinicians have few treatment options available, but with greater transparency and proactiveness from clinical trial sponsors regarding the posting of trial results, there would be less risk of unforeseen adverse outcomes, especially in the treatment of mild-moderate covid- as in the pioneer trial ( ) . public health decision-makers, guideline developers, clinicians and patients rely on clinical trials, systematic reviews and meta-analyses to inform treatment. evidence gaps and publication bias therefore influence clinical practice and drug usage worldwide, particularly in a treatment landscape as changeable as the covid- pandemic. if clinical decisions are based on incomplete evidence, this can result in avoidable morbidity and mortality if unsafe drugs or ineffective treatments are given on a large scale. sponsors and researchers alike also carry an ethical responsibility towards clinical trial participants, who consent to participate in research in order to contribute to scientific understanding and improved clinical practice, to make results publicly available ( ) . clinical trial results that are not made available publicly do not fulfil this expectation, betraying the trust of participants who may have given up time and health for the benefit of science. missing evidence also impacts the direction of future research, which is informed by the existing available body of literature. our study reveals an important knowledge gap regarding pharmaceuticals potentially being repurposed for covid- . however, the proportion of studies with results available in the academic literature is an approximation and there are several limitations to our study. our trial population was limited only to those registered on clinicaltrials.gov. while clinicaltrials.gov is the largest registry in the world, with over , registrations as of writing, additional trials on these therapies may have been registered elsewhere. however, it is unclear if these would be expected to report at a different rate than those registered to clinicaltrials.gov. our strategy for locating publications included only those listed on . cc-by . international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted june , . . scholar, open resources that should cover a majority of published clinical research. while including proprietary databases like scopus or ovid may have located some additional publications, we do not believe this would have substantially impacted our overall results ( ) . additionally, trials that were not registered at all or published in non-english language journals without inclusion of the nct number would not have been captured by our methodology. searcher heterogeneity and difficulty identifying results publication in the academic literature limits accuracy in any manual publication search, however, our search strategy was standardised and produced a high level of agreement between assessors in a check of a % random sample ( . %). however, the discordance present between reviewers only illuminates the inherent difficulties in finding results for the drugs in question, especially if the trial id number was not included in line with consort standards ( ) . our findings add to the existing evidence of the dearth of clinical trial reporting on public registries. this analysis investigated clinical trials of drugs currently being considered for use for covid- . however, given the diversity of drug classes included in this report, findings are likely to be representative of many pharmaceuticals. this presents a major problem for researchers attempting to summarise safety and efficacy by pooling trial data ( ) . since academic publications often summarise key findings only, secondary research efforts are impinged. clinicaltrials.gov provides a forum to share complete safety and efficacy data reports, as well as facilitating consistent data reporting in a timely manner ( , ) . prior research has shown that results reported to clincialtrials.gov were often more complete, especially for safety data, when compared to matched journal publications ( ) ( ) ( ) . however, this depends on researchers registering trials and uploading results in a timely manner, within twelve months of the primary completion date. the international committee of medical journal editors (icmje) and the editorial offices of medical journals could play an important role in improving the lack of timely results posting on clinical trial registries by demanding submission of a link to summary results on public registries before academic publication, although this may mean that the publication bias of positive, 'publishable' results may trickle down to reporting on public registries as well. furthermore, public funders and institutional publication funds could demand that trial sponsors post their results before allocating funding for academic publication. these funding bodies could also deny individual sponsors funding if they have in the past violated clinical trial reporting rules ( ) . at the very least, journals should conform to the consort statement in ensuring that registry ids are clearly indicated in the abstract, full-text, and meta-data of published clinical trials in order promote easier discoverability and record linkage between registries and publications ( ) . finally, clinical trial sponsors, such as universities, hospitals, public research institutions, and pharmaceutical companies, should themselves work towards improving their institutional clinical trial reporting performance by making use of available resources that provide detailed step-bystep instructions as to how to go about this task ( ) . especially during the covid- pandemic, it is of great importance that trials sponsors release summary results on these registries retrospectively to inform decision making around existing treatments being re-purposed for covid- . overall, our findings reveal a significant evidence gap for the safety of drugs being repurposed for covid- . this uncertainty could cause a large burden of extra morbidity in the global pandemic. we . cc-by . international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted june , . . recommend caution in experimental drug use for non-severe disease and urge trial sponsors to report missing results retrospectively. medicine during the covid- pandemic cannot be evidence-based if a large proportion of the evidence is missing. fr and tp conceptualised the study, devised the methodology, and coordinated the research team. sk recruited the research team. all authors were involved in data collection and analysis, and contributed to the final manuscript. data is publicly available from the u.s. national library of medicine (https://clinicaltrials.gov/ct /home). i, the submitting author has the right to grant and does grant on behalf of all authors of the work (as defined in the below author licence), an exclusive licence and/or a non-exclusive licence for contributions from authors who are: i) uk crown employees; ii) where bmj has agreed a cc-by licence shall apply, and/or iii) in accordance with the terms applicable for us federal government officers or employees acting as part of their official duties; on a worldwide, perpetual, irrevocable, royalty-free basis to bmj publishing group ltd ("bmj") its licensees and where the relevant journal is co-owned by bmj to the coowners of the journal, to publish the work in bmj open and any other bmj products and to exploit all rights, as set out in our licence. the submitting author accepts and understands that any supply made under these terms is made by bmj to the submitting author unless you are acting as an employee on behalf of your employer or a postgraduate student of an affiliated institution which is paying any applicable article publishing charge ("apc") for open access articles. where the submitting author wishes to make the work available on an open access basis (and intends to pay the relevant apc), the terms of reuse of such open access shall be governed by a creative commons licence -details of these licences and which creative commons licence will apply to this work are set out in our licence referred to above. . cc-by . international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. (which was not certified by peer review) the copyright holder for this preprint this version posted june , . . https://doi.org/ . / . . . doi: medrxiv preprint . cc-by . international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. (which was not certified by peer review) the copyright holder for this preprint this version posted june , . . https://doi.org/ . / . . . doi: medrxiv preprint covid- map -johns hopkins coronavirus resource center initial clinical results announced for favipiravir treatment of novel coronavirus pneumonia -viral clearance in four days remdesivir and chloroquine effectively inhibit the recently emerged novel coronavirus ( -ncov) in vitro treating covid- -off-label drug use, compassionate use, and randomized clinical trials during pandemics pharmacologic treatments for coronavirus disease (covid- ): a review eua hydroxychloroquine sulfate health care provider fact sheet -emergency use authorisation (eua) outcomes of hydroxychloroquine usage in united states veterans hospitalized with covid- . medrxiv effect of high vs low doses of chloroquine diphosphate as adjunctive therapy for patients hospitalized with severe acute respiratory syndrome coronavirus (sars-cov- ) infection management of hypertension in ethnic minorities dolutegravir plus two different prodrugs of tenofovir to treat hiv phase trials of new antiretrovirals are not representative of the global hiv epidemic icmje | about icmje | clinical trials registration. . fdaaa and the final rule -clinicaltrials.gov achieving excellence in clinical trial reporting -bih quest center to help develop the safest, most effective coronavirus tests, treatments, and vaccines comparison of serious adverse events posted at clinicaltrials.gov and published in corresponding journal articles timing and completeness of trial results posted at clinicaltrials.gov and published in journals. dickersin k, editor reporting discrepancies between the clinicaltrials.gov results database and peer-reviewed publications a review of the safety of favipiravir -a potential treatment in the covid- pandemic? therapeutic options for the novel coronavirus ( -ncov) potential treatments for covid- ; a narrative literature review available from: www.consort-statement.org . mchugh ml. interrater reliability: the kappa statistic a political determinant of covid- japanese flu drug "clearly effective" in treating coronavirus, says china | world news | the guardian. . positive results from initial lopinavir-ritonavir covid- clinical trial pioneer study tests treatments for mild to moderate covid- -chelsea and westminster hospital nhs foundation trust a trial of lopinavir-ritonavir in adults hospitalized with severe covid- wma declaration of helsinki -ethical principles for medical research involving human subjects -wma -the world medical association web of science, and scopus: a systematic comparison of citations in subject categories statement: updated guidelines for reporting parallel group randomised trials dissemination and publication of research findings: an updated review of related biases challenges for funders in monitoring compliance with policies on clinical trials registration and reporting: analysis of funding and registry data in the uk we are very grateful for the help of research assistants that helped with compiling the data for this study: helen woodward, joshua card-gowers, joshua lucas, sultan hussein, mina aries, spatikha sitaram, holly melvin, lauren hargreaves, stefano santorini, frances kenworthy, khalifa saif elyazal ali, maymunah malik, shiron rajendran, oliver wright, catherine dominic, holly beckett, tricia tay and daphne lenz. we are also thankful for the comments of till bruckner, peter grabitz, and nicholas devito on the draft protocol. key: cord- -i rzo j authors: lorusso, domenica; ray-coquard, isabelle; oaknin, ana; banerjee, susana title: clinical research disruption in the post-covid- era: will the pandemic lead to change? date: - - journal: esmo open doi: . /esmoopen- - sha: doc_id: cord_uid: i rzo j nan the unprecedented situation we are facing has strongly disrupted the clinical research rules. nevertheless, for the scientific community, it may represent the opportunity to learn important lessons. the covid- pandemic suggests that it is possible to alleviate redundancy in clinical trials, and while preserving the rigour of a study, can offer a new, less burdened and more inclusive vision of clinical research for the scientific community of tomorrow. this perspective article describes clinicians' vision of how the pandemic could change the roles of clinical research. since the beginning of the sars-cov outbreak in wuhan, more than million people have been infected all around the world and more than have died from the disease so far. in this scenario, europe is facing one of the worst crises that our national health systems have ever encountered in the last years. six months after the first covid- diagnosis, the lockdown is being eased in european countries and our lives are slowly adapting to 'a new normality'. providing care to immunocompromised patients with cancer during this pandemic has been extremely challenging and oncologists face many challenges in providing cancer care during the covid- outbreak. data from china reported that patients with cancer who are infected with covid- are at . times the risk of requiring mechanical ventilation or intensive care unit (icu) admission, compared with the general population. additionally, the limitation of resources in outpatient settings, including administrative staff and specialists, has hindered the routine care of patients. national and international cancer societies published priority-driven guidelines for the management of oncohaematological patients on therapy during the covid- pandemic and recommended considering treatment delays and modifications on a case-by-case basis, taking into account the characteristics of the patient and the disease. in addition to routine patient care, the imperative of reducing the number of non-urgent visits to the hospitals, which characterised the last months, had implications for research institutions performing clinical trials. an italian survey of medical oncologists reported that both clinical research and scientific activities were reduced in over % of respondents. although conversion to telemedicine has maintained the continuity of care for many patients, the covid- pandemic has massively disrupted clinical research and many cancer centres halted clinical trial activities including patient recruitment. regulatory agencies have disseminated extraordinary measures to guide healthcare workers to continue clinical trials ensuring patient safety and maintaining data quality. the implementation of these measures has helped mitigate the negative effects of the pandemic on the clinical research field (https:// ec. europa. eu/ health/ sites/ health/ files/ files/ eudralex/ vol / guid ance clin ical trials_ covid _ en. pdf). hospitals have needed to prioritise clinical activities managing patients and staff suffering from covid- . therefore, carrying out clinical trials according to the rigid interpretation of gcp-ich (good clinical practices-international conference on harmonisation) rules was not always practically feasible. as we adapt to 'the new normal', there is a feeling among some healthcare workers that the important lessons learnt during the pandemics will disappear with the end of the emergency phase. in the field of clinical research in oncology, for instance, we learnt that a reasonable balance can be achieved between maintaining the scientific integrity of the study, patient safety and regulatory burden. during the covid- era, both individual researchers and research organisations have realised that there is a different way of delivering clinical research and that 'flexibility' and 'altruism', which have been keywords of the covid- era, may represent one of the legacies as we move forward within the post-covid- world. practical solutions used during the pandemic that merit the consideration of long-term implementation in clinical research include the following. . telemedicine: the covid- pandemic led to an unprecedented change in clinical operations, motivating physicians and healthcare systems worldwide to rapidly implement telemedicine programmes to reduce or replace in-person visits and to allow workforce sustainability and staffing. before the pandemic, telemedicine was underused while actually, it is quickly becoming the preferred mode of delivering care for patients with cancer including follow-up, on-treatment and second opinion consultations. when asked about the perception of safety to attend research visits 'remotely' or in-person, half of clinical trial participants preferred phone visits or telehealth that is enhanced by face-to-face communication. a cochrane review examined the impact of telemedicine involving remote monitoring compared with in-person or telephone visits for chronic conditions, including diabetes and heart failures, and found similar health outcomes. larson et al showed that telemedical intervention in patients with cancer is comparable with face-to-face interaction meetings regarding the quality of life but no data exist about the oncological outcome in patients who received telemedical advice. these measures, although necessary in the context of the pandemic, serve as an opportunity to reconsider the utility of frequent in-person hospital visits for patients enrolled in clinical trials, particularly when the therapeutic is an oral drug that can be delivered by courier service. moreover, less than % of cancer clinical trials adequately accrue ethnic minorities (or . with respect to white patients in cancer clinical trials) and a potential explanation is that travel burden to centres mandated by specific trials (eg, rare cancers) implies time off from work and family commitments for both patients and careers leading to financial repercussions. therefore, reducing the frequency of in-person visits could potentially increase access to clinical trials participation and diversity in recruitment. at present, several activities can be potentially provided by electronic tools (informed consent discussion and signature, medical history collection focused on addressing eligibility criteria during the screening procedures, evaluation of the quality of life and safety with electronic patient reported outcome (epro)) and this could be permanently accepted in clinical trials procedures. the implementation of telemedicine requires funding, user guidelines, data protection integrity and management of reimbursement policies. lack of privacy and security standards plays an important role in the legal challenges facing telemedicine and may have considerable implications for the acceptance of telemedicine services. moreover, the relationship between telemedicine reimbursement rules and access to care is complex; although the covid- pandemic has certainly brought increased coverage for telemedicine services, nationwide standardisation of payment policies is still lacking. with the second-largest burden of covid- in the world, for instance, italy does not currently include telemedicine in the essential levels of care granted to all citizens within the national health service and no formal input was given on telemedicine by health authorities, despite high pressure on health services during the first phase of the epidemic. however, the time has arrived to change this situation, and experts from different fields should work together on this important issue. . remote monitoring visits: given the pandemic, alternative mechanisms of oversight and monitoring have been implemented including remote monitoring. local data protection policies in many parts of europe often precludes the remote source data verification (eg, providing the sponsor with copies or remote access to electronic medical records). it is evident that such a model cannot work alone, nevertheless, a mixed (on-site and remote), risk-based model that takes into consideration national and local restrictions and the urgency of source data verification can be permanently implemented in the new research organisational model. remote monitoring for some clinical trials is feasible and cost saving for the sponsor. this may be particularly relevant for academic-sponsored trials, given that on-site monitoring can account for about % of the total trial budget. . laboratory tests: during the pandemic, when it was not feasible for patients to travel to the clinical trial centre, it was acceptable that blood tests, imaging or other diagnostic tests were done at a closer local facility provided it is certified as per national requirements. the ability for tests to be carried out outside of the trial centre perhaps should be continued and integrated into clinical trial procedures-at least for some safety blood tests that do not represent the primary endpoint of the trial, and for radiological tests when a centralised evaluation (blinded independent central review (bicr)) is planned for progression-free survival end point analysis, or when overall survival is the main objective of the study. moreover, as a general strategy, methods and frequencies of safety assessments should be rationally determined in trial protocols taking into account preclinical and clinical safety data, be scientifically and ethically justified and balanced with the risks associated with hospital visits. finally, the issue of funding needs to be addressed and the cost of these extra hospitals' procedures need to be traced and reimbursed to the laboratories providing the procedures by the sponsor. this may be easier if research networks are established and oncologists and radiologists of pe-ripheral centres are trained as subinvestigators with delegated duties under the supervision of the main study team. uation. in italy, there are ecs actively evaluating trials and the median lead time between clinical trial application (cta) submission and the site initiation visit (siv) is days (median ec evaluation time days). other studies report even longer duration reaching up to - months for the whole process of trial activation. during the emergency, aifa (italian medical agency) appointed a unique ec for evaluating all covid- interventional studies and this dramatically reduced the time of approval (about days between cta and siv). the outstanding lesson we learnt is that is it not more deferrable the brave decision to identify a single ec for trial (or at least for the area) to speed up the process of approval, particularly when looking at the upcoming implementation of the european portal for cta submission that imposes the identification of a central ec and defines strict times for evaluation. when the clinicaltrials directive (euctd) adopted in to facilitate and improve clinical research within europe is strictly followed, the median approval duration is days. lastly, a simplification of approval procedures may translate in a significant reduction of the costs of submission: a swiss trial reported a median time of days and a median cost of . dollars for the submission preparation of a randomised clinical trial to the authorities. . contract negotiation: the example of the tocilizumab trial. community-based research programmes face many barriers to participation in clinical trials, and research contract and budget negotiations have been consistently identified as time-consuming procedures and a barrier to study participation. american society of clinical oncology 's (asco) community research forum conducted a survey about specific challenges in clinical research among clinicians: % of the respondents acknowledged barriers in the process of trial activation in terms of budget negotiation and legal review. after the publication of the survey, asco recommended the standardisation of negotiation processes and the creation of contract templates as necessary tools to implement the trial activation process. during the pandemic, the national cancer institute of naples promoted a therapeutic trial with tocilizumab in patients with covid- . the drug was supplied free of charge by the drug company, an electronic crf for data entry and drug order was created on the web system of the coordinating centre assuring drug delivery in hours, and a single contract was signed between agenzia italiana del farmaco (aifa) and industry without any administrative acts required at peripheral centres. the fortuitous combinations of all these facilities translated in the opening of centres in less than weeks. aside from the exceptional circumstances of a trial using a drug reducing icu necessity during the emergency, this model suggests that the obligatory implementation of a national contract template, with an agreed line listing activity costs and minimal local negotiations to meet hospital requirements, could dramatically speed up the global process of contracts leading to a faster opening of clinical trials and access for patients. the velocity in trial activation is not an obvious guarantee of trial success in terms of results (actemra/roactemra phase trial in ospitalized patients with covid- associated penumonia (covac-ta) trial is a clear example of this) but undoubtedly contribute to the efficiency of the system and would be beneficial to all interested stakeholders, including industry sponsors, the research sites and the patients who may ultimately benefit from participation in clinical cancer research. . remote regulation audits: regulatory audits sponsored by the authorities are essential to confirm the quality and veracity of clinical data before placing a new molecule or new strategy at the disposal of patients. these on-site audits generally take an enormous amount of preparation time, an attendance time that varies from to full days and mobilises in addition to government personnel, local teams and sponsor teams over many weeks. again, the remote visits and remote controls as set up for certain studies during confinement could be an additional added value in the optimal organisation of tomorrow. . data sharing and generosity: the example of the ter-avolt (thoracic cancers international covid- collaboration) registry. the idea to collect data on mortality and disease outcome in patients with lung cancer affected by covid- was launched in march with a simple email immediately spread among the international community involved in lung cancer treatment, after the chinese warning that the fatality rate in patients with lung cancer was higher than in other tumours. the teravolt registry involved countries worldwide and was endorsed by a number of international oncology societies and physicians who accepted to collect and share data without any form of financial support. in less than a month, data on patients were collected and analysed, and actually, more than patients have been registered in the ecrf. the scientific community realised that, aside from individualism and personal academic glory, the necessity to collect data to take better care of patients with lung cancer was a priority and responded promptly. in the era of big data and learning machines, the generous and altruistic sharing of knowledge and data, could represent an unimaginable step forward and an unprecedented turning-point in the treatment of cancer. . meetings: during the covid- era, the majority of research conferences have been cancelled or postponed on an unparalleled scale, and attendance at confirmed meetings is likely to be lower than expected due to the fear of the disease. basically, all the international congress that took place in the last months as asco, esmo breast, aacr and society of gynecologic oncology (sgo) were performed remotely and represented a great success with more people attendant remotely than on-site, reduced time away from the hospital for the physicians, less cost and carbon footprint. nevertheless, besides the current emphasis on online formats, we must honestly recognise that about half of the global population has no internet connectivity. it is time for the global scientific community to recognise the challenges that our less fortunate colleagues regularly face and capture this exceptional opportunity to build a more equitable global community for the scientists of tomorrow. this pandemic has represented an unparalleled threat for all of us, but also a tremendous opportunity for gaining a new vision in the world of clinical research. covid- has pointed out that sometimes, high level of bureaucracy in research rules place unnecessary burdens on patients and clinicians and it suggests that it is time to alleviate bureaucracy and introduce some practical changes into research organisation that will possibly promote patient access to trials and reduce the costs of the clinical research. nevertheless, it is of utmost importance to underline that bureaucracy alleviation does not mean laxity with dramatic consequences on the quality and consistency of clinical research and a careful balance needs to be maintained between the simplification of the procedures and the reliability of data. moreover, it is time to remember to ourselves that it is patient care, not personal glory, that provides a sense of meaning in our roles, and to reconsider a more generous approach in sharing information with colleagues in order to build a robust scientific community of tomorrow. given that the pandemic and its consequences are unlikely to dissipate soon, the time is arrived to fundamentally rethink study designs and procedures in order to optimise clinical cancer research. moreover, robust adaptations could make the field more resilient to future pandemics the extent to which changes should be implemented will vary by trial type and phase and although it could be easier to maximise translatability to routine practice for a phase iii trial with a standard arm comparator, it would be more challenging for a first in the human study with new class agents. all these relevant changes will require a profound renewal of our tight global structures. however, if we have the will to have all innovative changes in place in a proper time frame and sharing a common vision and mission on research, then we will be creating a new era in clinical research. the who has encouraged all of us to think innovatively and as walter disney said 'if you can dream it, you can do it'. contributors authors provided equal contribution to the conception, writing and final approval of the paper. funding the authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors. competing interests none declared. provenance and peer review commissioned; externally peer reviewed. open access this is an open access article distributed in accordance with the creative commons attribution non commercial (cc by-nc . ) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, any changes made are indicated, and the use is non-commercial. see: http:// creativecommons. org/ licenses/ by-nc/ . /. cancer patients in sars-cov- infection: a nationwide analysis in china managing cancer care during the covid- pandemic: agility and collaboration toward a common goal call for ensuring cancer care continuity during covid- pandemic assessing the impact of the covid- outbreak on the attitudes and practice of italian oncologists toward breast cancer care and related research activities virtually perfect? telemedicine for covid- keep calm and log on: telemedicine for covid- pandemic response conducting clinical research during the covid- pandemic: investigator and participant perspectives interactive telemedicine: effects on professional practice and health care outcomes the effect of telehealth interventions on quality of life of cancer patients: a systematic review and meta-analysis participation in cancer clinical trials: race-, sex-, and age-based disparities at what cost to clinical trial enrollment? a retrospective study of patient travel burden in cancer clinical trials department of commerce the department of health and human services, national telecommunications and information administration lack of reimbursement barrier to telehealth adoption reducing clinical trial monitoring resource allocation and costs through remote access to electronic medical records rethinking cancer clinical trials for covid- and beyond regulatory approvals in a large multinational clinical trial: the esprit experience does the european clinical trials directive really improve clinical trial approval time? resource use, costs, and approval times for planning and preparing a randomized clinical trial before and after the implementation of the new swiss human research legislation challenges with research contract negotiations in community-based cancer research covid- in patients with thoracic malignancies (teravolt): first results of an international, registry-based, cohort study the global academic research organization network: data sharing to cure diseases and enable learning health systems world health organization. who director general's opening remarks at the media briefing on covid- - key: cord- -vlmoa dr authors: mcculloch, peter; sedrakyan, art title: covid- has no effect on gravity date: - - journal: bmj surg interv health technol doi: . /bmjsit- - sha: doc_id: cord_uid: vlmoa dr nan covid- has transformed our world. none of us alive have ever seen anything quite like it in scale, except the very few who still remember world war . like that conflict, the pandemic has elicited responses from governments unparalleled in scope and speed. massive restrictions on people's liberties have been accompanied by massive efforts to prevent the complete collapse of the economy. extraordinary things have happened as a result. right-wing governments have funded workers' wages while they are idle. indians in some cities have discovered that their grandparents were telling the truth when they said you used to be able to see the himalayas. people in atomized cosmopolitan neighborhoods are finding out who their neighbors are, and looking after them. of course medical research has been hugely affected, this being a medical crisis. governments have swept aside sedate procedures which normally add months or years to the life cycle of research projects, and offered huge sums to speed any projects which offer hope of a solution. many researchers too are galvanized by the urgent need to do something and eager to offer their talents to what all deem a worthwhile cause. some of the results have been truly dramatic. the recovery trial of drugs which might ameliorate the effects of the virus went from protocol to first patient enrolled in days, and recruited over patients in weeks, and the oxford vaccine trial launched last week are among the most impressive examples. these achievements required not just money, but the willing cooperation of many people who would not normally have countenanced such haste. but the crisis has, as crises do, brought out the worst as well as the best in us. the remarkable research achievements listed above have been paralleled by a tsunami of proposals and projects whose chance of improving the lives of patients, like their prospects of contributing to the sum of medical knowledge, is near to zero. it would be invidious (and legally risky) to name names, but many proposals we have seen are quite clearly doomed to fail because of obvious flaws in design or in basic logic, or because they are asking a question of absolutely no importance. do we really need detailed studies of the effect of the virus on practice in every conceivable specialty? qualitative research with affected medical personnel has a valuable place, and will help us to understand the responses we have seen to the epidemic, from the heroic to the disgraceful, but some of the proposed studies of reactions to it are not research-journalism, perhaps, history possibly, or in rare cases art, but not science. trials and observational studies whose focus is too poorly defined to make a valid conclusion possible, or which rely on data which clearly will not be possible to collect are exercises in futility. much of this is classic research waste, as was recently pointed out in the bmj. it is also critical that we don't make premature or exaggerated claims. the adaptive covid- treatment trial of remdesivir was highlighted before any details were available even on preprint servers, and inferences made publicly about mortality reduction, although the trial did not show this. on the same day another randomized clinical trial was published showing no trend towards improved survival with remdesivir, and meta-analysis of these two trials shows no difference in mortality outcomes (p value . ) (see figure ). why is this happening? and what should we do about it? while hope for cure is biasing many policy leaders, the lure of easy money is also hard to resist, and scientists are as susceptible as anyone else to a bonanza. this is undoubtedly one major underlying cause of the covid-associated flood of junk science. an opportunity to do something large and impressive is presenting itself to many researchers, because so much goodwill and effort is being put into the drive to do something constructive by researchers, patients, clinicians and even the muchmaligned bureaucrats who run our systems. but bad science will leave its footprint everywhere: in our memory, in public trust and in overcoming fear: it will not help us either in open access figure relative risk of mortality in the trials of remdesivir. the short or the long run. the laws of scientific inference and statistics have not been affected by the virus, and studies whose design guarantees they cannot produce a valid result still will not do so during the crisis. and of course, applying for funding in the full knowledge that it will not be used for the purposes stated, or that it cannot yield the knowledge ostensibly sought, is still unethical. the crisis has shown that the normal processes of peer review and prioritization, both in funding and in publication, can be radically accelerated, but should be robust to protect the conduct of meaningful clinical research. those responsible for managing these activities have a responsibility to ensure that funds are not wasted. many processes already have an executive filter for clearly inappropriate submissions before peer review is even activated, and this needs to be exercised whether or not the terms covid or coronavirus appear in the title or abstract. we hope the research world, like the rest of society, will keep some of the helpful adaptations it has made to cope with the crisis. however, it is also essential that the rigor of the scientific process is maintained if it is to continue to have value and meaning. this is not just the responsibility of the regulators, funders and publishers. the scientific community also needs to reflect and conduct itself according to the high standards of integrity it claims to espouse. twitter art sedrakyan @artsytwits funding the authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors. competing interests none declared. patient consent for publication not required. provenance and peer review commissioned; internally peer reviewed. open access this is an open access article distributed in accordance with the creative commons attribution non commercial (cc by-nc . ) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. see: http:// creativecommons. org/ licenses/ by-nc/ . /. peter mcculloch http:// orcid. org/ - - - recovery trial: the uk covid- study resetting expectations for clinical trials covid- : what do we know so far about a vaccine? mike clarke: how can we avoid research waste during the covid- pandemic and plan for the future available: https:// clinicaltrials. gov/ ct / show/ nct fda will reportedly authorize use of remdesivir for covid- after trial shows 'positive effect' on recovery time remdesivir in adults with severe covid- : a randomised, doubleblind, placebo-controlled, multicentre trial key: cord- -e hdx u authors: li, jie; pavlov, ivan; laffey, john g.; roca, oriol; mirza, sara; perez, yonatan; mcnicholas, bairbre; cosgrave, david; vines, david; tavernier, elsa; ehrmann, stephan title: meta-trial of awake prone positioning with nasal high flow therapy: invitation to join a pandemic collaborative research effort date: - - journal: j crit care doi: . /j.jcrc. . . sha: doc_id: cord_uid: e hdx u • awake prone positioning is a promising intervention to improve patient's oxygenation and likely to reduce the need for intubation. despite reports of its use during the covid- pandemic, controlled evidence is lacking. • a meta-trial, with harmonized inclusion and outcome criteria and a global interim analysis plan, may enable to get a high level of evidence within the short period of time required by the pandemic, through performing a prospective meta-analysis of the generated data across countries. • clinicians and researchers around the world are invited to join the “meta-trial of awake prone positioning with nasal high flow therapy” project by contacting the core investigator group: awake.prone.meta.trial@gmail.com. the ongoing coronavirus disease pandemic (covid- ) poses a great challenge to healthcare systems worldwide. beyond challenges for direct patient care, optimal conduct of research under the specific conditions of the pandemic is a matter of concern and discussion. we present the meta-trial concept as a scientifically, clinically, ethically and socially sound method to carry out optimal clinical research in the setting of a pandemic. we take the example of such a meta-trial, set up to investigate prone positioning among awake patients undergoing nasal high flow therapy and invite journal readers to join this collaborative research effort. the meta-trial concept enables researchers to combine the agility of smaller national trials into a much larger international project in a short period of time (table ) . meta-trial interim analysis enables to detect a positive or negative response to the scientific question as soon as an adequate sample size is reached across several j o u r n a l p r e -p r o o f countries, thus potentially speeding up the research process dramatically ( , ). adherence to methodological standards of individual trials represents a guarantee of a high level of overall final quality. furthermore, by estimating the treatment effect across the various trials upfront, the meta-trial may provide stronger evidence in favor of external validity and replicability of the individual trials. to the best of our knowledge, the meta-trial concept has never been experienced in real life across several countries, and feasibility uncertainties do exist. the present project may serve as a guidance for future research projects set up in a pandemic context. j o u r n a l p r e -p r o o f  awake prone positioning is a promising intervention to improve patient's oxygenation and likely to reduce the need for intubation. despite reports of its use during the covid- pandemic, controlled evidence is lacking.  a meta-trial, with harmonized inclusion and outcome criteria and a global interim analysis plan, may enable to get a high level of evidence within the short period of time required by the pandemic, through performing a prospective meta-analysis of the generated data across countries.  clinicians and researchers around the world are invited to join the "meta-trial of awake prone positioning with nasal high flow therapy" project by contacting the core investigator group: awake.prone.meta.trial@gmail.com j o u r n a l p r e -p r o o f intensive care management of coronarvirus disease (covid- ): challenges and recommendations patient eligibility for randomized controlled trials in critical care medicine: an international two-center observational study prone position for acute respiratory distress syndrome. a systematic review and meta-analysis finding alternatives to the dogma of power based sample size calculation: is a fixed sample size prospective metaexperiment a potential alternative clinical research networks are key to accurate and timely assessment of pandemic clinical severity key: cord- -cesemos authors: block, keith i. title: integrative cancer therapies: learning from covid- date: - - journal: integr cancer ther doi: . / sha: doc_id: cord_uid: cesemos nan integrative therapies have been associated with the coronavirus pandemic since shortly after it began. traditional chinese medicine was used with covid- (coronavirus diseaes- ) patients at the start of the pandemic and featured in the chinese national plan. in this journal, yan et al discussed the use of traditional chinese medicine therapies with cancer patients afflicted with covid- . they suggested the use of tai chi and qi gong as indoor exercise during times when the public is restricted to home to help strengthen the constitution. they also raised an alarm about therapies like massage and acupuncture that involve close contact with patients, and suggest they be rigidly considered or forbidden for cancer patients in endemic areas, with stronger personal protection provisions for both patients and therapists. their caution was prescient. since covid- is now widespread throughout the globe, integrative and conventional medical practices alike have restricted or eliminated "high-touch" therapies that involve close personal contact, ranging from dentistry to surgery, not to mention massage and acupuncture. we hear reports of integrative therapy departments in cancer-focused hospitals being shut down, with widespread halts to ongoing research. for health professionals in "high-touch" occupations practicing in areas with high covid- presence, the economic devastation is palpable, with closed private clinics, and layoffs in hospitals. this is raising serious concerns of how covid- may shape the future of integrative cancer therapies. cancer patients are reported to have severe outcomes in covid- . medical practices are already instituting ways to reduce personal contact through, for example, expanded telehealth visits. how will medical administrators view some of the staples of integrative cancer therapies in the next few years? reiki, yoga classes, meditation classes, acupuncture, art therapy, and similar interventions may be considered hazardous to immunocompromised cancer patients, and thus medically contraindicated. furthermore, the cost to health care systems from the covid- pandemic is astounding. recent projections show a cost to the us health care system of $ billion if % of the us population is infected with covid- or $ . billion in direct costs if only % of the population is infected. the loss of revenue from delayed or cancelled elective procedures compounds the hospitals' financial plight. will administrators be eager to restart programs that rely on inperson services for high-risk patients? we are especially concerned about a possible imminent contraction or restructuring of in-person hospital-based integrative care for cancer patients. integrative cancer care may need to rely more on services that can be delivered by telehealth in the future, such as consultations with integrative physicians and remotely delivered mindfulness or exercise therapies. in-person yoga and meditation classes, acupuncture, and other in-person therapies may eventually reappear in smaller private consultation settings that cancer patients may access on their own well after the end of treatment, as their risk levels decrease. but, to the extent that integrative care for cancer patients does reemerge in the conventional care setting, there are areas for which telehealth may be quite effective. not only has covid- suddenly converted us to a reliance on telehealth that is likely to persist in the future, it has also highlighted the use of some integrative therapies commonly used by cancer patients that have previously been thought to be too controversial for conventional clinics, but that might bear further research attention. specifically, high-dose intravenous vitamin c has come to the attention of the conventional medical community as a potential covid- therapy. in the remainder of this editorial, we will discuss both proposed areas in which telehealth may be particularly effective for integrative care, and the potential emergence of the "alternative" intravenous therapies. one of the lessons of covid- is that we must pay attention to the underlying health of patients. comorbidities, among them diabetes, hypertension, cardiovascular or cerebrovascular disease, and chronic obstructive pulmonary disease as well as cancer are widely known to increase the severity of covid- . the growing understanding of the biology of covid- reminds us of the importance of understanding the biology of cancer, especially from an integrative treatment approach. and indeed, some of the biology driving covid- overlaps with processes driving cancer: inflammation, effects of reactive oxygen species, immunity, and even deficiencies in vitamin d. overlapping comorbidities and biology both have significant effects on cancer and covid- outcomes. diabetes, for instance, influences the course of both covid- and cancer, increasing both mortality and morbidity. diabetic patients in a large clinical trial in metastatic colorectal cancer, for instance, had an overall median survival of . months, while nondiabetics survived . months (p < . ). markers of systemic inflammatory response, interleukin- (il- ) and c-reactive protein (crp), both were related to overall survival in metastatic colorectal cancer. and adequate vitamin d levels predicted a % reduction in cancer mortality (risk ratio = . , p = . ) in a meta-analysis of randomized trials. the overall influence of comorbidities and dysfunctional biology on cancer outcomes is crystallized in the measurement of performance status. performance status is commonly assessed in oncology trials and clinical treatment, and widely known to be associated with treatment response, tolerance, and survival. for instance, meta-analyses of randomized trials of chemotherapy in colorectal cancer patients found that performance status predicted mortality, [ ] [ ] [ ] in addition to treatment side effects. performance status integrates multiple aspects of a patient's health. improvement of performance status will contribute to both improved daily well-being and better response to conventional treatment, major goals of integrative cancer therapy. comprehensive integrative treatment approaches that aim at multiple targets are ideally suited to improvement of performance status and can, we believe, form the foundation of interventions that can be delivered by integrative physicians and other practitioners using telehealth platforms. such multitargeted interventions have been described in the literature, [ ] [ ] [ ] [ ] and one experimental study of such a system published in this journal even included remote delivery of lifestyle counseling using facetime. while in-person counseling has significant advantages, the safety and potential economy of telehealth consultations with integrative physicians, dietitians, and specialists in psychosocial oncology may allow for very relevant interventions, particularly for patients in treatment. referrals to in-person therapies can be made for lower-risk patients, perhaps at locations outside the treating hospital if necessary. webinar-style presentations can also offer patients helpful information. lifestyle interventions to lower inflammation and inflammatory cytokines can be examined as an example of one facet of a multi-targeted integrative program. lifestyle interventions that can be delivered by telehealth include nutrition and supplement counseling, exercise direction, and psychosocial oncology interventions. a dietary intervention that routinely brings about changes in multiple cytokines and overall inflammation is weight loss. in breast cancer survivors, positive correlations were observed of body mass index and body fat with crp, il- , il- , and tumor necrosis factor-α (tnf-α). plant-based diets in other populations were correlated with reduced il- , crp, and sicam (soluble intercellular adhesion molecule), though not tnf-α. healthful diets may affect recurrence or survival, so such changes are of clinical relevance. a review of randomized trials in breast cancer found that low-fat diets were associated with better survival, and a diet rich in phytoestrogens with reduced risk of recurrence. furthermore, a recently published -year follow-up of the women's health initiative trial found that among participants who suffered from breast cancer, those who followed the low-fat diet had a reduced incidence of breast cancer-related death. other lifestyle changes to reduce inflammation and inflammatory cytokines that can be directed or imparted by telehealth include supplements, exercise, and mind-body medicine. according to a meta-analysis, fish oil supplements reduce il- in surgical patients, and crp in chemotherapy patients. randomized trials also showed maintenance of crp in treatment-naïve breast cancer patients versus un-supplemented controls, and improvement in crp as well as survival in fish oil-supplemented hematological malignancy patients. , a trial of walking and resistance exercise in chemotherapy patients observed a shift to an anti-inflammatory cytokine profile, related to reductions in interferon-γ. finally, a randomized trial of mindfulness-based stress reduction reduced il- as well as cortisol in the experimental group. integrative oncology practitioners may not be aware of the increasing interest of intravenous vitamin c in treatment of patients with sepsis. sepsis causes notable vitamin c deficiencies, and the intravenous administration of elevated doses helps overcome the rampant oxidative stress and inflammation observed in hospitalized septic patients. multiple trials have demonstrated safety, and recent studies suggest promising results on mortality. high-dose intravenous vitamin c has also been used in acute respiratory distress syndrome, and promising data in this setting suggest that it may be effective in covid- . three clinical trials of intravenous vitamin c in hospitalized patients with severe covid- are listed in clinicaltrials. gov as of early may (nct , nct , nct ). many of us are used to seeing intravenous vitamin c treated as a scandalously alternative cancer treatment, and its emergence in the intensive care units of hospitals with the goal of treating oxidative stress and inflammation is both surprising and heartening. along with the previously published beneficial effects of parenteral fish oil emulsions in cancer patients, these vitamin c trials raise the question of the potentials of other unconventional intravenous treatments in cancer patients. injectable administration of traditional herbal formulas has been routine in chinese medicine for years, shown by multiple meta-analyses, as has the use of injectable viscum album in europe. as is the case with intravenous vitamin c in cancer, the benefits of these alternative therapies are still somewhat in question due to inferior quality of clinical trial design. however, the apparent safety shown in scientific studies, together with the increased awareness of potential benefits of vitamin c in a completely conventional hospital setting, raises the profile of injectable herbal therapies. three other intravenous therapies based on phytochemicals are coming into use in north america and elsewhere, and we propose to briefly examine the reasons for the interest in these therapies. they are intravenous curcumin, quercetin, and resveratrol. it is important to note that these therapies, as well as intravenous vitamin c, should only be given in facilities equipped with appropriate emergency supplies, including respiratory support for patients, because of the infrequent but real potential for anaphylactic reactions. chemotherapy drugs can cause also anaphylactic shock, so oncology units are among the facilities that routinely contain such emergency supplies. interest in intravenous administration of these phytochemicals came about because of their poor oral bioavailability and the need for higher blood levels of these compounds for effective treatment. although curcumin has been used in numerous clinical studies that show some promise in treatment of cancer, its poor oral availability, and the need to take very large oral doses, is well known. interestingly, a publication reported a case of adenoid cystic carcinoma (acc) of the salivary gland treated with intravenous curcumin alongside imatinib. acc is a generally chemoresistant tumor, and the patient in question had metastatic disease that did not respond to cisplatin and etoposide. imatinib, a monoclonal antibody therapy for tumors expressing c-kit, was not found to be typically effective in acc tumors. however, the patient's tumor expressed nuclear factor κb (nf-κb) as well as c-kit, and the patient's physicians thus chose to treat with imatinib and intravenous curcumin (an inhibitor of nf-κb). at months after starting treatment, a near complete response to the combination was observed, with no adverse reactions. the authors report that their clinical experience with more than prior curcumin infusions also suggests safety. a phase i study of an intravenous liposomal curcumin formulation, specifically designed to overcome bioavailability concerns, reported safe dosing up to mg/m in patients. in this study, patient with metastatic prostate cancer had a temporary decrease in prostate-specific antigen from to ng/ml, stable disease on scans, a reduction of lactate dehydrogenase to normal range, and improved performance status. a patient with metastatic colorectal cancer had a temporary decline in carcinoembryonic antigen from to µg/ml from the intravenous curcumin alone. while these are promising results, further clinical trials are clearly called for. animal studies of human xenograft tumors, however, also provide some interesting perspectives on possible lines of investigation. curcumin retarded growth of mcf- and mda-mb- breast cancer xenografts in mice ; a curcumin preparation with increased bioavailability inhibited esophageal squamous cell cancer xenografts in mice ; and a turmeric extract, including turmerones to promote curcumin uptake, inhibited colon tumor xenografts, and retarded chemotherapy-related immunosuppression in mice. previous reviews of curcumin have indicated it may improve the activities of various chemotherapy drugs; recent studies highlight effects with -fu (fluorouracil) in gastric cancer cells, paclitaxel in breast cancer cells, and cisplatin in lung cancer cells. a phase i clinical trial of intravenous quercetin found that a safe dose could be given weekly or in -week intervals, although some studies have raised questions about how long intravenous quercetin remains in the body. , an aim of the phase i study was to determine whether quercetin could reduce tyrosine kinase activation; tyrosine kinase inhibitors are an important class of anticancer drugs. such inhibition was observed in lymphocytes for up to hours following dosing. a patient with metastatic hepatocellular carcinoma treated at the lowest dose level used in the trial had a sustained fall in serum α-feto protein tumor marker and alkaline phosphatase. a metastatic ovarian cancer patient was treated with quercetin while receiving carboplatin after previous treatment with other chemotherapy regimens. this patient's ca fell from to units/ml after months of treatment. intravenous quercetin was also given with a stabilizing agent once daily for days at a dose of . g at the start of medical tuberculosis treatment, and showed no adverse effects while hastening symptomatic recovery and radiological resolution of lung damage. these studies suggest safety and potential efficacy that could be explored in further trials. studies of quercetin administered to human tumor xenografts in mice have observed, for instance, inhibition of implanted prostate cancer growth and sensitization to docetaxel, along with reduction of ki expression beyond that observed with docetaxel treatment ; reduction of cancer stem cells and expression of the signaling protein notch- in colon cancer xenografts during radiation treatment ; and inhibition of the growth of lung cancer xenografts. quercetin has other potentials for combination with medical cancer treatments. it increased the effectiveness of doxorubicin and etoposide in lymphoid leukemia cell lines, and the effectiveness of doxorubicin in myeloid leukemia lines, and improved the activity of cisplatin in nasopharyngeal cancer cells. intravenous administration of resveratrol at a very low dose ( . mg) was compared with an oral dose of mg (also a low dose) in a human trial. the intravenous dose raised blood levels quickly although the resveratrol was then absorbed by tissues and metabolized (some metabolites of resveratrol may be active compounds). levels in the blood declined quickly after the first hours and more slowly thereafter. trials of intravenous doses of resveratrol that might be more clinically relevant, and that are being used in clinics, are not available. safety concerns have not arisen in animal studies. resveratrol was noted to inhibit lung metastasis in a breast cancer xenograft in mice, and to increase the antitumor effects of radiation and cisplatin on head and neck cancer xenografts. it also reduced glycolysis, liver metastasis, and tumor growth in ovarian cancer xenografts. a recent review of resveratrol found more than laboratory studies indicating chemoprotective effects or synergistic effects of resveratrol with cancer chemotherapy. it is also notable that all phytochemicals affect multiple molecular targets in their mechanisms of action. curcumin and resveratrol were both noted to affect multiple hallmarks of cancer in a comprehensive review. some of the targets that these agents affect are also targets of conventional drug treatment, such as the androgen receptor (curcumin, resveratrol, quercetin), [ ] [ ] [ ] her- (curcumin, quercetin), , kras (quercetin), and vascular endothelial growth factor (curcumin, resveratrol, quercetin). , while targets such as androgen receptor, her- , kras, and vascular endothelial growth factor can be managed by conventional treatment, these phytochemicals also each affect a number of other important molecular targets in cancer that are not directly affected by conventional drugs, but may nevertheless be relevant for cancer control. for example, curcumin is well known as an inhibitor of nf-κb, and also inhibits hypoxiainducible factor α (hif- α), activator protein- , epidermal growth factor receptor, and matrix metalloproteinases and . resveratrol also inhibits nf-κb, phosphatidylinositol -kinase, sirtuins, telomerase, and mitogen-activated protein kinase. quercetin inhibits janus kinase-signal transducer and activator of transcription, pi k, protein kinase b (akt), p , and cyclooxygenase- . each of these also modulates many other molecular targets, suggesting the potential for systemic effects. the coronavirus pandemic will change the world in many ways, and is likely to bring a significant rethinking of many aspects of medical practice. the use of telehealth may eventually overcome barriers many patients face in accessing different aspects of integrative cancer therapies, such as travel to treatment centers for exercise or meditation classes at times when they are fatigued from their treatment or disease. telehealth can make consultations with integrative practitioners more accessible to a geographically scattered population-an important fact, since there are as yet relatively few integratively trained health professionals. covid- may make us reconsider general approaches to treatment that emphasize patients' biology and performance status, and may open the door for research on new integrative therapeutics. but substantial research needs to go into the practicality and efficacy of all of these. we hope to see such research begin to flourish again as the coronavirus crisis passes. the author declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: keith i. block is the owner and medical director of the block center for integrative cancer treatment, skokie, il, usa. the author received no financial support for the research, authorship, and/or publication of this article. effectiveness of integrative medicine in covid- challenges and countermeasures of integrative cancer therapy in the epidemic of covid- more than in virginia health care industry laid off amid pandemic clinical characteristics and outcomes of cancer patients with covid- the potential health care costs and resource use associated with covid- in the united states. health aff (millwood). ; hlth aff a randomized controlled trial of mhealth mindfulness intervention for cancer patients and informal cancer caregivers: a feasibility study within an integrated health care delivery system editorial: low population mortality from covid- in countries south of latitude degrees north supports vitamin d as a factor determining severity diabetes: an overview for clinical oncologists diabetes and clinical outcome in patients with metastatic colorectal cancer: calgb (alliance) interleukin- and c-reactive protein as prognostic biomarkers in metastatic colorectal cancer the role of vitamin d supplementation for primary prevention of cancer: meta-analysis of randomized controlled trials pooled safety and efficacy analysis examining the effect of performance status on outcomes in nine first-line treatment trials using individual data from patients with metastatic colorectal cancer determinants of early mortality among , patients with colon cancer who participated in clinical trials from the adjuvant colon cancer endpoints database personalizing survival predictions in advanced colorectal cancer: the arcad nomogram project designing a broadspectrum integrative approach for cancer prevention and treatment integrative treatment for colorectal cancer: a comprehensive approach a comprehensive lifestyle randomized clinical trial: design and initial patient experience effect of comprehensive lifestyle changes on telomerase activity and telomere length in men with biopsy-proven low-risk prostate cancer: -year follow-up of a descriptive pilot study an anti-inflammatory dietary intervention to reduce breast cancer recurrence risk: study design and baseline data effect of plant-based diets on obesity-related inflammatory profiles: a systematic review and meta-analysis of intervention trials impact of nutrition on breast cancer mortality and risk of recurrence, a review of the evidence dietary modification and breast cancer mortality: long-term followup of the women's health initiative randomized trial a systematic review and meta-analysis of the n- polyunsaturated fatty acids effects on inflammatory markers in colorectal cancer the effects of epa and dha enriched fish oil on nutritional and immunological markers of treatment naïve breast cancer patients: a randomized double-blind controlled trial oral fish oil positively influences nutritional-inflammatory risk in patients with haematological malignancies during chemotherapy with an impact on long-term survival: a randomised clinical trial effects of exercise on inflammation in patients receiving chemotherapy: a nationwide ncorp randomized clinical trial a large randomized trial: effects of mindfulness-based stress reduction (mbsr) for breast cancer (bc) survivors on salivary cortisol and il- the emerging role of vitamin c as a treatment for sepsis intravenous vitamin c for reduction of cytokines storm in acute respiratory distress syndrome the effect of supplementary omegaven® on the clinical outcome of patients with advanced esophagogastric adenocarcinoma receiving palliative epirubicin, oxaliplatin, and capecitabine chemotherapy: a phase ii clinical trial shenqi fuzheng injection in the treatment of breast cancer: a meta-analysis of randomized controlled trials safety of combined treatment with monoclonal antibodies and viscum album l preparations the effect of vitamin c (ascorbic acid) in the treatment of patients with cancer: a systematic review the therapeutic potential of curcumin: a review of clinical trials successful treatment of c-kit-positive metastatic adenoid cystic carcinoma (acc) with a combination of curcumin plus imatinib: a case report a phase dose-escalation study on the safety, tolerability and activity of liposomal curcumin (lipocurc™) in patients with locally advanced or metastatic cancer curcumin inhibits the tumorigenesis of breast cancer by blocking tafazzin/yesassociated protein axis combination treatment with highly bioavailable curcumin and nqo inhibitor exhibits potent antitumor effects on esophageal squamous cell carcinoma turmeric extract, with absorbable curcumin, has potent anti-metastatic effect in vitro and in vivo curcumin enhances the anticancer effect of -fluorouracil against gastric cancer through down-regulation of cox- and nf-κb signaling pathways paclitaxel and curcumin coadministration in novel cationic pegylated niosomal formulations exhibit enhanced synergistic antitumor efficacy curcumin reverses cisplatin resistance in cisplatin-resistant lung caner cells by inhibiting fa/brca pathway phase i clinical trial of the flavonoid quercetin: pharmacokinetics and evidence for in vivo tyrosine kinase inhibition pharmacokinetics and bioavailability of the flavonol quercetin in humans efficacy and safety of quercetin and polyvinylpyrrolidone in treatment of patients with newly diagnosed destructive pulmonary tuberculosis in comparison with standard antimycobacterial therapy quercetin reverses docetaxel resistance in prostate cancer via androgen receptor and pi k/ akt signaling pathways quercetin pretreatment enhances the radiosensitivity of colon cancer cells by targeting notch- pathway quercetin is the active component of yang-yin-qing-fei-tang to induce apoptosis in non-small cell lung cancer quercetin in cancer treatment, alone or in combination with conventional therapeutics polyphenols act synergistically with doxorubicin and etoposide in leukaemia cell lines quercetin-induced inhibition and synergistic activity with cisplatin-a chemotherapeutic strategy for nasopharyngeal carcinoma cells high absorption but very low bioavailability of oral resveratrol in humans resveratrol inhibits the migration and metastasis of mda-mb- human breast cancer by reversing tgf-β -induced epithelial-mesenchymal transition resveratrol-induced reg iii expression enhances chemo-and radiosensitivity in head and neck cancer in xenograft mice resveratrol inhibits the proliferation and induces the apoptosis in ovarian cancer cells via inhibiting glycolysis and targeting ampk/ mtor signaling pathway a review of resveratrol as a potent chemoprotective and synergistic agent in cancer chemotherapy mechanism of anti-cancer activity of curcumin on androgen-dependent and androgen-independent prostate cancer resveratrol inhibits dht-induced progression of prostate cancer cell line through interfering with the ar and cxcr pathway quercetin targets hnrnpa to overcome enzalutamide resistance in prostate cancer cells curcumin and its analogues (pgv- and pgv- ) enhance sensitivity of resistant mcf- cells to doxorubicin through inhibition of her and nf-kb activation molecular targets underlying the anticancer effects of quercetin: an update quercetin and ursolic acid are potent inhibitors of proliferation and inducers of apoptosis in both kras and braf mutated human colorectal cancer cells key: cord- -khvcoqti authors: scott, ian a. title: covid‐ pandemic and the tension between the need to act and the need to know date: - - journal: intern med j doi: . /imj. sha: doc_id: cord_uid: khvcoqti nan pandemics associated with a deadly and contagious virus, such as covid- , generate a giant thirst for instant knowledge in how to treat the disease. the imperative becomes doing something, anything, that we think may help. this creates tension between exploiting whatever actionable clues we can gleam from observations and anecdotes versus conducting proper clinical trialsthe quick and dirty versus the slow and rigorous. can we increase the tempo of discovery while staying faithful to scientific method? in identifying effective therapies, there are already hundreds of covid- trials registered worldwide, with numbers increasing daily. this is a positivethe rapid mobilisation of trialists around the world, many suspending their prior research programmes to focus on covid- and enlisting the support of multiple funders. but with so many different trials, there is also the potential for wasteful duplication, competition for limited resources, including recruitable patients, and reporting of poorly designed and underpowered studies that result in premature rejection of promising drugs or premature adoption of ineffective ones as standards of care. the false promise of rushed science a pandemic as serious as covid- will compel some clinicians and patients to try unproven therapies based on theory, in vitro data, animal models, clinical anecdotes, observational studies and uncontrolled trials that may later be shown to be misleading. this was the case during the novel influenza a (h n ) viral pandemic in , when low-quality and incomplete data suggested oseltamivir was potentially beneficial in preventing complications and deaths among patients hospitalised with influenza. as a result, countries stockpiled and used the drug extensively and at great expense, but to this day, there has been no high-quality randomised controlled trial (rct) to confirm definitively the efficacy of oseltamivir. numerous studies have already been reported during the current pandemic, many as pre-prints, in an effort to disseminate results quickly, but which are yet to undergo peer review. others have undergone fast-track review, which may be less stringent or probing than the usual process, which can take many weeks. studies reporting promising results have featured cohort studies, case series and even single case reports that, at any other time, would have been regarded as no more than hypothesis-generating on account of their high risk of bias due to their observational design. instead, some now receive considerable publicity as 'treatments' from media outlets, helped along in some instances by endorsement as 'game changers' by high-profile celebrities and politicians. medications lacking any approval for any indication are also being widely used outside of clinical trial protocols. unproven therapies have become accepted as new standards of care, thus extinguishing the equipoise needed to undertake rct. as an example, concerned by reports of increased incidence of thrombosis and disseminated intravascular coagulation in seriously ill covid- patients, , the results of a very low-quality observational study of the use of anticoagulants (ac) , led one us hospital system to protocolise the use of highdose ac in patients with covid- admitted to intensive care (box ). while more rigorous propensity-matched observational studies comparing one treatment group to another lend useful insights, the best adjustment methods can still miss major systematic biases, especially when responding to a rapidly spreading and deadly pandemic. in extreme times, rct too have been flawed by illspecified inclusion criteria, convenience sampling, small samples from single centres with inadequate power, use of surrogate outcome measures (such as reduction in viral shedding, radiological progression, numbers of 'cough days' ) , ad hoc administration of a host of cotreatments (such as steroids, various antiviral drugs and antibiotics), underpowered subgroup analyses, early termination and short in-hospital duration. unsurprisingly, such flaws cause different trials of the same therapy, such as hydroxychloroquine, to report conflicting findings. , too much faith has been prematurely placed on what seemed to be useful therapies, as exemplified by the off-label use of corticosteroids and hydroxy chloroquine, despite evidence of no effect or even possible excess mortality. such publicity also led to inappropriate overprescribing of hydroxychloroquine to treat mild cases and even for prophylaxis, leading to shortages of the drug for patients with proven indications for its use, such as rheumatoid arthritis and systemic lupus erythematosus. hopefully, the same problems will not occur with remdesivir, whichdespite limited and conflicting evidence of clinical improvement from only two placebo-controlled rct , and one non-controlled cohort study has now become a 'standard of care' in the united states for covid- patients with severe pneumonia. the urgency to investigate and report therapeutic trials during a pandemic is not surprising, but the human desire of researchers and journal editors to be the first to report and publish potentially ground-breaking research also needs consideration. many study protocols are being designed in haste by inexperienced researchers, with inadequate attention being paid to both methodological rigour and appropriate informed patient consent. unscrupulous investigators have fabricated or manipulated data , or submitted duplicate articles relating to the same patient populations, which imperils the accuracy of subsequent estimates of therapeutic outcomes and precludes valid meta-analyses if individual patient data cannot be obtained to reveal such duplication. the websites of many mainstream journals now feature covid- resource centres containing pre-proofs and early views of articles that help boost their citation and website hit rates, but without any ranking or labelling of articles according to quality. regulators, such as the us food and drug administration, are pressured to expedite approval of investigational therapies under emergency user access or compassionate use schemes. there are many challenges to conducting rct in a pandemicrandomisation to placebo seems unethical; conventional trials are slow and cumbersome; and conducting multi-site trials that require participating researchers to agree on inclusion and diagnostic criteria, drug administration schedules and outcome measures is particularly challenging. this multiplicity of study designs also poses difficulties for meta-analysts trying to render estimates of therapeutic effects more precise and certain. so how can rct be rendered more doable and informative during a pandemic? first, trialists should wait until sufficient data have been gathered to allow a better understanding of the disease process and its natural history and then use this to design more definitive trials. based on the assumption that sars-cov- was a coronavirus similar to those that caused severe acute respiratory syndrome (sars) and middle east respiratory syndrome (mers), drugs that had shown antiviral effects on the latter, at least in vitro, were quickly repurposed to treat the former, and their effects were assessed in observational studies. it has now become clear that, while sharing many traits, sars-cov- is genetically different to its predecessors, as are the varied clinical manifestations of infection, such as profound hypoxia, extensive inflammatory activation, endothelial damage and coagulopathies. these observations suggest multiple pathogenic pathways activated to varying degrees across different individuals at different times. second, this complex pathogenesis means that no single drug is likely to be effective, and even for multiple drugs, significant heterogeneity of treatment effects is likely among individuals and at various stages of the disease. separating out these effects, and determining which drugs to use and when (early in the disease course or only at deterioration), will likely require large-scale trials with multiple treatment arms that are sufficiently powered to enable analyses of primary and, where indicated, secondary outcomes across different patient subgroups. conventional multi-arm trials take time and expose patients to ineffective or even harmful treatments for the duration of the trial in the absence of frequent interim analyses. weighing the pros and cons of multiple competing protocols also wastes time and resources. it would be more efficient to use designs that leverage a common structure for trial entry, data collection and testing of multiple therapies. adaptive study designs and platform protocols allow rapid cycle testing of different therapies using response-adaptive randomisation (box ). , such trials take account of the heterogeneity of the patient population (e.g. based on age or other patient characteristics, illness severity, prognostic factors) and aim to find, in the shortest time possible, the best treatment, or combination of treatments, for patient subgroups defined by these variables. what may take months for traditional two-arm sequential trials to produce an answer could be reduced to months using adaptive designs. such designs underpin several ongoing covid- trials, such as australasian covid- trial (ascot), adaptive covid- treatment trial (actt), world health organization solidarity, randomised embedded multifactorial adaptive platform trial for community-acquired pneumonia and randomised evaluation of covid- therapy (recovery). third, trials must strive to be multi-site and for investigators to share individual patient data from similar trials within multi-trialist collaborations. these groups need to share a common taxonomy of outcome measures for which the minimal clinically important difference between box how a flawed observational study engenders a new but unproven standard of care researchers from mount sinai hospital in new york city reported a retrospective observational study of patients with covid- hospitalised in march and april . they found that, among patients prescribed therapeutic doses of anticoagulation (ac), the in-hospital mortality was . %, with a median survival of days, compared with . % and a lower median survival of days among patients who did not receive this treatment. however, among patients who required mechanical ventilation (n = ), inhospital mortality was %, with a median survival of days, for ac patients but %, with median survival of days, in non-ac patients. the adjusted hazard ratio for mortality was . per day ( % confidence interval . - . , p < . ), while bleeding events were similar in both groups: % versus %. the authors acknowledged their study was limited by: indication bias for ac, with no reporting of why ac was commenced in some patients but not others at varying times throughout the hospitalisation; non-standardised use and dosing of oral, subcutaneous or intravenous ac; subgroup analysis with a lack of metrics to further classify illness severity in the mechanically ventilated patients; absence of data regarding the precise cause of death (coagulopathy-related or otherwise); and other unobserved confounders. the median duration of ac was only days, which makes such a large decrease in mortality from such a short exposure to the drug among ventilated patients implausible. the authors also omitted mentioning immortal time bias. looking at the survival curves of both groups, at day , about % of the patients in the non-ac group had died, but nearly all the ac patients were still alive. but to receive ac at day , the patient had to survive, or be 'immortal', to that point in time, with credit being given to ac for those days of survival. in contrast, the non-ac arm includes all the patients who did not live long enough to receive ac and who could have died any time during their hospitalisation, including on day , and are thus considered 'mortal.' in a similar fashion, any other intervention given to a patient who survived to day , such as a garlic necklace, could be given credit for preventing death up to that point in time. despite these multiple flaws, the authors concluded that systemic ac may be associated with improved survival after adjusting for mechanical ventilation. the second author of the paper also happened to be the editor-in-chief of the journal in which the article was published, thus raising concerns about the rigour of the peer review process. this author also spoke to the media extolling the virtues of ac in all covid- patients admitted to intensive care and announced that the mount sinai hospital system had changed its protocols to begin giving such patients therapeutic doses of ac. multiple commentators took to twitter exposing the flaws of the study within hours of publication, emphasising that observational studies are prone to bias, often report over-inflated effect sizes andif adopted as new standards of careimpede the ability to mount robust rct capable of providing more definitive results. flawed data can be worse than no data, and observational studies should not be used to establish a new normal. internal medicine journal ( ) - treatment groups, from a patient's perspective, has been determined. this would allow for additional analyses, even if the analyses of the combined data were not preplanned, and would be considered exploratory. the goal is to expand what is known about possible treatments so that future trials can be improved using the adaptive designs already mentioned. fourth, funding agencies responsible to taxpayers need the political cover and authority to support international studies; pharmaceutical companies need support and incentives from regulatory authorities to participate in collaborative trials; and academic investigators need a structure that provides academic credit and incentive to collaborate in efforts where they might otherwise perceive anonymity and loss of control and intellectual property. an example of such multi-sectoral collaboration is the recently announced accelerating covid- therapeutic interventions and vaccines (activ) partnership in the united states whose industry partners agree to support the prioritisation of therapeutic and vaccine candidates, no matter who has developed them, and to share their respective clinical trial capacities, irrespective of the agent to be studied. for their part, the public partners led by the national institutes of health resolve to streamline research and regulatory issues to drive expedited evaluation and rapid scale-up and manufacturing of candidate therapies with predicted successful outcomes. finally, every clinician involved in managing covid- patients must commit to acknowledging equipoise and box rapidly cycling trial designs adaptive randomisation adaptive trials allow pre-specified changes in key trial characteristics while it is being conducted in response to information accumulating during the trial. adaptive randomisation (ar) allows changes to be made to the probabilities of participants being randomised to one treatment (or treatment combination) versus another during the trial with the aim of allocating a greater proportion of patients to treatments that are demonstrating evidence of better performance than others. this evidence of better performance can comprise information from the trial itself, evidence emanating from other trials and expert opinion from multiple groups or societies. bayesian statistical methods are used to continually update trials with new information as it becomes available while at the same time maintaining trial integrity, thus allowing trials to 'learn as they go'. this level of flexibility is difficult with classical, non-bayesian approaches that have a less informative focus on what 'works' or 'doesn't work' according to a statistical test. the bayesian approach is to define and recursively update the probability that a treatment works based on combining information more naturally, better resembling how clinicians think in the real world. non-bayesian trials struggle to confirm whether a treatment works under uncertainty because the sample size and design features of the trial rely on assumptions about how the treatment will work. the trial design cannot be modified easily, so if those assumptions, including sample size calculations, are ultimately incorrect, the trial may finish without providing any useful evidence about what treatments are effective. a bayesian adaptive trial can swiftly and more efficiently learn about existing treatments, abandon any that prove futile and expand to include new and promising candidates. it has all the advantages of classic group sequential designs but can also alter maximum sample size, switch end-point from non-inferiority to superiority, alter number and spacing of interim analyses, investigate a larger dose range in order to select effective doses, incorporate biomarkers that may predict differential treatment response and proceed to completion with increased enrolment and resolution of responses in all enrolled patients, instead of being terminated early with risk of compromise from unknown or unadjudicated responses. platform protocols platform protocols facilitate the study of multiple targeted therapies in a perpetual manner, with therapies allowed to enter or leave the platform on the basis of a decision algorithm or stopping rule. the platform trial is ongoing over time, with no fixed finish date, and is governed by a master protocol that envisions adding and dropping strata. at trial start, entering patients are assigned to different strata (potentially on the basis of illness severity, such as severe (a), moderate (b) or mild (c) covid- disease). strata a patients are then randomly assigned to one of three groups, testing two investigational drugs (drugs and ) against placebo. when investigational drug meets the pre-specified criteria for success (based on, in many instances, the bayesian likelihood of a treatment benefit), drug replaces the placebo group as the control. from this point, newly recruited patients are randomised to another investigational drug (drug x), and the new control group becomes drug , while recruitment of patients into the previous protocol comparing drugs and completes enrolment and is ceased. in a similar manner, strata b patients may also be randomised to drugs and or placebo or to different drugs (drugs and ) or placebo. in a similar manner to strata a, whichever drug shows superiority in strata b then becomes the control group for newly recruited patients into that strata once patient enrolment is completed for the first protocol. the same process applies to strata c. the design can also accommodate comparisons of drug combinations (e.g. drugs + vs placebo or drugs and x vs drug ). the statistical methods throughout involve randomised treatment assignment, sharing of common control patients and sequential interim analyses with the possibility of stopping early for futility. internal medicine journal ( ) - seeking definitive evidence of the benefits and harms of any proposed treatment by enrolling patients into multisite rct that are being transparently conducted in communication with global partners. this strategy allows clinicians to be satisfied that they are doing everything possible for their severely ill patients while contributing to new knowledgean approach with which patients also agree. we must avoid the 'just do it' option of administering therapies based on unreliable observational evidence and instead commit to reducing uncertainty by testing therapies in rct as the 'must learn' alternative. it is not a matter of choosing between one or the other. for unproven covid- therapies in the united states severe acute respiratory syndrome coronavirus (sars-cov- ) and coronavirus disease- (covid- ): the epidemic and the challenges prominent changes in blood coagulation of patients with sars-cov- infection incidence of thrombotic complications in critically ill icu patients with covid- association of treatment dose anticoagulation with in-hospital survival among hospitalized patients with covid- immortal time bias in observational studies of drug effects using propensity score methods to create target populations in observational clinical research efficacy of hydroxychloroquine in patients with covid- : results of a randomized clinical trial hydroxychloroquine in patients mainly with mild to moderate covid- : an open-label, randomized, controlled trial treating covid- -off-label drug use, compassionate use, and randomized clinical trials during pandemics clinical evidence does not support corticosteroid treatment for -ncov lung injury outcomes of hydroxychloroquine usage in united states veterans hospitalized with covid- remdesivir in adults with severe covid- : a randomised, double-blind, placebo-controlled, multicentre trial remdesivir for the treatment of covid- -preliminary report ethics committee reviews of applications for research studies at hospital in china during the novel coronavirus epidemic retraction-hydroxychloroquine or chloroquine with or without a macrolide for treatment of covid- : a multinational registry analysis retraction: cardiovascular disease, drug therapy, and mortality in covid- editorial concern -possible reporting of the same patients with covid- in different reports adaptive designs for clinical trials © royal australasian college of physicians parmigiani g et al. bayesian adaptive randomized trial design for patients with recurrent glioblastoma efficiencies of platform clinical trials: a vision of the future australasian covid- trial (ascot) covid- : trials of four potential treatments to generate 'robust data' of what works australian and new zealand intensive care society. a randomised, embedded, multi-factorial, adaptive platform trial for community-acquired pneumonia (re-map) university of oxford nuffield department of population health. randomised evaluation of covid- therapy (recovery) minimal clinically important difference: defining what really matters to patients accelerating covid- therapeutic interventions and vaccines (activ). an unprecedented partnership for unprecedented times what the public think about participation in medical research during an influenza pandemic: an international crosssectional survey key: cord- - xb hipt authors: rubio-san-simón, a.; verdú-amorós, j.; hladun, r.; juan-ribelles, a.; molero, m.; guerra-garcía, p.; pérez-martínez, a.; castañeda, a.; cañete, a.; de rojas, t.; moreno, l.; bautista, f. title: challenges in early phase clinical trials for childhood cancer during the covid- pandemic: a report from the new agents group of the spanish society of paediatric haematology and oncology (sehop) date: - - journal: clin transl oncol doi: . /s - - - sha: doc_id: cord_uid: xb hipt purpose: the covid- pandemic has forced healthcare stakeholders towards challenging decisions. we analyse the impact of the pandemic on the conduct of phase i–ii trials for paediatric cancer during the first month of state of alarm in spain. methods: a questionnaire was sent to all five itcc-accredited spanish paediatric oncology early phase clinical trial units, including questions about impact on staff activities, recruitment, patient care, supply of investigational products, and legal aspects. results: all units suffered personnel shortages and difficulties in enrolling patients, treatment continuity, or performing trial assessments. monitoring activity was frequently postponed ( %), and % of on-going trials interrupted recruitment. only two patients could be recruited during this period ( % reduction in the expected rate). conclusions: the covid- crisis has significantly impacted clinical research practice and access to innovation for children with cancer. structural and functional changes are under way to better cope with the expected future restrictions. the covid- pandemic has affected more than million people worldwide and , have died in spain as of nd may [ ] and healthcare stakeholders have taken challenging decisions to safeguard public health. in spain, on march th, the government declared the state of alarm [ ] . as a consequence, the spanish healthcare system has suffered unprecedented adjustments to prioritize resources for the fight against covid- [ ] . outpatient follow-up and elective procedures have been cancelled and healthcare providers reallocated to assist patients with covid- . although sars-cov- is suspected to infect children milder than adults [ ] , adjustment of healthcare facilities has been established also for them [ , ] . the countermeasures adopted by the authorities can impact hardly in clinical research, which often is perceived as a "desirable but not essential" activity. actions undertaken can affect the conduct of clinical trials in many ways [e.g., completion of trial assessments, visits, and provision of investigational medicinal products (imps)]. furthermore, pharmaceutical companies may refocus resources into areas related to covid- (e.g., vaccines or antiviral drugs), thus affecting other research areas. this situation forces sponsors, investigators, and patients to adapt. the european medicines agency and the spanish agency of medicines and medical devices have issued guidelines on the management of clinical trial activities [ , ] both are aligned in the intention to guarantee trial activity, patient safety, and action traceability. however, all parties (i.e., healthcare authorities, hospitals, regulators, and sponsors) have implemented measures differently depending on the impact of the pandemic in their regions and their capacity to adapt, although not all coordinated nor evidence-based. cancer research is particularly challenging, because patients are remarkably vulnerable: their baseline condition needs close surveillance and delays in diagnosis or treatment are potentially fatal [ ] ; in parallel, cancer patients are at increased risk for severe covid- disease [ ] [ ] [ ] we evaluated the impact of the covid- pandemic on the early phase clinical trial activity in paediatric oncology during the first month of state of alarm in spain ( th march- th april ). in spain, the innovative therapies for children with cancer consortium (itcc), an international scientific organisation dedicated to early drug development for childhood cancers in centres from european countries, has accredited five paediatric cancer centres to deliver early phase clinical trials [ ] . all five centres were contacted and sent a -item questionnaire between th and th april investigating the impact of the pandemic on actively recruiting phase i-ii clinical trials in solid and haematological paediatric malignancies and on molecular cancer platforms (appendix i). descriptive statistics of the survey are provided. all units suffered shortages on on-site staff in all professional strata ( table ). the median decrease in personnel was % (range - ). main causes were: institutional contingency policies ( %), covid- -infected workers ( %), and relocation to other hospital areas ( %). all units managed to perform remote data entry, although two were not fully equipped for and had difficulties to perform their activities. seventy percent ( / ) planned site initiation visits (siv) were postponed and % (n = ) were performed remotely, all as per decision of the sponsor. seventy-three percent ( / ) planned monitoring visits (mv) which were postponed and % (n = ) were performed remotely, mostly due to institutional decision ( %). the median number of studies that stopped recruitment per site was (range - ), a % on average (sd . ) of actively recruiting studies. the decision to stop was taken per sponsor ( %) or per local institutions ( %). one unit interrupted recruitment in all trials (n = ) following institutional decision. during this period, only two new patients were enrolled across the whole of spain, % less than the monthly average in (n = ). all units continued on-site patient care with limitations. two units had restrictions to perform assessments due to local prevention strategies. out of planned patient visits, ( %) had to be rescheduled. most were conducted on-site ( %, ), while % ( ) were conducted by phone and % ( ) in other hospitals closer to family home. only one patient experienced treatment delay (parents' decision). seven patients that were potentially eligible could not be enrolled. of those, four received non-trial treatments and three are on standby. the reasons for not enrolling were: recruitment interrupted by sponsor (n = ) and not considered safe to be included by investigators (n = ) provision of imps and research devices, as well as shipment of research samples, were uniformly warranted. for six trials, two units were able to ship imps to patients' homes or to local hospitals. three units provided larger amounts of trial medications to patients to minimise hospital visits. the completion of three planned contracts was delayed. all units have been provided with contingency plans by all investigators foresee a lower recruitment in compared to . most are planning changes over the next months to promote homeworking and remote sivs/mvs. all investigators believe that this crisis will make units better prepared for future crises. following the analysis of this survey results and the collected expert consensus, we propose a set of recommendations to preserve patient safety, scientific integrity, and research value (fig. ) . this study reveals that paediatric cancer clinical research has been significantly impacted by the covid- pandemic. shortage of personnel has been a major issue in all units, as a consequence of contingency plans restricting on-site work. while physicians and nurses' activities have been severely compromised as their activities are essentially patient-centred, it may have less impact on other professional strata such as study coordinators or data managers, provided that they have appropriate means to perform their work remotely. unfortunately, this was not the case for all institutions. providing homeworking solutions is essential under the perspective of sustained restrictions over the next months. although redeployment of staff to other areas in the fig. set of recommendations to preserve patient safety, scientific integrity and research value fight against covid- is necessary, it has an undeniable impact on areas with highly specialized, multidisciplinary activities, as is clinical research. our study shows drastic suspension/delays of sivs and mvs. delays in siv are understandable decisions because of the inherent complexity of early phase studies, but they need to be rescheduled as soon as possible. however, mvs are key to ensure safety and data quality of on-going trials, and hence, it is necessary that sponsors implement alternative mechanisms to resume it in collaboration with the sites while preserving patient data protection (e.g., remote access to electronic medical records with regular contacts with the study coordinators). all units have experienced restrictions in treatment delivery or ability to conduct trial assessments. nevertheless, all units have implemented strategies to mitigate these risks while delivering treatment to the patients, complying with trial regulations and in agreement with sponsors. the outbreak is expected to increase protocol deviations, highlighting the need for a fluid communication between sites, ethics committees, sponsors, and regulators. regulators should take a sensible approach when reviewing these deviations, especially when the trial participant's best interest has been taken into account and no additional risk has been posed [ ] . in our study, only one patient had his treatment delayed due to family's fears of travelling. while patients and caregivers are advised to restrict hospital visits, they may delay medical consultation when facing adverse events during a trial. cortiula et al. [ ] highlighted the negative implications of excessive focus on covid- and of overshadowing the other aspects of clinical practice, especially in cancer care. only two patients could be recruited in this period, a % reduction in the expected recruitment rate, while seven potentially eligible patients could not be recruited. clinical trials are most of the times the unique way to provide access to new drugs with significant patient benefit (e.g., trk inhibitors for trk-fusion cancers [ ] ), or at the very least, to innovative therapies to children with no curative options. whereas covid- has a low mortality in children [ ] , more than % of children with relapsed and refractory cancers will continue to die and, hence, access to novel therapies needs to be assured. moreover, the stalling process on experimental medicines will extend the already lengthy marketing process and have an impact on the companies' market value [ ] . therefore, long-term consequences of the pandemic on clinical research remain unpredictable. although there are multiple publications about generic emergency preparedness in health settings, there is minimal information focusing on clinical trial sites [ , ] and we provide recommendations on this respect (fig. ) . all consulted investigators conclude that this crisis will make trial units better prepared for future emergencies and they are planning changes in their organization. our results show that the covid- crisis has had a major impact on conducting paediatric oncology research. our conclusions will hopefully help investigators, sponsors, and authorities to address the issues derived from this pandemic and facilitate the construction of a consensus strategy for future crises. this would allow for maintaining highquality care, standardizing procedures, and prioritizing an efficient use of resources, to ensure safe access to therapies in the context of clinical trials. author contributions all authors contributed to the study conception and design. material preparation, data collection, and analysis were performed by alba rubio-san-simón and francisco bautista. the first draft of the manuscript was written by alba rubio-san-simón and francisco bautista, and all authors commented on previous versions of the manuscript. all authors read and approved the final manuscript. funding there was no funding for this study conflicts of interest f bautista had a consultant or advisory role for bayer, amgen, sanofi, and eusapharma, received honoraria for speaking at symposia from amgen and jazz pharmaceuticals, and support for attending symposia from takeda, eusapharma, shire, and jazz pharmaceuticals. l moreno had a consulting or advisory role for novartis, astrazeneca, roche genentech, bayer, amgen, and mun-dipharma. he received honoraria from celgene and novartis for educational events and travel expenses from mundipharma, celgene, amgen. l moreno is a member of the executive committee of siopen, non-profit organization that receives royalties for the sales of dinutuximab beta. a. cañete had a consulting or advisory role for eusa pharma and bayer. she received honoraria from usa pharma for educational events and travel expenses. a. canete is a member of the executive committee of siopen, non-profit organization that receives royalties for the sales of dinutuximab beta. the rest of the authors declare that they have no conflict of interest. ethical approval all procedures performed in studies involving human participants were in accordance with the ethical standards of the participant institutions research committes and with the helsinki declaration and its later amendments or comparable ethical standards. informed consent informed consent was obtained from all individuals participating in clinical trials. covid- ) situation in numbers (by who region infection prevention and control and preparedness for covid- in healthcare settings -second update systematic review of covid- in children shows milder cases and a better prognosis than adults diagnosis, treatment, and prevention of novel coronavirus infection in children: experts' consensus statement recomendaciones sobre el manejo clínico de la infección por el «nuevo coronavirus» sars-cov . grupo de trabajo de la asociación española de pediatría (aep) medidas excepcionales aplicables a los ensayos clínicos para gestionar los problemas derivados de la emergencia por covid- -agencia española de medicamentos y productos sanitarios guidance on the management of clinical trials during the covid- (coronavirus) pandemic managing covid- in the oncology clinic and avoiding the distraction effect cancer patients in sars-cov- infection: a nationwide analysis in china. the lancet oncology zhonghua xue ye xue za zhi early advice on managing children with cancer during the covid- pandemic and a call for sharing experiences. pediatr blood cancer the role of the "innovative therapies for children with cancer correction: larotrectinib for paediatric solid tumours harbouring ntrk gene fusions: phase results from a multicentre, openlabel, phase / study how to improve rd productivity: the pharmaceutical industry's grand challenge the action to control cardiovascular risk in diabetes (accord) trial and hurricane katrina: lessons for managing clinical trials during and after a natural disaster mitigating the impact of disasters and emergencies on clinical trials site conduct: a site perspective following major and minor unforeseen events key: cord- -ihrj hr authors: alnaamani, khalid; alsinani, siham; barkun, alan n title: medical research during the covid- pandemic date: - - journal: world j clin cases doi: . /wjcc.v .i . sha: doc_id: cord_uid: ihrj hr the current pandemic of coronavirus disease (covid- ) which was first detected in wuhan, china in december is caused by the novel coronavirus named severe acute respiratory syndrome coronavirus- (sars-cov- ). the virus has quickly spread to a large number of countries leading to a great number of deaths. unfortunately, till today there is no specific treatment or vaccination for sars-cov- . most of the suggested treatment medications are based on in vitro laboratory investigations, experimental animal models, or previous clinical experience in treating similar viruses such as sars-cov- or other retroviral infections. the running of any clinical trial during a pandemic is affected at multiple levels. reasons for this include patient hesitancy or inability to continue investigative treatments due to self-isolation/quarantine, or limited access to public places (including hospitals). additional barriers relate to health care professionals being committed to other critical tasks or quarantining themselves due to contact with covid- positive patients. the best research approaches are those that adapt to such external unplanned obstacles. ongoing clinical trials before covid- pandemic have the potential for identifying important therapies in the long-term if they can be completed as planned. however, these clinical trials may require modifications due a pandemic such as this one to ensure the rights, safety, and wellbeing of participants as well as medical staff involved in the conduction of clinical trials. clinical trials initiated during the pandemic must be time-efficient and flexible due to high contagiousness of severe acute respiratory syndrome coronavirus , the significant number of reported deaths, and time constraints needed to perform high quality clinical trials, enrolling adequate sample sizes. collaboration between different countries as well as implementation of innovative clinical trial designs are essential to successfully complete such initiatives during the current pandemic. studies looking at the long term sequalae of covid- are also of importance as recent publications describe multi-organ involvement. long term follow-up of covid- survivors is thus also important to identify possible physical and mental health sequellae. the current pandemic of coronavirus disease (covid- ) which was first detected in wuhan, china in december is caused by the novel coronavirus named severe acute respiratory syndrome coronavirus- (sars-cov- ). the virus has quickly spread to a large number of countries leading to a great number of deaths. unfortunately, till today there is no specific treatment or vaccination for sars-cov- . most of the suggested treatment medications are based on in vitro laboratory investigations, experimental animal models, or previous clinical experience in treating similar viruses such as sars-cov- or other retroviral infections. the running of any clinical trial during a pandemic is affected at multiple levels. reasons for this include patient hesitancy or inability to continue investigative treatments due to self-isolation/quarantine, or limited access to public places (including hospitals). additional barriers relate to health care professionals being committed to other critical tasks or quarantining themselves due to contact with covid- positive patients. the best research approaches are those that adapt to such external unplanned obstacles. ongoing clinical trials before covid- pandemic have the potential for identifying important therapies in the long-term if they can be completed as planned. however, these clinical trials may require modifications due a pandemic such as this one to ensure the rights, safety, and wellbeing of participants as well as medical staff involved in the conduction of clinical trials. clinical trials initiated during the pandemic must be time-efficient and flexible due to high contagiousness of severe acute respiratory syndrome coronavirus , the significant number of reported deaths, and time constraints needed to perform high quality clinical trials, enrolling adequate sample sizes. collaboration between different countries as well as implementation of innovative clinical trial designs are essential to successfully complete such initiatives during the current pandemic. studies looking at the long the number of infectious diseases outbreaks over the past few decades have increased. this has created an enormous additional burden on the health systems of many countries around the world. as a majority of research activities during outbreaks targeting the causal organism must be completed in a timely fashion, it is of no surprise that effective preventative measures such as vaccination or therapies have not been available for many previous pandemic infectious diseases; in fact most of the previously used medications during past outbreaks have not been based on high quality evidence [ ] . despite the dedication of enormous resources, the advancement in health care systems and collaboration between different investigators across the world, only a small number of patients over the last decade have in fact benefited from clinical research performed during different outbreaks of respiratory viruses such as was the case for the severe acute respiratory syndrome (sars), the hin flu virus (swine flu) or the middle east respiratory syndrome. most of the medical research allocated resources and funding tend to happen during the disease outbreak. unfortunately lack of continuation of funding once the outbreak is over has led to disruption of many clinical trials assessing preventative, diagnostic, and therapeutic strategies [ ] . the current pandemic of coronavirus disease (covid- ) which was first detected in wuhan, china in december is caused by the novel coronavirus named severe acute respiratory syndrome coronavirus- (sars-cov- ) [ ] . the virus eventually spread to different regions of china and to the rest of the world leading to more than three million confirmed cases and hundreds of thousands of deaths at the time of writing of this manuscript [ ] . initial reports from the highly affected three countries china, italy and spain suggest that covid- has caused high mortality rate and/or overwhelmed regional intensive care units (icu) capacity [ , ] . the virus spreads via contact with infected upper respiratory droplets. the clinical presentation varies markedly from asymptomatic carriers to patients with acute respiratory distress syndrome requiring icu admission and mechanical ventilation [ ] . certain groups of patients are at higher risk for developing complications and death. high mortality rates have been associated with advanced age, presence of underlying disease, presence of secondary infection and elevated inflammatory markers [ ] . medical personnel in many countries have also been affected, leading to absence from work and, tragically, death in some cases. unfortunately, till today there is no specific treatment or vaccination for sars-cov- reports from initially affected countries describe using different medications such as hydroxychloroquine with azithromycin and in some adding anti-viral medications such as lopinavir/ritonavir, remdesivir or favipiravir for severe cases [ ] [ ] [ ] [ ] [ ] . other management options include inter-leukin and extracted immunoglobulin from recovered patients that have also been used with variable results [ ] . most of the suggested treatment medications are based on in vitro laboratory investigations, experimental animal models, or previous experience in treating similar viruses such as sars-cov- or other retroviral infections. it is important to note that in most of these trials there was no control group, therefore the safety profile and effectiveness of the studied medications cannot be accurately determined. due to the very contagious nature of sars-cov- , overwhelming the capacity of the health care system including out-patient clinics, in-hospital wards and icus, and in light of the significant mortality in the absence of effective treatment or prophylactic vaccination, focused medical research on sars-cov- is a necessity during this pandemic, with all patients deserving evidence-based care. the entire world needs a coordinated effort of all the available knowledge and resources to fight this pandemic and possibly similar future pandemics. but it is challenging to conduct research during pandemics. indeed, the running of clinical trials during a pandemic is affected at multiple levels. reasons for this include patient hesitancy or inability to continue investigative treatments due to selfisolation/quarantine, or limited access to public places (including hospitals). additional barriers relate to health care professionals being committed to other critical tasks or quarantining themselves due to contact with covid- positive patients [ ] . interruption of supply chains and monitoring of clinical trials are additional obstacles to medical research during pandemics ( figure ). the best research approaches are those that will adapt to such external unplanned obstacles. as example, similar to other research institutions, we are facing many challenges and obstacles at our own centers. perhaps one of the most important barriers are a patient's understandable anxiety and fear related to the current covid- pandemic. indeed, they are very hesitant to visit health care facilities to participate in new clinical trials, let alone continue in ongoing ones. some patients have also been quarantined due to contact with confirmed covid- cases. investigators at our institutions especially those involved in sponsored clinical trials are also facing challenges related to interruptions in the supply chains for investigational products, coupled to the inability of trial personnel to perform visits and/or investigations on enrolled patients. ethics committees must apply rigorous standards to authorize research in accordance with the principles of justice, equity and solidarity. lapses were reported during previous outbreaks such as in the west africa ebola virus (ev) outbreak leading to incomplete studies and inconclusive results [ ] [ ] [ ] . although we must strive to avoid publication bias (only submitting "positive studies"), the importance of unpublished data or not publishing incomplete data and inconclusive results must also be stressed. take for example the randomized controlled trial performed during an ev outbreak in looking at a triple monoclonal antibody cocktail (zmapp) as treatment for ev. the investigators concluded that the estimated effect of zmapp appeared to be beneficial, but the results in fact did not meet the prespecified statistical threshold for efficacy, thus leading to an underpowered negative trial [ ] . an example of unpublished results that need to be widely acknowledged because of a negative outcome leading to early termination is that of a brazilian study (clorocovid- ) which was a parallel, double-blind, randomized, phase iib clinical trial, which started on march , , aiming to assess safety and efficacy of chloroquine diphosphate (cq) in the treatment of hospitalized patients with severe respiratory syndrome secondary to sars-cov- infection. the high dose of cq mg at twice daily for d led to a higher fatality rate ( %) by day of the trial compared to the lower dose group and historical data from similar patients not using cq [ ] . medical research must therefore be adapted to the new socio-economic reality during the covid- pandemic. such studies can be divided into three main categories: ongoing medical research that antedated the pandemic, new research that is initiated during the pandemic, and future research on covid- that will follow it. standard of care and may represent a patient's only hope. furthermore, discontinuation of ongoing trials represents a waste of resources, and of time and efforts of patients who have completed the trial as well as medical and research personal involved in their design and implementation. supporting the efforts to continue ongoing clinical trials and modifying them, if necessary, is thus essential. the covid- pandemic poses significant challenges for many ongoing clinical research trials because of additional challenges. indeed, examples of barriers faced by sponsors and investigators include interruptions to the supply chains for the investigational products if applicable, the inability of trials personnel to perform visits and/or laboratory or imaging tests on enrolled patients, data entry, serious adverse events notification, and more generally follow various additional aspects of published trials protocols. due to above constrains, as a result, a series of measures need to be instituted, ensuring the rights, safety, and wellbeing of participants as well as the medical staff involved in the conduction of clinical trials. both sponsors and investigators must understand the critical situation faced by the medical authorities around the globe. fairness is thus a better way of approaching the ethical issues faced by clinical trial staff and regulatory agencies during this pandemic. fairness refers to the equitable distribution of benefits. it is about meeting everyone's individual needs; therefore cultural, racial, social or other biases must be avoided. in addition, clinical trials sponsors as well as investigators must uphold the spirit of existing laws while remaining fair to all individuals involved in the clinical trials [ ] . clinical trial sponsors and investigators must decide whether trial subject safety considerations allow continuation of the protocols, or rather temporarily halting administration of the interventional productor patient recruitment. many research regulatory authorities such as the united states food and drug administration and the clinical trials expert group at the european medicines agency have published statements to regulate medical research during the covid- pandemic in order to ensure the safeguard and preservation of rights of patients, medical staff and personnel involved in conducting, monitoring and the follow-up data collection and analysis of clinical trials [ , ] . local institutional review boards (irbs) are also encouraged to revise and modify their policies and procedures to adapt to the current situation without compromising patient care while ensuring the safety of all personnel involved in clinical trials. additional extensions to study periods beyond those already provided by local irbs must be allowed in order to accommodate for the unpredictable study operational time lost during this covid- pandemic. adopted modifications must not compromise the integrity of the study methodology while the trial must continue to be conducted in accordance with good august , volume issue clinical practices and comply with local and national regulatory requirements. modifications of protocols such as delaying recruitment of patients, postponing certain assessments and monitoring subjects through phone calls or virtual visits, limiting study visits, and preventing large gathering of study subjects as well as permitting certain tests to be done in nearby clinics rather than centrally, where appropriate, will most likely ensure patient and medical staff safety without compromising progression of the trials. sponsors as well as investigators should develop a process to identify, document, assess and report all protocol deviations to the sponsor and local irbs. these protocol deviations must also be documented to better complete and interpret subsequent analyses of the study findings. as it is not clear when the pandemic will end, it is important that investigators put the aforementioned strategies into place for an adequate period of time. medical research during covid- pandemic should focus on the epidemiology of the sars-cov- , its modes of transmission, covid- clinical presentations, effectiveness of preventative measures and possible modalities of treatment. medical research must be time-efficient and flexible due to high contagiousness of the sars-cov- , the large number of reported deaths, and time constraints needed to perform high quality clinical trials. regulatory processes must facilitate the generation of data from observational studies in real world settings that may be very important in better guiding the development of vaccines and therapies of covid- patients, and subsequently informing well designed, adequately powered, and well-funded randomized controlled trials. the latter should take priority, at least temporarily, over other competing clinical trials. moreover, patients and clinicians should be encouraged to participate in such initiatives [ ] . time limitation is a major factor leading to the incompleteness of clinical trials during infectious diseases outbreaks. during the ev outbreak in , the national institutes of health took several months to plan and launch the prevail ii trial. unfortunately, this trial was never completed while drawing on very significant resources [ ] . to overcome issues related to research during pandemic crises and to avoid publication bias a group of public health experts have advocated the use of a model called "core protocol" [ ] . this model describes the implementation of multiple interventions addressing a single disease at the same time such as using different off label medications to treat covid- , or of a single intervention targeting multiple diseases across different outbreaks such as remdesivir which was used to treat ev and now is being studies in covid- . this approach aims to minimize the time required to complete high quality clinical trials during outbreaks, with recognition of the additional important role of timely interim analyses to limit futility or harm and enhance the early recognition of efficacy. subsequent publication of results would only be carried out if pre-set criteria of efficacy or effectiveness are met (which may not be the case until further enrolment is reached during subsequent pandemics). the rapid and unpredictable nature of many pandemic infectious diseases such as covid- makes it impossible for a limited number of researchers to produce meaningful clinical results. to overcome such an obstacle, multi-center trials involving different countries with two-way exchange of information, protocols and procedures is an essential part of the global approach to any pandemic situation [ ] . such collaboration should be supported and facilitated by regulators in order to preserve resources and avoid underpowered studies, while achieving the planned outcomes. this will also help countries with limited resources that report on lower numbers of cases due to unavailability of testing. an excellent example of this kind of collaboration is the solidarity trial-a world health organization-funded international study of potential treatments for covid- to be conducted in asia, south africa, europe, and the americas [ ] . in pandemic times, therapeutic priority must be given to research focusing on interventions. investigators must concentrate on studying the effectiveness of already developed and approved medications used for similar viruses. the development of new drugs and testing through multi-phases during such emergencies may only be of limited benefit for the current pandemic affected patients, however results will of course be of help for similar future pandemics. august , volume issue as discussed above, modifications of policies and regulations established by local irbs during covid- to maintain ongoing studies should also be applied to new clinical trial submissions involving covid- patients. it is essential to discover new drugs to effectively fight what are likely repeated infectious diseases outbreaks in time. off-label use of available medications has disadvantages such as unknown efficacy, side effects and drug-drug interactions. therefore, when the pandemic is over, clinical trials that started during the pandemic of covid- should continue with their findings published once the planned accrual and/or efficacy, harm, or futility endpoints have been reached. other studies that may have used the earlier described core protocol model should extend across future multiple infectious disease outbreaks [ ] . due to the limited time available to enroll the large sample sizes of patients required for many high-quality clinical trials, and the enormous resources required during emergency pandemic diseases, earlier preparation for future outbreaks is becoming crucial for producing meaningful research findings. such planning should include multi-center participation from different countries to ensure sufficient sample size and representative diverse patient demographics favoring broad generalizability of results and meaningful, adequately powered subgroup analyses [ ] . such participation requires research capacity building, especially in countries with limited resources as well as support and collaboration from regulatory bodies to ensure success [ ] . the rapid action from concept to implementation of clinical studies is crucial during infectious disease outbreaks. this kind of approach will require innovative research designs such as "n of " trials [ ] adaptive platform trials [ ] , or ring vaccination designs [ ] with pre-outbreak approval, rapid implementation and timely effective methods of data exchange across global collaborations. continuation of research on banked patient materials such as fluids and tissue samples collected during outbreaks after appropriate consent will likely further enhance diagnostic and therapeutic discoveries. studies looking at the long term sequalae of covid- are also of importance. as recent publications describe multi-organ involvement, long term follow-up of covid- survivors can assist in identifying possible physical and mental health sequellae. laboratory based research such as life-saving vaccines and therapies, particularly against the new coronavirus must continue. cell lines research are essential for the production of vaccines, as are testing drug metabolism, cytotoxicity and antibody production during a pandemic [ ] [ ] [ ] . maintenance of cell lines requires uninterrupted examination of cell morphology and periodic change of medium [ ] . this kind of maintenance could be affected during covid- pandemic due to reduction of research laboratory staff and interruption in supply chains. currently, millions of animals are used in research laboratories around the globe [ ] . indeed, animals need to be looked after, and breeding lines must be kept intact. however, in some laboratories the work force has been reduced with researchers asked to stay away from their laboratories to minimize their risk of exposure to sars-cov- . these understandable measures have nonetheless affected the care and wellbeing of experimental animals. some researchers were even asked to dispose of animals while others had to interrupt their research in a way that will likely require the use of another animals once back to work [ ] . to minimize the impact of covid- on laboratory-based research, researchers should follow local institutions' safety regulations. many laboratory-based research institutions have established new regulation to ensure safety of investigators and continuity of work [ ] . some of these new regulations include reducing laboratory workforce to those conducting essential work, establishing a system of reporting symptoms and signs suggestive of covid- , working in small groups and keeping a distance of at least two meters between staff while also implementing working shifts. the who r&d blueprint: review of emerging infectious diseases requiring urgent research and development efforts sustained research fund and dedicated research center for preparing next pandemic, precision clinical medicine, pbaa cov- : an emerging coronavirus that causes a global threat the global battle against sars-cov- and covid- real estimates of mortality following covid- infection the epidemiology and pathogenesis of coronavirus disease (covid- ) outbreak the origin, transmission and clinical therapies on coronavirus disease (covid- ) outbreak -an update on the status clinical predictors of mortality due to covid- based on an analysis of data of patients from wuhan, china breakthrough: chloroquine phosphate has shown apparent efficacy in treatment of covid- associated pneumonia in clinical studies efficacy of hydroxychloroquine in patients with covid- : results of a randomized clinical trial hydroxychloroquine and azithromycin as a treatment of covid- : results of an open-label non-randomized clinical trial remdesivir and chloroquine effectively inhibit the recently emerged novel coronavirus ( -ncov) in vitro therapeutic options for the novel coronavirus ( -ncov) tocilizumab, an anti-il- receptor antibody, to treat covid- -related respiratory failure: a case report preserving clinical trial integrity during the coronavirus pandemic implementation of an ebola virus disease vaccine clinical trial during the ebola epidemic in liberia: design, procedures, and challenges schuchat a; strive study team. the sierra leone trial to introduce a vaccine against ebola: an evaluation of rvsv∆g-zebov-gp vaccine tolerability and safety during the west africa ebola outbreak chloroquine diphosphate in two different dosages as adjunctive therapy of hospitalized patients with severe respiratory syndrome in the context of coronavirus (sars-cov- ) infection: preliminary safety results of a randomized, double-blinded, phase iib clinical trial (clorocovid- study) a comparative analysis of moral principles and behavioral norms in eight ethical codes relevant to health sciences librarianship, medical informatics, and the health professions fda guidance on conduct of clinical trials of medical products during covid- pandemic: guidance for industry, investigators, and institutional review boards. us food and drug administration ema guidance on the management of clinical trials during the covid- (coronavirus) pandemic. the european medicines agency treating covid- -off-label drug use, compassionate use, and randomized clinical trials during pandemics creating a framework for conducting randomized clinical trials during disease outbreaks modernising epidemic science: enabling patient-centred research during epidemics director-general's opening remarks at the media briefing on covid- advancing preparedness for clinical research during infectious disease epidemics research as a part of public health emergency response the n-of- clinical trial: the ultimate strategy for individualizing medicine? performance of different clinical trial designs to evaluate treatments during an epidemic ethical rationale for the ebola "ring vaccination" trial design designing cell lines for viral vaccine production: where do we stand? cell lines: a tool for in vitro drug metabolism studies cell culture processes for monoclonal antibody production guidelines for the use of cell lines in biomedical research the flaws and human harms of animal experimentation care for laboratory animals during covid- crisis science-ing from home key: cord- - kmder authors: meyer, r. daniel; ratitch, bohdana; wolbers, marcel; marchenko, olga; quan, hui; li, daniel; fletcher, chrissie; li, xin; wright, david; shentu, yue; englert, stefan; shen, wei; dey, jyotirmoy; liu, thomas; zhou, ming; bohidar, norman; zhao, peng-liang; hale, michael title: statistical issues and recommendations for clinical trials conducted during the covid- pandemic date: - - journal: nan doi: nan sha: doc_id: cord_uid: kmder the covid- pandemic has had and continues to have major impacts on planned and ongoing clinical trials. its effects on trial data create multiple potential statistical issues. the scale of impact is unprecedented, but when viewed individually, many of the issues are well defined and feasible to address. a number of strategies and recommendations are put forward to assess and address issues related to estimands, missing data, validity and modifications of statistical analysis methods, need for additional analyses, ability to meet objectives and overall trial interpretability. the covid- outbreak emerging in china in december quickly became a global pandemic as declared by the world health organization in march . as of today, still only a few months into the pandemic, the disease and public health control measures are having very substantial impact on clinical trials globally. quarantines, site restrictions, travel restrictions affecting participants and site staff, covid- infections of study participants, and interruptions to the supply chain for study medication have led to operational problems, including difficulties in adhering to study protocols. trial sponsors have rapidly responded to this crisis, where the overriding primary concern has been to protect participant safety. some trials have been halted or enrolment suspended in the interest of participant safety. for ongoing trials, sponsors have implemented a variety of mitigations to assure safety of participants and address operational issues. the downstream effects of protocol deviations and trial conduct modifications lead to varying degrees of impacts on clinical trial data. the impacts, described in more detail in later sections, raise important statistical issues for the trial. in the extreme, trial integrity, interpretability, or the ability of the trial to meet its objectives could be compromised. intermediate to that, planned statistical analyses may need to be revised or supplemented to provide a thorough and appropriate interpretation of trial results. this paper offers a spectrum of recommendations to address the issues related to study objectives, inference, and statistical analyses. the major categories of impacts and mitigations are summarized in figure . the issues we discuss here largely involve ongoing trials, started before but conducted during the pandemic for non-covid- related therapies. many of the issues and recommendations will also apply to new trials. regulatory agencies have rapidly published guidance for clinical trial sponsors to address covid- issues (fda , ema a , b . the current paper is influenced by and expands upon these important guidance documents. the paper is organized as follows. in section we describe overall trial impact assessment. section considers assessment of impacts on the trial through the estimand framework. section summarizes recommendations for revised and supplemental analyses that may be needed for the trial, including the likely mechanisms of missing data and the recommended statistical approaches to address missingness. section outlines additional considerations for trial-level impact. a summary of recommendations is given in section . for trials impacted by the pandemic, assessing the change of the benefit/risk for participants is the first step in the decision-making process (fda ). all recommendations in this article presuppose that appropriate steps have been taken to assure participant safety. sponsors are advised to perform a risk assessment based on aggregated and blinded data to evaluate the likelihood of a trial to deliver interpretable results. it must start as a forwardlooking assessment in anticipation of effects not yet seen but with some likelihood to occur. it should continue throughout the conduct of the study in light of the evolving situation and accumulating data, considering regional differences in the infection status and pandemic-  determine what additional information needs to be collected in the study database or in the form of input from study investigators in order to adequately monitor, document, and address pandemic-related issues (feasibility to obtain such information and its quality may vary and this needs to be considered as part of the risk factors);  understand reasons for treatment or study discontinuation and the impact on planned estimands and intercurrent events;  evaluate extent of missing data and specific reasons for missingness;  assess changes in enrollment and in study population over time;  evaluate the protocol-specified assumptions and the likelihood that the trial would be able to achieve its goals;  ideally, verify the usability of data captured from alternative methods (e.g., virtual audio or video visits) before implementing them. such data may add more variability or not be interpretable;  determine any changes to planned analyses and analysis population definitions, or additional sensitivity analyses that need to be pre-specified prior to unblinding. based on the risk evaluations above, many sponsors have developed standardized metrics of trial operational status, such as rates of missed visits, discontinuations from treatment and study, protocol deviations, adverse events (aes), to reinforce a consistent approach to risk monitoring and assessment. such metrics are useful to identify trials that are more acutely impacted and to monitor the overall state of a portfolio of trials. the ich e (r ) addendum (ich, ) defines the estimand framework for ensuring that all aspects of study design, conduct, analysis, and interpretation align with the study objectives. it also provides a rigorous basis to discuss potential pandemic-related disruptions and to put them in the context of study objectives and design elements. for an affected trial, the first major question is whether the primary objective, and therefore the primary estimand, should target the treatment effect without potentially confounding influences of covid- . we recommend that for most studies started before the pandemic, the original primary objective should be maintained as designed, implying a treatment effect that is not confounded by pandemic-related disruptions. (for new studies, this definition of treatment effect may also be reasonable, but depends on many aspects of the trial design.) this does not automatically imply a broad "hypothetical estimand" with the same hypothetical scenario for all possible pandemic-related intercurrent events (ice). confounding may need to be addressed in different ways for different types of ices depending on the study context. we discuss this in section . our discussion is mainly geared towards considerations for the primary efficacy estimands, but similar logic can be applied to other study estimands. strategies for handling non-pandemic related ices should remain unchanged. here we discuss handling of pandemic-related ices only. the estimand framework allows for different strategies to be used for different types of ices and such estimands will likely be the most appropriate in the current context. ices should be considered pandemic-related if they occur as a result of pandemicrelated factors and are not attributed to other non-pandemic related reasons, e.g., treatment discontinuation due to lack of efficacy or a toxicity. pandemic-related ices of importance should first be categorized in terms of their impact on study treatment adherence (e.g., study treatment discontinuation) or ability to ascertain the target outcomes (e.g., death).  admitted to intensive care. * covid- related deaths and initiation of treatment for covid- infections may also be considered as ices if they occur after the completion of study treatment or after other non-pandemic-related ices and before the time point associated with the endpoint of interest. certain types of non-adherence to study treatment may not normally be considered as ices but may need to be in the context of the pandemic. for example, an ice of significantly reduced compliance or temporary treatment interruption may not have been anticipated at study design but could be now if considered likely due to pandemic-related disruptions. for some studies with significant pandemic-related treatment interruptions, the minimal duration of interruption expected to dilute the treatment effect could be defined. different strategies can be used for interruptions exceeding this duration as opposed to shorter interruptions. sensitivity analyses can be used to assess robustness of inference to the choice of cut-off. for time-to-event endpoints, it is tempting to define the minimally acceptable level of drug compliance on a participant level according to the observed exposure to study drug from the first dose of study drug until the event or censoring and exclude participants without a minimally acceptable level of compliance. however, such an approach could introduce immortal time bias and should therefore be avoided (van walraven et al, ) . a special consideration may be warranted for participants receiving experimental treatment for covid- regardless of whether they remain on study treatment. also, in studies where mortality was not originally expected, death due to covid- should be considered as a potential ice. most of the ice types listed against the "participant's adherence to study treatment" attribute in table (e.g., study treatment discontinuation) due to non-pandemic related reasons are likely addressed in the primary estimand prior to the pandemic. we recommend starting with an examination of whether the original strategy is justifiable when these ices occur due to the pandemic. if not, a different strategy should be chosen. we outline some high-level considerations in this respect.  treatment policy strategy, in which ices are considered irrelevant in defining the treatment effect, will typically not be of scientific interest for most pandemic-related ices because the conclusions would not generalize in the absence of the pandemic. for example, the treatment effect estimated under the treatment policy for premature treatment discontinuations caused by pandemic-related disruptions will reflect the effect of a regimen where discontinuations and changes in therapy occur due to pandemic-related factors (e.g., disruptions in study drug supply) which would not be aligned with the primary study objective. initiation of treatment for covid- infection after an earlier non-pandemic-related ice that was planned to be handled by the treatment policy strategy but prior to observation of an efficacy or safety endpoint will also need to be considered carefully and cannot simply be deemed irrelevant. under the treatment policy strategy, the estimated treatment effect may reflect the effects of infection and its treatment, which are presumably not of interest for the primary objective. a decision to use a treatment policy approach for pandemic-related ices may be justifiable if the percentage of participants with such events is low and this strategy was planned for similar non-pandemic related ices. this strategy may also be considered for handling ices corresponding to relatively short treatment interruptions. the treatment policy strategy should be avoided for pandemic-related ices of premature study treatment discontinuation in non-inferiority and equivalence studies, as similarity between treatment groups may artificially increase with the number of such events. similar considerations also apply to the composite strategy.  composite strategies, in which ices are incorporated into the definition of the outcome variable, are unlikely to be appropriate for most pandemic-related ices. for example, study treatment discontinuation due to pandemic-related disruptions should not be counted as treatment failure in the same way as discontinuation due to lack of efficacy or adverse reactions. a more nuanced consideration may be needed in studies of respiratory conditions, where covid- complications may be considered with a composite strategy as a form of unfavourable outcome. (see also a discussion on covid- related deaths further below.)  principal stratification strategy stratifying on a covid- related event (e.g., serious complications or death due to covid- ) is unlikely to be of interest for the primary estimand because it would limit conclusions to a sub-population of participants defined based on factors not relevant in the context of the future clinical practice.  while-on-treatment strategy may continue to be appropriate under certain conditions if it was originally planned for non-pandemic-related ices. this strategy is typically justifiable when treatment duration is not relevant for establishing treatment effect (e.g., treatment of pain in palliative care), but certain conditions may need to be considered, such as a minimum treatment duration required to reliably measure treatment outcomes.  hypothetical strategy, in which the interest is in the treatment effect if the ice did not occur, is a natural choice for most pandemic-related ices. this would especially apply to ices of study treatment disruption for pandemic reasons. for such participants, the hypothetical scenario where they would continue in the study in the same way as similar participants with an undisrupted access to treatment is reasonable. it is not necessary to assume a hypothetical scenario where such participants would fully adhere to the study treatment through the end of the study. rather, a hypothetical scenario may include a mixture of cases who adhere to the study treatment and those who don't adhere for non-pandemic reasons. discussions with regulatory agencies may be helpful to reach an agreement on the details prior to the final study unblinding. although estimation methods are not part of the estimand consideration, the ability to estimate treatment effects in a robust manner under a hypothetical strategy based on available data should not be taken for granted and should be assessed as the estimand definition is finalized. this aspect should be part of the overall risk assessment and decisions on the choice of the mitigation strategies. the discussion above highlights the need to capture the information associated with pandemic-related factors, such as those listed in table . this can be done either through designated fields in the case report form (crf) or through a detailed and structured capture of protocol deviations. an ice of death due to covid- requires careful consideration and the appropriate strategy depends on the disease under study and clinical endpoint. in disease areas with minimal mortality where death is not a component of the endpoint, a hypothetical strategy for deaths related to covid- infections may be recommended. for studies in more severe diseases where death is part of the endpoint, it is inevitable that more than one estimand will be of interest when evaluating the benefit of treatment for regulatory purposes. a pragmatic approach which includes covid- -related deaths in the outcome, i.e., which uses a composite strategy, is suitable if the number of covid- -related deaths is low or if there is a desire to reflect the impact of the pandemic in the treatment effect estimate. (see also the related section on competing risks analyses in section . . .) using a hypothetical strategy for deaths related to covid- infections will be important in evaluating the benefit of treatment in the absence of covid- (for example, when the disease is eradicated or effective treatment options emerge). it is acknowledged that such trials frequently include elderly, frail, or immunocompromised participants and it may be difficult to adjudicate a death as caused by covid- or whether the participant died with covid- . while treatment policy, composite, and principal stratification strategies may not be of interest for the primary estimand, they may be of interest for supplementary estimands when there is a scientific rationale to investigate the study treatment either in subpopulations of participants stratified based on covid- infection and outcomes and/or together with a concomitant use of treatments administered for covid- . for example, this may be of interest for studies in respiratory diseases or conditions suspected to be risk factors for covid- complications. the relevance of such estimands will also depend on the evolution of this pandemic, whether the virus is eventually eradicated or persists like seasonal flu. in the latter case, the reality and clinical practice are still likely to be different from the current crisis management conditions as the society and clinical practice adapt to deal with a new disease. in general, treatment condition of interest should remain the same as originally intended. however, operationally, the mode of treatment delivery may need to be changed due to pandemic-related reasons, e.g., treatment self-administered by the participant at home rather than in the clinic by a health-care professional. when such changes are feasible, they may be considered to reduce the frequency of visits to the clinic, and therefore reduce the risk of infection exposure. pandemic-related complications with study treatment adherence and concomitant medications should be considered as ices and handled with an appropriate strategy. the extent of such ices should be evaluated in terms of whether the treatment(s) received by participants during the study remain sufficiently representative of what was intended. this may include treatment interruptions, reduced compliance, and access to any background, rescue, and subsequent therapies planned to be covered by the treatment policy strategy. to align with the primary study objective, the target participant population should remain as originally planned, i.e., should not be altered simply due to the pandemic. protocol amendments unrelated to the pandemic to further qualify the study population should still be possible. the trial inclusion/exclusion criteria should also remain largely unchanged relative to those that would be in place in absence of the pandemic, except for the possible exclusion of active covid- infections. the clinical endpoint should generally remain as originally planned. in cases where alternative measurement modalities may be necessary during the pandemic, for example, central labs vs. local labs, remote assessments of questionnaires instead of in-clinic, etc., it must be assured that clinical endpoint measurement is not compromised and potential effects on endpoint variability should be assessed (see section . . ). in cases where pandemic-related ices, such as covid- deaths, are handled using the composite strategy, the definition of the endpoint may need to be adjusted. in cases of numerous delays between randomization and start of treatment, where the endpoint is defined relative to the date of randomization (e.g., in time-to-event endpoints), consideration may be given to redefine the endpoint start date to start of treatment. however, in the context of openlabel studies this may not be advisable. population-level summary describing outcomes for each treatment and comparison between treatments should remain unchanged, in general. in rare situations, a summary measure may need to be changed, for example, if the originally planned endpoint is numeric and a composite strategy is used for covid- deaths to rank them worse than any value in survived participants. in this case, a summary measure may be changed from mean to median. another example could be a hazard ratio (hr) from a cox proportional hazard regression, a summary measure of treatment effect commonly used for trials with time-toevent endpoints. if the assumption of proportional hazards is not satisfied, the estimated hr depends on the specific censoring pattern observed in the trial, which is influenced both by participant accrual and dropout patterns (rufibach, ) . external validity and interpretability of the hr needs to be carefully considered if censoring patterns are affected during the pandemic in ways that are not representative of non-pandemic conditions and if additional pandemic-related censoring depending on covariates such as the participant's age or comorbidities are observed. similarly, the validity of the log-rank test relies on the assumptions that the survival probabilities are the same for participants recruited early and late in the trial and that the events happened at recorded times. such assumptions may need to be assessed. supportive estimands with alternative summary measures could be considered (see e.g., boyd et al., ; nguyen and gillen, ; mao et al., ) . planned statistical analyses may need to be modified due to effects of the pandemic on trials. additional sensitivity and supplementary analyses may be needed to properly understand and characterize the treatment effect. depending on the trial, modifications in planned analyses may range from relatively minor, e.g., for trials with relatively low impact, to major, e.g. in settings where study drug administration and visits are severely disrupted by the pandemic. a general summary of analysis considerations is provided in table and detailed discussions are presented in subsequent sections. all planned modifications and additional analyses should be documented in the sap prior to data unblinding and in the clinical study report. additional post hoc exploratory analyses may also be necessary after study unblinding to fully document the impact of the pandemic and characterize the treatment effect in this context.  review all planned main and sensitivity analyses to ensure alignment with the revised estimand(s).  review / amend methods for handling of missing data, or censoring rules, to accommodate pandemic-related missingness.  summarize the occurrence of pandemic-related ices and protocol deviations.  summarize the number of missed or unusable assessments for all key endpoints.  summarize the number of assessments performed using alternative modalities.  summarize study population characteristics before and after pandemic onset. additional sensitivity and supportive analyses  plan additional analyses for sensitivity to pandemic-related missingness.  consider the need for additional, alternative summary measures of treatment effect.  consider exploring inclusion of additional auxiliary variables, interaction effects, and time-varying exogenous covariates in the analysis methods.  consider subgroup analyses based on subgroups defined by pandemic impact, e.g. primary endpoint visits before or after pandemic onset.  consider the need for evaluation of potential impact of alternative data collection modalities.  consider sources of data external to the trial, for example to justify use of alternative modalities.  plan for additional safety analyses. all planned efficacy analyses should be re-assessed considering the guidance provided in sections and in terms of handling of pandemic-related missing data (see section . ). the core analysis methodology should not change. however, depending on the revisions to the estimand, the strategies chosen for pandemic-related ices, and the handling of pandemicrelated missing data, some changes to the planned analyses may be warranted. additional analyses will frequently also be required to assess the impact of the pandemic disruption. special considerations may be needed for studies and endpoints where participant outcomes could be directly impacted by the pandemic, e.g., in respiratory diseases or quality of life endpoints. in studies where enrolment is halted due to the pandemic, sponsors should compare the populations enrolled before and after the halt. more generally, shifts in the population of enrolled participants over the course of the pandemic should be evaluated. baseline characteristics (including demographic, baseline disease characteristics, and relevant medical history) could be summarized by enrolment period to assess whether there are any relevant differences in the enrolled population relative to the pandemic time periods. shifts could be associated with regional differences in rates of enrollment because start and stop of enrollment is likely to vary by country as pandemic measures are implemented or lifted. sponsors should make every effort to minimize the amount of missing data without compromising safety of participants and study personnel during the covid- pandemic and placing undue burden on the healthcare system. whenever feasible and safe for participants and sites, participants should be retained in the trial and assessments continued, with priority for the primary efficacy endpoint and safety endpoints, followed by the key secondary endpoints. despite best efforts, sponsors should prepare for the possibility of increased amounts and/or distinct patterns of missing data. in the framework of ich e (r ), an assessment or endpoint value is considered missing if it was planned to be collected and considered useful for a specific estimand but ended up being unavailable for analysis. in case of ices that are addressed by a hypothetical strategy, endpoint values are not directly observable under the hypothetical scenario. such data are not missing in the sense of the ich e (r ) definition, however, they need to be modelled in the analysis, often using methods similar to those for handling of missing data. in the remainder of the paper, we will discuss methods for handling of missing data, and note that such methods can be useful for modelling unobserved data after ices, if the modelling assumptions align with the hypothetical scenarios chosen for addressing the corresponding ices. sponsors should assess and summarize patterns (amount and reasons) of pandemic-related missing data in affected trials. data may be missing because a) planned assessments could not be performed; b) collected data is deemed unusable for analysis, e.g., out-of-window; or c) data under a desired hypothetical scenario cannot be observed after an ice (e.g., censored). additionally, each pandemic-related missingness instance also has specific reasons and circumstances. at a high level, reasons for pandemic-related missing data could be structural (e.g., government enforced closures or sites stopping study-related activities) or they could be participant-specific (e.g., individual covid- disease and complications or individual concerns for covid- ). table outlines the aspects that together provide a comprehensive picture for assessing the impact of missing data and planning how to handle them in analysis. sponsors should, therefore, capture such information in the clinical study database as much as possible. the last two rows of table reflect circumstances similar to those that are considered in the context of ices (see table ). since missing data may occur both in the presence of ices (e.g., handled by a hypothetical strategy) and in the absence of ices (e.g., participant continues to adhere to study treatment but misses some assessments), we list them here as well. table . attributes of pandemic-related missing data. row summarizes reasons of missing data, rows - summarize related conditions that contribute to those reasons.  assessment missing due to a participant's premature discontinuation from the study overall for pandemicrelated reasons;  assessment missing due to missed study visits/procedures while a participant remains in the study (intermittent missing data);  assessment delayed (out-of-window) and deemed unusable for an analysis;  a composite score (e.g., acr in rheumatoid arthritis) cannot be calculated because some components are missing;  assessment deemed to be influenced by pandemic-related factors and deemed unusable for a particular analysis because the interpretability of the results may be impacted (e.g., in assessments of quality of life, activity/functional scales, healthcare utilization, etc);  recorded data cannot be properly verified or adjudicated due to covid- -related factors and deemed to be unreliable for analysis;  assessment performed after an intercurrent event intended to be handled with a hypothetical strategy and collected data are deemed unusable for this estimand. assessment accessibility  site (facilities or staff) unavailable to perform studyrelated assessments;  site/assessment procedure available but participant is unable/unwilling to get assessment done due to personal pandemic-related reasons. sponsors should also consider a potential for under-reporting of symptoms and aes during the pandemic due to missed study visits or altered assessment modalities, e.g., a telephone follow-up instead of physical exam. (see section . .) sponsors should also consider reporting patterns of missingness along several dimensions: over time (in terms of study visits as well as periods before and after the start of pandemic disruptions), with respect to certain demographic and baseline disease characteristics, as well as co-morbidities considered to be potential risk factors associated with covid- infection or outcomes, and across geographic regions. blinded summaries of missing data patterns prior to study unblinding may inform the choice of missing data handling strategies. it may be useful to compare missing data patterns from the current studies with similar historical studies, especially with respect to missingness in subgroups. after study unblinding (for final or unblinded dmc analyses), missing data patterns should be summarized overall and by treatment arm. although in most cases pandemicrelated missingness, especially structural missingness, would not be expected to differ between treatment arms, such a possibility should not be ruled out. in special circumstances, such as in an open-label study, missing visits may be related to treatment if the experimental treatment must be administrated at the site while the control treatment could be administrated at home, there may be more missing assessments in the control group. this may result in biased treatment effect estimates if mitigating strategies are not implemented. this could also be the case for a single cohort trial using an external control. sponsors should generally maintain the same approaches for handling of non-pandemicrelated missing data as originally planned in the protocol and sap. for pandemic-related missingness, appropriate strategies will need to be identified in the context of each estimand and analysis method. which strategy is most appropriate should be considered in light of the underlying context and reasons for missingness as shown in table and in alignment with the estimand for which the analysis is performed. three cases are described. ( ) when data are missing without the participant having an ice, i.e., participant continues to adhere to study treatment but has some endpoint values missing: the missing data modelling should be based on clinically plausible assumptions of what the missing values could have been given the fact the participant continues to adhere to study treatment and the participant's observed data. ( ) when data are modelled in presence of an ice: the strategy defined in the estimand for addressing that ice should be considered. provide an adequate selection of tools to deal with pandemic-related missingness (see e.g., molenberghs and kenward, ; nrc, ; ratitch, , mallinckrodt et al., ) . methods for dealing with missing data are often categorized based on the type of assumptions that can be made with respect to the missingness mechanism. using molenberghs and kenward ( ) 's classification of missingness mechanisms aligned with longitudinal trials with missing data, data are missing completely at random (mcar) if the probability of missingness is independent of all participant-related factors or, conditional upon appropriate pre-randomization covariates, the probability of missingness does not depend on either the observed or unobserved outcomes. (we note that in the framework of little and rubin ( ) , mcar is defined as independent of any observed or unobserved factors. this definition was subsequently generalized to encompass dependence on prerandomization covariates, also referred to as covariate-dependent missingness, and mcar is now used in the literature in both cases.) some types of pandemic-related missingness may be considered mcar, e.g., if it is due to a site suspending all activities related to clinical trials. consideration may be given to whether such participants should be excluded from the primary analysis set depending on the amount of data collected prior to or after the pandemic. for example, when all (most) data are missing for some participants, imputing their data based on a model from participants with available data would not add any new information to the inference, while excluding such participants is unlikely to introduce bias. when participants have data only for early visits before the expected treatment effect onset and the rest of the data are mcar, then including such participants in the analysis set may not add information for inference about treatment effect while adding uncertainty due to missing data. data are mar if, conditional upon appropriate (pre-randomization) covariates and observed outcomes (e.g., before participant discontinued from the study), the probability of missingness does not depend on unobserved outcomes. if relevant site-specific and participant-specific information related to missingness is collected during the study, most of the pandemic-related missingness can be considered mcar or mar. the definitions of mcar and mar mechanisms are based on conditional independence of missing data given a set of covariates and observed outcomes that explain missingness. the factors explaining pandemic-related missingness may include additional covariates and, in the case of mar, post-randomization outcomes. for example, missingness during the pandemic may depend on additional baseline characteristics, e.g., age and co-morbidities, as well as post-randomization pandemic-related outcomes, such as covid- infection complications. in the case of covariate-dependent missingness, regression adjustment for the appropriate baseline covariates is sufficient for correct inference, though this complicates the analysis model and the interpretation of the treatment effect for models such as logistic or cox regression where conditional and marginal treatment effect estimates do not agree. under mcar and mar, some modelling frameworks such as direct likelihood, e.g., mixed models for repeated measures (mmrm), can take advantage of separability between parameter spaces of the distribution of the outcome and that of missingness. in this case, missingness can be considered ignorable (molenberghs and kenward, ) , and the factors related only to missingness do not need to be included in the inference about marginal effects of treatment on outcome. this does not, however, apply to all inferential frameworks. multiple imputation (mi) methodology (rubin, ) may be helpful in this respect as it allows inclusion of auxiliary variables (both pre-and post-randomization) in the imputation model while utilizing the previously planned analysis model. multiple imputation with auxiliary variables may be used for various types of endpoints, including continuous, binary, count, and time-to-event and coupled with various inferential methods in the analysis step. the use of mi with rubin's rule for combining inferences from multiple imputed datasets may introduce some inefficiencies and impact study power, although some alternatives exist (see, e.g., schomaker and heumann, ; hughes et al., ) . for implementing a hypothetical strategy for covid- related ices in the context of time-to-event endpoints, regression models (e.g., cox regression) adjusted for relevant baseline covariates (in the mcar setting) or multiple imputation (in the mar setting) is recommended. competing risks analyses which treat pandemic-related ices that fully or partially censor the outcome, e.g., covid- related deaths, as competing events are not compatible with a hypothetical strategy. a further complication in the interpretation of competing risks analyses in this context is that participants are not at risk for the competing event from their time origin (e.g., randomization or start of treatment) onwards but only during the pandemic which is not experienced synchronously across the trial cohort. this compromises the validity of common competing risks analyses and prevents interpretation of results from such analyses to be generalized to the population. the implication of assuming specific missingness mechanism is that missing outcomes can be modelled using observed pre-randomization covariates or covariates and post-randomization outcomes (mar) from other "similar" participants, conditional on the observed data. for pandemic-related missingness, it is important to evaluate whether there are sufficient observed data from "similar" participants to perform such modelling, even if factors leading to missingness are known and collected. for example, if missingness and endpoint depend on age, and few older participants have available endpoint data, it may not be appropriate to model missing data of older participants using a model obtained primarily from available data of younger participants (possibly resulting in unreliable extrapolation). similarly, severe complications of a covid- infection may be due to these participants having a worse health state overall and modelling their outcomes based on data from participants without such complications may not be justifiable. additional assumptions about participants with missing data versus those with observed data may need to be postulated and justified, perhaps based on historical data. this is where the consideration of pandemic-related factors surrounding missingness, such as those mentioned in table , is important. pandemic-related missing data may need to be considered mnar if missingness and study outcomes depend on covid- related risk factors, treatment, and infection status but such data are not collected. the missingness mechanism may also be mnar if it depends on unobserved study outcomes. in the context of the pandemic, it may arise when participants with milder disease or lower treatment response are more inclined to discontinue the study or treatment and if their outcomes and reasons for discontinuation are not documented before discontinuation. analysis under mnar requires additional unverifiable assumptions but may be avoided through collection of relevant data. analysis of sensitivity to departures from mar assumptions should be considered, for example, models assuming plausible mnar mechanisms (see e.g., carpenter et al., ; mallinckrodt et al., ) . modifications to planned main analyses needed to handle pandemic-related missing data should be specified in the sap prior to study unblinding. see section . . for a discussion of sensitivity analyses with respect to missing data. additional sensitivity and supplementary analyses will frequently be required to assess the impact of pandemic-related disruptions on the trial. for non-pandemic-related events and missing data, the originally planned sensitivity analyses should be performed, but simply applying the pre-planned sensitivity analyses to both pandemic and non-pandemic ices and missing data may be problematic for three reasons. first, the objective to estimate treatment effects in the absence of a covid- pandemic may mandate different strategies for pandemic-related and unrelated events. second, as discussed in section . . , an mcar or mar assumption is frequently plausible for covid- -related events whereas this may not be the case for other missing data. third, sensitivity analyses to missing data could become excessively conservative if the amount of pandemic-related missing data is large. while a relatively large proportion of missing data could normally be indicative of issues in trial design and execution leading to greater uncertainty in trial results, that premise is tenuous when an excess of missing data is attributed to the pandemic. subgroup analyses for primary and secondary endpoints by enrolment or pandemic (see section . . ) period are recommended. if subgroup analyses are indicative of potential treatment effect heterogeneity, the potential for this to be rooted in regional differences should be considered. issues of multiregional clinical trials as described in ich ( ) may be magnified by the pandemic. in addition, dynamic period-dependent treatment effects could also be assessed in an exploratory fashion. for example, in models for longitudinal and time-to-event endpoints, one could include interaction terms between the treatment assignment and the exogenous time-varying covariate describing the patients' dynamic status during follow-up. however, results from such analyses may be nontrivial to interpret and generalize. more liberal visit time-windows may be appropriate for visits depending on the specific trial. sensitivity analyses should assess the robustness of results to out-of-schedule visits by either including them or treating them as missing data. a tipping point analysis (ratitch et al., ) may be used to assess how severe a departure from the missing data assumptions made by the main estimator must be to overturn the conclusions from the primary analysis. tipping point adjustments may vary between the pandemic vs. non-pandemic missingness and by reason of missingness. for example, one could tip missing data due to pandemic-unrelated missing data but use standard mar imputation for pandemic-related missing data. historical data may be useful to put in context the plausibility of the assumptions and tipping point adjustments. when main analysis relies on imputation techniques, sensitivity analyses can be done by using an extended set of variables in the imputation algorithm. when dealing with missing data in a context of intercurrent events handled with the hypothetical strategy, one could vary the assumed probability that the participant would have adhered to treatment through the end of the study vs that they would have had other non-pandemic-related events and impute their outcomes accordingly. for binary responder analysis, it is not uncommon to treat participants who have missing assessments as non-responders when a proportion of such cases is very small. for pandemic-related delayed and missed assessments, especially those occurring in absence of ices, it may be preferable to use a hypothetical strategy based on a mar assumption instead. however, anon-responder imputation for both non-pandemic and pandemic-related missing response assessments could be reported as a supplementary analysis. for time-toevent endpoints, we recommend the usage of interval-censored methods to account for cases where the event of interest is known to have occurred during a period of missing or delayed visits but the time to event is not precisely known in sensitivity analyses (bogaerts, komarek, and lesaffre, ) . alternative measurements of endpoints may be necessary during the pandemic period. a careful study-specific assessment is necessary to judge whether these alternative measurements are exchangeable with standard protocol assessments. ideally, exchangeability is established at the time of implementation based on information external to the trial. if not, blinded data analyses can support this, e.g. comparisons of the distribution of alternative measurements to the original measurement. however, if the validity of the new instrument has not been established previously, it will be challenging to rigorously demonstrate equivalence using data from the trial alone. if exchangeability can be established or assumed, the main estimator could include data from both the original and the alternative data collection. the sensitivity analysis would then include data collected according to the original protocol only and treat other data as missing. if the validity of the exchangeability assumption is uncertain, the opposite approach can be taken. modelling the interaction between treatment and assessment method can be undertaken as an alternate sensitivity analysis. outcomes it will be important to understand the pandemic effect on trial outcomes. there are several schools of thought on how this could be done. previous sections in this paper have described the need for collecting data describing how events such as missed visits and treatment interruptions can be attributed to the pandemic. these data are incorporated into definitions of pandemic-related ices and missing data, and the strategies for handling those in the analysis. statistical analyses of trial data will then be properly adjusted for pandemic effects. in many ways this is the ideal approach as it incorporates what is known for each participant directly into the analysis, and in a way that is very well understood. this is a standard approach to adjusting statistical analysis for inevitable perturbations in clinical trials. this approach has the disadvantage of needing to collect detailed data on pandemicrelatedness, which may not be feasible in some circumstances. another approach involves the use of pandemic time periods defined external to the trial database (e.g., to define pre-/during/post-pandemic phases as described in ema/chmp b). this approach requires the accurate and precise definition of pandemic periods. this is simpler to apply in single-country studies where the impact of the pandemic and local containment measures may be relatively homogeneous across participants. however, even for a single country, the pandemic may evolve in a gradual fashion, complicating the definition of pandemic start and stop dates, and the impact of the pandemic on study participants will likely not be homogenous. moreover, there may be several waves of infection outbreaks. the definition could prove even more challenging to implement in global trials because the start and stop dates of these periods and the impact of the pandemic on study participants may well vary by region. in practice, a standardized and pragmatic definition based on regionally reported numbers of covid- cases and deaths over time and/or start date and stop dates of local containment measures will likely be required. once pandemic periods have been defined, time-varying indicator variables for visits occurring during different pandemic periods could be incorporated as appropriate in statistical models or in ice definitions, as discussed previously. when this approach is used, the rationale for defining the pandemic start and stop dates should be documented. the situation is evolving rapidly and at this is point, it is not possible to provide definitive recommendation on the definition, implementation, and interpretation of these pandemic periods. in a third approach to generally assessing pandemic effects, each participant in the trial could be categorized according to the extent of pandemic impact on their treatment and assessments collected in the study database (details of protocol deviation, ices, missing assessments, pandemic-related reasons for discontinuation etc.). for trials with a fixed follow-up duration and minimal loss to follow-up, the categorization could be integrated in standard analyses, for example in defining subgroups for standard subgroup analysis. there is insufficient evidence currently to favour a single approach to this issue. sponsors are preparing to do at least the first two approaches, as these have been the subject of regulatory guidance. until we see how they play out and how the pandemic evolves, etc., it's sensible to consider multiple methods of summarizing pandemic effects. standard safety summaries will include all aes as usual. however, additional separate analyses may be needed for events associated with covid- infections and unassociated events, respectively, to fully understand the safety profile. the determination whether aes and, particularly, deaths are covid- related should be made during trial monitoring before data unblinding to avoid bias. in many situations, safety reporting will remain unchanged. however, disruptions due to the covid- pandemic may lead to increases in treatment interruptions, discontinuations and study withdrawals as well as the occurrence of covid- infections and deaths. hence, the estimands framework outlined in section could also be useful for safety analyses and we refer to (unkel et al, ) for a general discussion of estimands as well as time-to-event and competing risks analyses for safety. trials that require physical visits to adequately assess safety of the intervention will need to have maintained a schedule of physical visits to satisfy the requirement. generally addressing the potential for bias in collection of ae data is beyond the scope of this paper. exposure considering the compliance rate-or follow-up-adjusted analyses could be done, e.g., comparing the adjusted rates before and during the pandemic or with historical data. we don't have other methodological recommendations at this time, and more research is needed. the cumulative impact of missing data and revised statistical models discussed in the previous sections contribute to an overall study-level impact. the cumulative effect could alter the likelihood of meeting trial objectives, or even the interpretability of the trial results. sponsors should assess potential impact of missing data on several aspects and it may be important to reach agreement with regulatory agencies on some of these questions:  feasibility of planned estimation methods given the data expected to be available;  potential for bias in treatment effect estimation if there are important differences in missingness patterns across treatment arms;  study power for the primary and key secondary objectives;  interpretability of study findings;  adequacy of safety database due to potential reduction in total drug exposure time and potential for underreporting of aes;  adequacy of regional evidence required for regulatory submissions. as discussed in the previous sections, covid- related factors impact trial data in many ways with consequences for power of the study, probability of success, sample size or other aspects of the trial design. quantifying the potential effects of the various pandemic factors on trial results can be done through clinical trial simulations. the simulation models will depend on the factors used in the original trial design, and incorporate adjustments to estimands, missing data handling and analysis methods as discussed in sections and . to maintain trial integrity, the simulations should be informed only by blinded data from the study and the assumed values for the design parameters from external sources. variability and treatment effect estimates may be modified from their original values used in trial design. trial properties such as power and probability of success can be updated accordingly. sample size adjustment can be considered based on the simulation outcomes, or more extensive modifications of the trial design also may be considered such as change in the primary endpoint, analysis method, or addition of interim analysis with associated adaptation (posch and proschan ; kieser and friede ; muller and schaefer ) . such changes are challenging, and should be discussed with regulatory agencies, but can be considered if trial integrity is maintained. for some trials, it may not be feasible to increase sample size and the trial will fall short of enrollment target. given the extraordinary circumstances, we advocate more flexibility to consider methods for quantifying evidence across multiple trials and sources, including use of historical control arm data and real-world data, although sources and methodology for selection of such data would need to be planned and agreed with regulatory agencies in advance. if the observed treatment effect after data unblinding is meaningful but does not meet the statistical criterion due to covid- effects, the sponsor can evaluate whether the study results will be acceptable for registration on the basis of the accumulated evidence from the program; alternatively, whether the trial results could be used to define the inferential prior for a smaller follow up trial (viele et al ) . for a trial with a dmc, the sponsor should ensure that the dmc is well-informed of all measures taken to protect participant safety and to address operational issues. known or potential shortcomings of the data should be communicated. the timing of the regular preplanned safety interim analyses may need to be re-assessed. in addition, revised or additional data presentations may be needed. in some circumstances of interim analysis discussed in this section, a dmc may need to be established if not already in existence. there could also be circumstances related to participant safety where there may be a need to urgently review unblinded data, and establishing an internal dmc that is appropriately firewalled from the rest of the study team is recommended (e.g., studies without an existing dmc where it could take many months to organize an external dmc). efficacy interim analyses should be conducted as planned with information level (e.g., number of participants with primary endpoint or specific information fraction) as described in the protocol, which may cause a delay in the expected timeline. intermediate unplanned efficacy or futility interim analyses are generally discouraged unless there are safety and ethical considerations. however, if it is not feasible to reach the planned information level, altering the plan for interim analysis would need to be considered, for example with timing based on calendar time. in cases with strong scientific rationale for an unplanned interim analysis, the dmc should be informed and consulted on the time and logistics of the analysis. if an estimand, planned analysis methods, and/or decision rules have been changed to address pandemic-related disruptions and missing data, these changes should be communicated to the dmc and documented in the dmc charter. we do not generally advocate utilizing a dmc for operational risk assessment / mitigation process, to prevent influence of unblinded data on trial conduct decisions (ema/chmp , fda ). if the sponsor decides to involve the dmc, details should be clearly defined and documented in the dmc charter, including additional responsibilities of dmc members and measures to prevent introduction of bias. as we have discussed, the covid- outbreak continues to have major impact on planned and ongoing clinical trials. the effects on trial data have multiple implications. in many cases these may go beyond the individual clinical trial and will need to be considered when such results are included with other trial results, such as an integrated summary of efficacy or safety. our goal was to describe the nature of the statistical issues arising from covid- potential impact on ongoing clinical trials and make general recommendations for solutions to address the issues. the following are the most important findings and recommendations:  risk assessment, mitigation measures, and all changes to study conduct, data collection, and analysis must be documented in statistical analysis plans and clinical study reports as appropriate. some changes may necessitate protocol amendments and consultation with regulatory agencies (fda , ema a , b .  implications of the operational mitigations for the statistical analysis of the trial data should be considered before implementing those mitigations, especially for key efficacy and safety endpoints. validity and exchangeability of alternate methods of data collection require careful consideration.  the estimand framework, comprised of five key attributes, provides a pathway for assessing the impact of the pandemic on key study objectives in a systematic and structured manner and may be useful regardless of whether estimands are formally defined in the study protocol.  as much as possible, we recommend that original objectives of the trial be maintained; but some impact to planned estimands may be unavoidable. pandemicrelated intercurrent events will likely need to be defined to properly and rigorously account for unexpected pandemic effects.  planned efficacy and safety analyses should be reviewed carefully for changes needed to ensure that the estimators and missing data strategies align with updated estimands. additional sensitivity and supportive analyses will be needed to fully describe the impact of the pandemic-related disruptions.  sponsors should make every effort to minimize missing data without compromising safety of participants and study personnel and without placing undue burden on the healthcare system. priority should be on the assessments which determine the primary endpoint, important safety endpoints, followed by the key secondary efficacy endpoints.  most data that are missing due to pandemic reasons may be argued to be mcar or mar, especially if missingness is due to structural reasons, but additional considerations may apply, especially for certain diseases and participant-specific missingness.  sponsors should carry out rigorous and systematic risk assessment concerning trial and data integrity and update it regularly. the ability of trials to meet their objectives should be assessed quantitatively, taking account of the impacts on trial estimands, missing data and missing data handling, and needed modifications to analysis methods. european medicines agency committee for medicinal products for human use (ema/chmp guidance to sponsors on how to manage clinical trials during the covid- pandemic food and drug administration (fda) ( ), guidance for clinical trial sponsors: establishment and operation of clinical trial data monitoring committees guidance on conduct of clinical trials of medical products during covid- public health emergency, guidance for industry, investigators, and institutional review boards addendum on estimands and sensitivity analysis in clinical trials to the guideline on statistical principles for clinical trials general principles for planning and design of multiregional clinical trials survival analysis with interval-censored data: a practical approach estimation of treatment effect under non-proportional hazards and conditionally independent censoring analysis of longitudinal trials with protocol deviation: a framework for relevant, accessible assumptions, and inference via multiple imputation comparison of imputation variance estimators simple procedures for blinded sample size adjustment that do not affect the type i error rate aligning estimators with estimands in clinical trials: putting the ich e (r ) guidelines into practice estimands, estimators, and sensitivity analysis in clinical trials on the propensity score weighting analysis with survival outcome: estimands, estimation, and inference missing data in clinical studies statistical analysis with missing data a general statistical principle for changing a design any time during the course of a trial robust inference in discrete hazard models for randomized clinical trials the prevention and treatment of missing data in clinical trials clinical trials with missing data: a guide for practitioners unplanned adaptations before breaking the blind missing data in clinical trials: from clinical assumptions to statistical analysis using pattern mixture models multiple imputation for nonresponse in surveys treatment effect quantification for time-to-event endpoints -estimands, analysis strategies, and beyond bootstrap inference when using multiple imputation on estimands and the analysis of adverse events in the presence of varying follow-up times within the benefit assessment of therapies time-dependent bias was common in survival analyses published in leading clinical journals use of historical control data for assessing treatment effects in clinical trials group sequential and confirmatory adaptive designs in clinical trials we are grateful for the help of colleagues at each of our companies who have devoted much time to addressing these issues in their ongoing clinical trials. they have generously shared their ideas, and this manuscript has benefited from this broad input. we also thank the members of the "biopharmaceutical statistics leaders consortium" who brought the team of authors together and provided valuable input; and the associate editor and reviewers who provided extensive and helpful input within tight timelines. key: cord- - xsh pis authors: juul, sophie; nielsen, emil eik; feinberg, joshua; siddiqui, faiza; jørgensen, caroline kamp; barot, emily; nielsen, niklas; bentzer, peter; veroniki, areti angeliki; thabane, lehana; bu, fanlong; klingenberg, sarah; gluud, christian; jakobsen, janus christian title: interventions for treatment of covid- : a living systematic review with meta-analyses and trial sequential analyses (the living project) date: - - journal: plos med doi: . /journal.pmed. sha: doc_id: cord_uid: xsh pis background: coronavirus disease (covid- ) is a rapidly spreading disease that has caused extensive burden to individuals, families, countries, and the world. effective treatments of covid- are urgently needed. methods and findings: this is the first edition of a living systematic review of randomized clinical trials comparing the effects of all treatment interventions for participants in all age groups with covid- . we planned to conduct aggregate data meta-analyses, trial sequential analyses, network meta-analysis, and individual patient data meta-analyses. our systematic review is based on preferred reporting items for systematic reviews and meta-analysis (prisma) and cochrane guidelines, and our -step procedure for better validation of clinical significance of meta-analysis results. we performed both fixed-effect and random-effects meta-analyses. primary outcomes were all-cause mortality and serious adverse events. secondary outcomes were admission to intensive care, mechanical ventilation, renal replacement therapy, quality of life, and nonserious adverse events. we used grading of recommendations assessment, development and evaluation (grade) to assess the certainty of evidence. we searched relevant databases and websites for published and unpublished trials until august , . two reviewers independently extracted data and assessed trial methodology. we included randomized clinical trials enrolling a total of , participants. all trials were at overall high risk of bias. we identified one trial randomizing , participants to dexamethasone versus standard care. this trial showed evidence of a beneficial effect of dexamethasone on all-cause mortality (rate ratio . ; % confidence interval [ci] . – . ; p < . ; low certainty) and on mechanical ventilation (risk ratio [rr] . ; % ci . – . ; p = . ; low certainty). it was possible to perform meta-analysis of comparisons. meta-analysis showed no evidence of a difference between remdesivir versus placebo on all-cause mortality (rr . ; % ci . – . ; p = . , i( ) = %; trials; very low certainty) or nonserious adverse events (rr . ; % ci . – . ; p = . , i( ) = %; trials; low certainty). meta-analysis showed evidence of a beneficial effect of remdesivir versus placebo on serious adverse events (rr . ; % ci . – . ; p = . , i( ) = %; trials; very low certainty) mainly driven by respiratory failure in one trial. meta-analyses and trial sequential analyses showed that we could exclude the possibility that hydroxychloroquine versus standard care reduced the risk of all-cause mortality (rr . ; % ci . – . ; p = . ; i( ) = %; trials; low certainty) and serious adverse events (rr . ; % ci . – . ; p = . ; i( ) = %; trials; low certainty) by % or more, and meta-analysis showed evidence of a harmful effect on nonserious adverse events (rr . ; % ci . – . ; p < . ; i( ) = %; trials; very low certainty). meta-analysis showed no evidence of a difference between lopinavir–ritonavir versus standard care on serious adverse events (rr . ; % ci . – . ; p = . , i( ) = %; trials; very low certainty) or nonserious adverse events (rr . ; % ci . – . ; p = . , i( ) = %; trials; very low certainty). meta-analysis showed no evidence of a difference between convalescent plasma versus standard care on all-cause mortality (rr . ; % ci . – . ; p = . , i( ) = %; trials; very low certainty). five single trials showed statistically significant results but were underpowered to confirm or reject realistic intervention effects. none of the remaining trials showed evidence of a difference on our predefined outcomes. because of the lack of relevant data, it was not possible to perform other meta-analyses, network meta-analysis, or individual patient data meta-analyses. the main limitation of this living review is the paucity of data currently available. furthermore, the included trials were all at risks of systematic errors and random errors. conclusions: our results show that dexamethasone and remdesivir might be beneficial for covid- patients, but the certainty of the evidence was low to very low, so more trials are needed. we can exclude the possibility of hydroxychloroquine versus standard care reducing the risk of death and serious adverse events by % or more. otherwise, no evidence-based treatment for covid- currently exists. this review will continuously inform best practice in treatment and clinical research of covid- . dexamethasone versus standard care. this trial showed evidence of a beneficial effect of dexamethasone on all-cause mortality (rate ratio . ; % confidence interval [ci] . - . ; p < . ; low certainty) and on mechanical ventilation (risk ratio [rr] . ; % ci . - . ; p = . ; low certainty). it was possible to perform meta-analysis of comparisons. meta-analysis showed no evidence of a difference between remdesivir versus placebo on all-cause mortality (rr . ; % ci . - . ; p = . , i = %; trials; very low certainty) or nonserious adverse events (rr . ; % ci . - . ; p = . , i = %; trials; low certainty). meta-analysis showed evidence of a beneficial effect of remdesivir versus placebo on serious adverse events (rr . ; % ci . - . ; p = . , i = %; trials; very low certainty) mainly driven by respiratory failure in one trial. meta-analyses and trial sequential analyses showed that we could exclude the possibility that hydroxychloroquine versus standard care reduced the risk of all-cause mortality (rr . ; % ci . - . ; p = . ; i = %; trials; low certainty) and serious adverse events (rr . ; % ci . - . ; p = . ; i = %; trials; low certainty) by % or more, and meta-analysis showed evidence of a harmful effect on nonserious adverse events (rr . ; % ci . - . ; p < . ; i = %; trials; very low certainty). meta-analysis showed no evidence of a difference between lopinavir-ritonavir versus standard care on serious adverse events (rr . ; % ci . - . ; p = . , i = %; trials; very low certainty) or nonserious adverse events (rr . ; % ci . - . ; p = . , i = %; trials; very low certainty). meta-analysis showed no evidence of a difference between convalescent plasma versus standard care on all-cause mortality (rr . ; % ci . - . ; p = . , i = %; trials; very low certainty). five single trials showed statistically significant results but were underpowered to confirm or reject realistic intervention effects. none of the remaining trials showed evidence of a difference on our predefined outcomes. because of the lack of relevant data, it was not possible to perform other meta-analyses, network meta-analysis, or individual patient data meta-analyses. the main limitation of this living review is the paucity of data currently available. furthermore, the included trials were all at risks of systematic errors and random errors. our results show that dexamethasone and remdesivir might be beneficial for covid - patients, but the certainty of the evidence was low to very low, so more trials are needed. we can exclude the possibility of hydroxychloroquine versus standard care reducing the risk of death and serious adverse events by % or more. otherwise, no evidence-based treatment for covid- currently exists. this review will continuously inform best practice in treatment and clinical research of covid- . • there is a need for a living systematic review evaluating the beneficial and harmful effects of all possible interventions for treatment of covid- . • we conducted the first edition of a living systematic review with meta-analyses and trial sequential analyses to compare the effects of all treatment interventions for covid- . • one single trial randomizing , patients showed evidence of a beneficial effect of dexamethasone versus standard care on all-cause mortality and mechanical ventilation. • meta-analysis of trials showed evidence of a beneficial effect of remdesivir versus placebo on serious adverse events, but we found no effects on all-cause mortality or adverse events considered not serious. • meta-analysis of trials showed evidence of a harmful effect of hydroxychloroquine on adverse events considered nonserious and did not seem to have any effect on all-cause mortality or serious adverse events. in , a novel coronavirus named severe acute respiratory syndrome coronavirus (sars--cov- ) caused an international outbreak of the respiratory illness (coronavirus disease ) [ ] . since the initial outbreak in china, sars-cov- has spread globally, and covid- is labeled a public health emergency of global concern by the world health organization [ ] . the full spectrum of covid- ranges from subclinical infection over mild, selflimiting respiratory tract illness to severe progressive pneumonia, multiorgan failure, and death [ ] . severe disease onset might result in death because of massive alveolar damage and progressive respiratory failure [ ] [ ] [ ] . no evidence-based treatment for covid- currently exist to augment widely used supportive care protocols [ ] . to control the growing covid- pandemic, we rely on quarantine, isolation, and infection-control measures to prevent disease spread [ ] and on supportive care, including oxygen and mechanical ventilation, for infected patients. many randomized clinical trials assessing the effects of different potential treatments for covid- are currently underway. however, a single trial can rarely validly assess the effects of any intervention, and there is an urgent need to continuously surveil and update the aggregated evidence base so that effective interventions, if such exist, are implemented in clinical practice [ ] . the present living systematic review with aggregate meta-analyses and trial sequential analyses aims to continuously inform evidence-based guideline recommendations for the treatment of covid- , taking risks of systematic errors ("bias"), risks of random errors ("play of chance"), and certainty of the findings into consideration [ ] . two authors (een and jf) independently screened relevant trials. six authors in pairs (sj, een, jf, fs, ckj, eb) independently extracted data using a standardized data extraction sheet. any discrepancies were resolved through discussion, or if required, through discussion with a third author (jcj). we contacted all trial authors if relevant data were unclear or missing. risk of bias was assessed with the cochrane risk of bias tool-version (rob ) [ , ] . six authors in pairs (sj, een, jf, fs, ckj, eb) independently assessed risk of bias. any discrepancies were resolved through discussion or, if required, through discussion with a third author (jcj). bias was assessed with the following domains: bias arising from the randomization process, bias due to deviations from the intended interventions, bias due to missing outcome data, bias in measurement of outcomes, and bias arising from selective reporting of results [ , ] . we contacted authors of trials with unclear or missing data. primary and secondary outcomes were predefined in our protocol [ ] . primary outcomes were all-cause mortality and serious adverse events (as defined by the international conference on harmonation-good clinical practice [ich-gcp] guidelines) [ , ] . secondary outcomes were admission to intensive care (as defined by trialists), receipt of mechanical ventilation (as defined by trialists), receipt of renal replacement therapy (as defined by trialists), quality of life, and nonserious adverse events. we classified nonserious adverse events as any adverse event not assessed as serious according to the ich-gcp definition. we chose to add time to clinical improvement as a post hoc outcome. we planned several subgroup analyses, which are described in detail in our protocol [ ] . for all outcomes, we used the trial results reported at maximum follow-up. we performed our aggregate data meta-analyses according to cochrane [ ] , keus and colleagues [ ] , and the -step assessment by jakobsen and colleagues [ ] for better validation of meta-analytic results in systematic reviews. review manager version . (the nordic cochrane centre, the cochrane collaboration, copenhagen, denmark) [ ] and stata (statacorp llc, college station, tx, usa) [ ] were used for all statistical analyses. we used risk ratios (rrs) for dichotomous outcomes. we planned to calculate the mean differences (mds) and standardized mean difference (smd) with % confidence intervals (ci) for continuous outcomes. we assessed a total of primary outcomes, and we therefore adjusted our thresholds for significance [ ] and considered a p-value of . or less as the threshold for statistical significance [ , ] . because we primarily considered results of secondary outcomes as hypothesis generating, we did not adjust the p-value for secondary outcomes. we conducted both randomeffects (inverse variance, dersimonian-laird) and fixed-effect (mantel-haenszel) meta-analyses for all analyses and chose the most conservative result as our primary result [ , , , ] . we used trial sequential analysis to control for random errors [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] . trial sequential analysis estimates the diversity-adjusted required information size (daris), which is the number of participants needed in a meta-analysis to detect or reject a certain intervention effect. statistical heterogeneity was quantified by calculating inconsistency (i square) for traditional meta-analyses and diversity (d square) for trial sequential analysis. we used grade to assess the certainty of evidence. we downgraded imprecision in grade by levels if the accrued number of participants were below % of the daris and one level if between % and % of daris. we did not downgrade if benefit, harm, futility, or daris were reached. trial sequential analysis will also be used in future updates to adjust the thresholds for significance according to repetitive testing. we used fisher's exact test to calculate p-values for all single trial results. on august , , our literature searches identified , records after duplicates were removed. we included a total of clinical trials randomizing , participants (fig ) . we identified several trials, including participants suspected of covid- [ , ] . none of the trials reported separate data on covid- positive participants compared to the remaining participants. we included trials if approximately % or more participants had a confirmed covid- diagnosis. we wrote to all authors requesting separate data on covid- confirmed participants, but we have received no responses yet. for at detailed overview of excluded trials, see s table. characteristics of included trials and the trial results can be found in s table. all trials were assessed as at high risk of bias (s table) . one trial compared dexamethasone versus standard care [ , ] . two trials compared remdesivir versus placebo [ , ] . two trials compared lopinavir-ritonavir added to standard care versus standard care alone [ , ] . eight trials compared hydroxychloroquine added to standard care versus standard care alone [ , , , , , , , ] . two trials compared convalescent plasma added to standard care versus standard care alone [ , ] . the remaining trials and comparisons included hydroxychloroquine with and without azithromycin added to standard care versus standard care alone [ ] , hydroxychloroquine versus placebo [ , ] , hydroxychloroquine and chloroquine added to standard care versus standard care alone [ ] , methylprednisolone added to standard care versus standard care alone [ ] , lopinavir-ritonavir versus umifenovir and versus standard care [ ] , favipravir versus umifenovir [ ] , high-flow nasal oxygenation versus standard mask oxygenation prior to fibreotic tracheal intubation [ ] , α-lipoic acid versus placebo [ ] , the combination of novaferon plus lopinavir-ritonavir versus novaferon and versus lopinavir-ritonavir [ ] , baloxavir marboxil versus favipiravir and versus standard care [ ] , versus days of remdesivir [ ] , interferon β- a added to standard care versus standard care alone [ ] , colchicine added to standard care versus standard care alone [ ] , high-dosage with low-dosage chloroquine diphosphate [ ] , intravenous immunoglobulin added to standard care versus standard care alone [ ] , ribavirin plus interferon alpha versus lopinavir/ritonavir plus interferon alpha versis ribavirin plus lopinavir/ritonavir plus interferon alpha [ ] , darunavir/cobicistat plus interferon alpha- b versus interferon alpha- b alone [ ] , lincomycin hcl versus azitromycin [ ] , mtc-methyl diphosphonate ( mtc-mdp) injection added to standard care versus standard care alone [ ] , and interferon alpha- b plus gamma versus interferon alpha- b alone [ ] . the maximum follow-up time ranged from [ , ] to [ , ] days after randomization. for several of our outcomes, it was not possible to conduct meta-analysis because of insufficient data. we identified one trial, the randomised evaluation of covid- therapy (recovery) trial, randomizing , participants to dexamethasone versus standard care [ , ] . maximum follow-up was days after randomization. the trial was assessed at high risk of bias (s table) , and the certainty of evidence was assessed at "low" for all-cause mortality, serious adverse events, and mechanical ventilation (s table) . the living project • all-cause mortality: / , died in the dexamethasone group compared with , / , in the standard care group (age-adjusted rate ratio, . ; % ci . - . ; p < . ). • serious adverse events: / , experienced one or more serious adverse events in the dexamethasone group compared with , / , in the standard care group (age-adjusted rate ratio, . ; % ci . - . ; p < . ). this data is based on mortality data only, as suggested by the ich-gcp definition of a serious adverse event [ ] . • intensive care: no data. • mechanical ventilation: / , received invasive mechanical ventilation in the dexamethasone group compared with / , in the standard care group (rr . ; % ci . - . ). • renal replacement therapy: no data. • quality of life: no data. • nonserious adverse events: no data. we identified another trial randomizing participants to a different glucocorticoid (methylprednisolone) than the recovery trial [ ] . it was not possible to perform meta-analysis, as approximately half of the participants in the experimental group were nonrandomized [ ] . we have contacted the trial authors and asked for separate data for all randomized participants, but we have not received a response yet. we identified trials comparing remdesivir versus placebo [ , ] . both trials were assessed as at high risk of bias (s table) . random-effects meta-analysis showed no evidence of a difference between remdesivir versus placebo on all-cause mortality (rr . ; % ci . - . ; p = . , i = %; trials; very low certainty) (fig ; s table) . visual inspection of the forest plot and measures to quantify heterogeneity (i = %) indicated heterogeneity. the outcome was assessed days after randomization in the first trial [ ] and days after randomization in the second trial [ ] . trial sequential analysis showed that we did not have enough information to confirm or reject that remdesivir versus placebo reduces all-cause mortality with a relative risk reduction of % [ ] (fig ) . the fixed-effect meta-analysis showed evidence of a beneficial effect of remdesivir versus placebo on all-cause mortality (rr . ; % ci . - . ; p = . ; i = %; trials; very low certainty) (s fig). random-effects meta-analysis showed evidence of a beneficial effect of remdesivir versus placebo on serious adverse events (rr . ; % ci . - . ; p = . , i = %; trials; very low certainty) (fig ; s table) . visual inspection of the forest plot and measures to quantify heterogeneity (i = %) indicated no heterogeneity. the outcome was assessed days after randomization in the first trial [ ] and days after randomization in the second trial [ ] . trial sequential analysis showed that we did not have enough information to confirm or reject that remdesivir versus placebo reduced the risk of serious adverse events with a relative risk reduction of % [ ] (fig ) . the difference between groups was mainly driven by a difference in respiratory failure. random-effects meta-analysis showed no evidence of a difference between remdesivir versus placebo on adverse events not considered serious (rr . ; % ci . - . ; p = . , i = trial sequential analysis on remdesivir versus placebo on all-cause mortality in trials at high risk of bias. the daris was calculated based on a mortality proportion in the control group of . %; risk ratio reduction of % in the experimental group; type i error of . %; and type ii error of % ( % power). diversity was %. the required information size was , participants. the cumulative z-curve (blue line) did not cross the trial sequential monitoring boundaries for benefit or harm. the cumulative z-curve did not cross the inner-wedge futility line (red outward sloping lines nor the daris). the green dotted line shows conventional boundaries (alpha %). daris, diversity-adjusted required information size; pc, proportion of participants in control group; rrr, relative risk reduction. table) . visual inspection of the forest plot and measures to quantify heterogeneity (i = %) indicated no major heterogeneity. the outcome was assessed days after randomization in the first trial [ ] and days after randomization in trial sequential analysis on remdesivir versus placebo on serious adverse events in trials at high risk of bias. the daris was calculated based on a proportion in the control group of . %; risk ratio reduction of % in the experimental group; type i error of . %; and type ii error of % ( % power). diversity was %. the required information size was , participants. the cumulative z-curve (blue line) did not cross the trial sequential monitoring boundaries for benefit or harm (red inward sloping lines). the cumulative z-curve did not cross the inner-wedge futility line (red outward sloping lines nor the daris). the green dotted line shows conventional boundaries (alpha %). daris, diversity-adjusted required information size; pc, proportion of participants in control group; rrr, relative risk reduction. the second trial [ ] . trial sequential analysis showed that we had enough information to reject that remdesivir versus placebo reduced the risk of nonserious adverse events with a relative risk reduction of % [ ] (s fig). random-effects meta-analysis showed evidence of a beneficial effects of remdesivir versus placebo on time to clinical improvement/recovery (log ratio of means − . ; % ci − . to − . ; p = . ; i = %; trials; very low certainty) (s fig). the trials defined this outcome differently. the first trial analyzed "time to recovery" defined as either discharge from the hospital or hospitalization for infection-control purposes only [ ] . the second trial analyzed "time to clinical improvement" defined as the time (in days) from randomization to the point of a decline of levels on a -point ordinal scale of clinical status (from = discharged to = death) or discharged alive from hospital, whichever came first [ ] . visual inspection of the forest plot and measures to quantify heterogeneity (i = %) indicated no heterogeneity. the outcome was assessed days after randomization in the first trial [ ] and days after randomization in the second trial [ ] . one trial assessing the effects of remdesivir reported assessment of viral load. the trial result indicated no evidence of a difference from day to day [ ] . we identified trials comparing hydroxychloroquine added to standard care versus standard care alone [ , , , , , , , ] . we also identified one trial that used placebo as an additional control intervention [ ] . all trials were assessed as at high risk of bias (s table) . one trial was not eligible for meta-analysis, as the results were not reported in a usable way; i.e., the results were reported as per-protocol, and several participants crossed over [ ] . fixed-effect meta-analysis showed no evidence of a difference between hydroxychloroquine versus standard care on all-cause mortality (rr . ; % ci . - . ; p = . ; i = %; trials; low certainty) (s fig, s table) . visual inspection of the forest plot and measures to quantify heterogeneity (i = %) indicated no heterogeneity. the assessment time points varied from [ ] to [ , , ] days after randomization. the trial sequential analysis showed that we had enough information to reject that hydroxychloroquine compared with standard care reduces all-cause mortality with a relative risk reduction of % (s fig). fixed-effect meta-analysis showed no evidence of a difference between hydroxychloroquine versus standard care on serious adverse events (rr . ; % ci . - . ; p = . ; i = %; trials; low certainty) (s fig, s table) . visual inspection of the forest plot and measures to quantify heterogeneity (i = %) indicated no heterogeneity. the assessment time points varied from [ ] to [ , , ] days after randomization. the trial sequential analysis showed that we had enough information to reject that hydroxychloroquine compared with standard care reduces all-cause mortality with a relative risk reduction of % (s fig). fixed-effect meta-analysis showed evidence of a beneficial effect of standard care on adverse events not considered serious (rr . ; % ci . - . ; p < . ; i = %; trials; very low certainty) (s fig, s table) . visual inspection of the forest plot and measures to quantify heterogeneity (i = %) indicated large heterogeneity. the assessment time points varied from [ , , ] to [ , ] days after randomization. it was not possible to perform trial sequential analysis due to high diversity [ ] . we identified trials comparing lopinavir-ritonavir added to standard care versus standard care alone [ , ] . both trials were assessed as at high risk of bias (s table) . random-effects meta-analysis showed no evidence of a difference between lopinavir-ritonavir versus standard care on serious adverse events (rr . ; % ci . - . ; p = . , i = %; trials; very low certainty) (s fig, s table) . visual inspection of the forest plot and measures to quantify heterogeneity (i = %) indicated no heterogeneity. the assessment time point was days after randomization in the first trial [ ] and days after randomization in the second trial [ ] . trial sequential analysis showed that we did not have enough information to confirm or reject that lopinavir-ritonavir versus standard care reduced the risk of serious adverse events with a relative risk reduction of % [ ] (s fig). random-effects meta-analysis showed no evidence of a difference between lopinavir-ritonavir versus standard care on adverse events not considered as serious (rr . ; % ci . - . ; p = . , i = %; trials; very low certainty) (s fig; s table) . visual inspection of the forest plot and measures to quantify heterogeneity (i = %) indicated substantial heterogeneity. the assessment time point was days after randomization in the first trial [ ] and days after randomization in the second trial [ ] . trial sequential analysis showed that we did not have enough information to confirm or reject that lopinavir-ritonavir compared with standard care reduces nonserious adverse events with a relative risk reduction of % [ ] . we identified trials comparing convalescent plasma added to standard care versus standard care alone [ , ] . both trials were assessed as at high risk of bias (s table) . random-effects meta-analysis showed no evidence of a difference between convalescent plasma versus standard care on all-cause mortality (rr . ; % ci . - . ; p = . , i = %; trials; very low certainty) (s fig; s table) . visual inspection of the forest plot and measures to quantify heterogeneity (i = %) indicated no heterogeneity. the outcome was assessed days after randomization in the first trial [ ] and days after randomization in the second trial [ ] . trial sequential analysis showed that we did not have enough information to confirm or reject that convalescent plasma reduces all-cause mortality with a relative risk reduction of % [ ] . because of a lack of relevant data, it was not possible to conduct other meta-analyses, individual patient data meta-analyses, or network meta-analysis. five single trials showed statistically significant results but were underpowered to confirm or reject realistic intervention effects. one trial randomizing participants compared versus days of remdesivir showed evidence of a beneficial effect of days of remdesivir on serious adverse events (p = . [fisher's exact test]) [ ] . one trial randomizing participants compared the immunomodulator interferon β- a added to standard care versus standard care alone showed evidence of a beneficial effect of interferon β- a on all-cause mortality (p = . ) [ ] . this trial also showed evidence of a beneficial effect of standard care on nonserious adverse events (p = . ) [ ] . one single trial randomizing participants compared high-dosage versus low-dosage chloroquine diphosphate showed evidence of a beneficial effect of low-dosage chloroquine on all-cause mortality (p = . ) [ ] . one single trial randomizing participants compared colchicine added to standard care versus standard care alone showed evidence of a beneficial effect of standard care on adverse events not considered serious (p = . ) [ ] . one single group trial randomizing participants to hydroxychloroquine with or without azithromycin versus standard care showed evidence of a harmful effect of hydroxychloroquine with azithromycin on adverse events not considered serious (p = . ) [ ] . none of the remaining single trial results showed evidence of a difference on our predefined review outcomes. two trials did not report the results in a usable way; one trial reported results of the experimental group with a proportion of participants being nonrandomized [ ] , and the second trial reported the results as per-protocol, and there was participant crossover [ ] . three trials did not report on our review outcomes [ , , ] . we have contacted all corresponding authors, but we have not been able to obtain outcomes for our analyses from the trialists yet. all trials were assessed as at high risk of bias (s table) . characteristics of the trials and their results on the review outcomes can be found in s table. certainty of the evidence was assessed as "low" or "very low" for all outcomes (s -s tables). on august , , a search on the cochrane covid- study register revealed , registered randomized clinical trials [ ] . from these, different interventions for treatment of covid- patients were identified [ ] . the most investigated experimental interventions were hydroxychloroquine ( trials), convalescent plasma ( trials), azithromycin ( trials), lopinavir and ritonavir ( trials), tocilizumab ( trials), chloroquine ( trials), favipiravir ( trials), remdesivir ( trials), sarilumab ( trials), and dexamethasone ( trials). eligible trials will continuously be included in the present living systematic review once results become available. we also identified one press release from the recovery trial that investigates a number of potential treatments for covid- versus standard care, including dexamethasone and hydroxychloroquine [ , , , , ] . the trial also investigates lopinavir-ritonavir (n = , ) versus standard care (n = , ). according to the press release, there was no significant difference in the primary outcome of -day mortality ( we performed the first edition of our living systematic review assessing the beneficial and harmful effects of all treatment interventions for covid- . we included trials, randomizing a total of , participants to an experimental versus a control intervention. our study showed that dexamethasone and remdesivir might be beneficial for covid- patients, but the certainty of the evidence was low to very low, so more trials are needed. we could reject that hydroxychloroquine is beneficial for covid- in reducing death and serious adverse events at the % relative risk reduction level. we identified one trial randomizing , participants to dexamethasone versus standard care. this trial showed evidence of a beneficial effect of dexamethasone on all-cause mortality and mechanical ventilation [ ] . however, the trial was assessed as at high risk of bias and had methodological limitations (s table, s table) . first, the trial is described as an adaptive trial, but the reporting of the trial does not comply with the consolidated standards of reporting trials (consort) extension on adaptive designs [ ] . for example, according to the consort extension, trialists should report: "elements of decision-making rules describing whether, how, and when the proposed trial adaptations will be used during the trial. it involves pre-specifying a set of actions guiding how decisions about implementing the trial adaptations are made given interim observed data (decision rules). it also involves pre-specifying limits or parameters to trigger trial adaptations (decision boundaries)" [ ] . however, in the protocol and the statistical analysis plan of the recovery trial, it is stated that the data monitoring committee will request data at a frequency relevant to the emerging data, and no other information is provided. third, apart from mortality, the trial did not assess other serious or nonserious adverse events as defined by the ich-gcp guidelines, which limits the validity of the trial. in the trial protocol, it is stated that suspected serious adverse reactions as well as suspected unexpected serious adverse reactions will be recorded, whereas other serious and nonserious adverse events will not be recorded. it is of utmost importance to always assess beneficial and harmful intervention effects on patient-important outcomes. fourth, the authors emphasize large beneficial effects of dexamethasone on mortality in certain specific subgroups of participants (participants receiving invasive mechanical ventilation or oxygen at baseline). however, these findings should be interpreted with caution because the trialists did not adjust the threshold for significance according to the multiple comparisons including these subgroup analyses, which results in an increased risk of type i errors [ ] . furthermore, these subgroups were not prespecified in any of the trial registries or the trial protocol but appear only in the statistical analysis plan first dated june , , a day after the last participant was randomized to the dexamethasone group. fifth, only % in the dexamethasone group and % in the standard care group had a confirmed covid- diagnosis at randomization. a total of % in the dexamethasone group and % in the standard care group had a negative sars-cov- test result [ ] . the relatively large proportion of participants without a confirmed covid- diagnosis included in this trial should be considered when interpreting the trial results. sixth, the trial was open-label, and hence, the participants, treatment providers, and outcome assessors were not blinded, which might bias the trial results [ ] . these limitations need to be considered when interpreting the trial results. it was possible for us to perform meta-analyses. meta-analysis showed evidence of a beneficial effect of remdesivir versus placebo on serious adverse events mainly driven by a difference in respiratory failure in the largest trial [ ] . results of the largest trial indicated that remdesivir resulted in a median recovery time of days, compared with a median of days in the placebo group [ ] . one single trial compared versus days of remdesivir and showed evidence of a beneficial effect of days of remdesivir on serious adverse events [ ] . however, this trial was open-label, did not use blinded outcome assessors, and was assessed at high risk of bias. the effects of remdesivir on other patient-important outcomes are unclear. furthermore, certainty of the evidence was assessed as "very low" for all-cause mortality and serious adverse events and "low" for nonserious adverse events. hence, the effects of remdesivir need to be confirmed in future trials at low risk of bias. meta-analysis showed no evidence of a difference between hydroxychloroquine versus standard care on all-cause mortality and serious adverse events, and trial sequential analysis showed that we had enough information to reject that hydroxychloroquine reduces the risk of all-cause mortality and serious adverse events with a relative risk reduction of %. meta-analysis showed a harmful effect of hydroxychloroquine on nonserious adverse events. meta-analysis showed no evidence of a difference between lopinavir-ritonavir versus standard care on serious adverse events and on nonserious adverse events. meta-analysis showed no evidence of a difference between convalescent plasma versus standard care on all-cause mortality. because of lack of relevant data, it was not possible to perform other meta-analyses, network meta-analysis, or individual patient data meta-analyses. a single trial compared the immunomodulator interferon β- a added to standard care versus standard care alone and showed evidence of a beneficial effect of interferon β- a on allcause mortality (p = . ) [ ] . this trial also showed evidence of a beneficial effect of standard care alone on nonserious adverse events (p = . ). one single trial compared high-dosage with low-dosage chloroquine diphosphate [ ] showed evidence of a beneficial effect of low-dosage chloroquine diphosphate on all-cause mortality (p = . ). one single trial randomizing participants compared colchicine added to standard care versus standard care alone showed evidence of a beneficial effect of standard care on nonserious adverse events (p = . ). one single group trial randomizing participants to hydroxychloroquine with or without azithromycin versus standard care showed evidence of a harmful effect of hydroxychloroquine with azithromycin on adverse events not considered serious (p = . ) [ ] . however, these single trials were underpowered to confirm or reject realistic intervention effects, and they were assessed as at high risk of bias. therefore, the trial results should be interpreted with great caution [ ] . our living systematic review has a number of strengths. the predefined methodology was based on the cochrane handbook for systematic reviews of interventions [ ] , the -step assessment suggested by jakobsen and colleagues [ ] , and trial sequential analysis [ ] . hence, this review considers both risks of systematic errors and risks of random errors. another strength is the living systematic review design, which allows us to continuously surveil and update the evidence base of existing interventions for treatment of covid- , resulting in a decreased timespan from evidence to clinical practice. this is particularly important in this international healthcare crisis, in which a large number of new randomized clinical trials are continuously registered and published. our living systematic review also has limitations. the primary limitation is the paucity of trials currently available, and the results from the current meta-analyses are of low or very low certainty. this must be considered when interpreting our meta-analysis results. secondly, the trials that we succeeded in including were all at risks of systematic errors and random errors. third, it was not possible to perform the planned individual patient data meta-analyses, network-meta-analysis, or the planned subgroup analyses because of the lack of relevant data. we contacted all trial authors requesting individual patient data, but until now, we only received one dataset [ ] . fourth, we included "time to clinical improvement" as an outcome post hoc. results of this outcome should be interpreted with caution, because it was not predefined and was chosen after the trials were included in the systematic review. we did not include the outcome "time to clinical improvement" in our protocol because this outcome is poorly defined and if outcome assessors are not adequately blinded, assessments of "improvement" may be biased. furthermore, time to clinical improvement is not one of the most patient-important outcomes, e.g., most patients would rather survive without complications than recover a few days sooner. we chose to meta-analyze this outcome even though the trials defined this outcome differently, i.e., time to clinical improvement [ ] and time to clinical recovery [ ] (see "results"). hence, this outcome result should be interpreted with caution and should only be considered hypothesis generating. fifth, all trials differed in the included participants' disease severity at trial intake (mild, moderate, severe, critically ill), which might result in clinical heterogeneity. we will explore disease severity as a subgroup analysis if this is warranted in later stages of this living systematic review. sixth, the included trials assessed the outcomes at different time points, which might contribute to increased heterogeneity. seventh, some data are included from preprints, and these might be subject to change following peer review. therefore, some results, bias risk assessments, and grade summaries might change in later editions of this living systematic review following inclusion of the published peer-reviewed manuscripts. who has recently stopped a clinical trial of the antimalaria drug hydroxychloroquine for treating covid- patients [ ] . however, this decision applies only to the conduct of the who solidarity trial and does not apply to the clinical use or research evaluation of hydroxychloroquine in pre-or postexposure prophylaxis in patients exposed to covid- . the decision was based on the results of a nonrandomized study published by the lancet on hydroxychloroquine and chloroquine and its effects on hospitalized covid- patients [ ] . this study was recently retracted because of several authors questioning the validity of the data used in the study. based on our data, we can reject that hydroxychloroquine offers benefit to covid- patients. we have identified important reviews that are comparable to our present project [ ] [ ] [ ] [ ] . the first is a network meta-analysis just published in bmj [ ] . however, at the time of publication, this project did not include all relevant trials [ , , [ ] [ ] [ ] [ ] [ ] [ ] ] , including the pivotal trials assessing the effects of hydroxychloroquine [ , , ] . the second project is a literature review published in jama using pubmed to identify relevant english-language articles published through march , , on pharmacological interventions for covid- . the search resulted in , total articles. this is because the authors also included case reports, case series, and review articles if they lacked randomised clinical trials. moreover, this review was only a narrative review without meta-analytic methods and trial sequential analysis [ ] . the third project is a living mapping of ongoing randomized clinical trials with network meta-analysis on all interventions for covid- . the authors are producing and disseminating preliminary results through an open platform [ ] . this review includes both prevention and treatment and does not use trial sequential analysis or similar methods to handle problems with multiplicity (repeating updating of meta-analysis, multiple comparisons due to inclusion of multiple interventions, assessing multiple outcomes) [ ] . the fourth project is a preprint of a rapid review assessing the effectiveness and safety of antiviral antibody treatments for covid- published in medrxiv [ ] . fifty-four studies were included in the review: controlled trials, cohort studies, retrospective medical record/database studies, and case reports or series. these studies included patients with severe acute respiratory syndrome (sars, n = ), middle east respiratory syndrome (mers, n = ), covid- (n = ), and unspecified coronavirus (n = ). the most common treatment was ribavirin (n = ), followed by oseltamivir (n = ), and the combination of lopinavir/ritonavir (n = ). the authors conclude that current evidence for the effectiveness and safety of antiviral therapies for covid- is inconclusive and suffers from a lack of well-designed prospective trials. moreover, this review was only a narrative review without meta-analytic methods and trial sequential analysis [ ] . our study showed that dexamethasone and remdesivir might be beneficial for covid- patients, but the certainty of the evidence was low to very low, so more trials are needed. we could reject that hydroxychloroquine versus standard care reduces the risk of death and serious adverse events with %. otherwise, no evidence-based treatment for covid- currently exists. this review will continuously inform best practice in treatment and clinical research of covid- . there is an urgent need for additional evidence, especially trials assessing the effects of dexamethasone and remdesivir. we erroneously reported the adjusted trial sequential analysis alpha as % in our published protocol [ ] . this has now been corrected to . % according to primary outcomes [ ] . further, we included "time to clinical improvement" as an outcome post hoc. trial sequential analysis on remdesivir versus placebo on nonserious adverse events in high risk of bias trials. the daris was calculated based on an event rate in the control group of %; risk ratio reduction of % in the experimental group; type i error of . %; and type ii error of % ( % power). diversity was %. the required information size was , participants. the cumulative z-curve (blue line) did not cross the trial sequential monitoring boundaries for benefit or harm (red inward sloping lines). the cumulative z-curve crossed the innerwedge futility line (red outward sloping lines). the green dotted line shows conventional boundaries (alpha %). daris, diversity-adjusted required information size; pc, proportion of participants in control group; rrr, relative risk reduction. trial sequential analysis on hydroxychloroquine versus standard care on serious adverse events in high risk of bias trials. the daris was calculated based on an event rate in the control group of . %; risk ratio reduction of % in the experimental group; type i error of . %; and type ii error of % ( % power). diversity was %. the required information size was , participants. the cumulative z-curve (blue line) did not cross the trial sequential monitoring boundaries for benefit or harm (inward sloping red lines) nor the conventional naive boundaries. the cumulative z-curve crossed the inner-wedge futility line (red outward sloping red lines and the daris). the green dotted line shows conventional boundaries (alpha %). daris, diversity-adjusted required information size; pc, proportion of participants in control group; rrr, relative risk reduction. trial sequential analysis on hydroxychloroquine versus standard care on serious adverse events in high risk of bias trials. the daris was calculated based on an event rate in the control group of . %; risk ratio reduction of % in the experimental group; type i error of . %; and type ii error of % ( % power). diversity was %. the required information size was , participants. the cumulative z-curve (blue line) did not cross the trial sequential monitoring boundaries for benefit or harm (inward sloping red lines) nor the conventional naive boundaries. the cumulative z-curve crossed the inner-wedge futility line (red outward sloping red lines and the daris). the green dotted line shows conventional boundaries (alpha %). daris, diversity-adjusted required information size; pc, proportion of participants in control group; rrr, relative risk reduction. trial sequential analysis on lopinavir-ritonavir versus standard care on serious adverse events in high risk of bias trials. the daris was calculated based on an event rate in the control group of . %; risk ratio reduction of % in the experimental group; type i error of . %; and type ii error of % ( % power). diversity was %. the required information size was , participants. the cumulative z-curve (blue line) did not cross the trial sequential monitoring boundaries for benefit or harm (inward sloping red lines) nor the conventional naive boundaries. the cumulative z-curve did not cross the inner-wedge futility line (red outward sloping red lines nor the daris). the green dotted line shows conventional boundaries (alpha %). daris, diversity-adjusted required information size; pc, proportion of participants in control group; rrr, relative risk reduction. clinical characteristics of coronavirus disease in china world health organization. novel coronavirus ( -ncov) a trial of lopinavir-ritonavir in adults hospitalized with severe covid- clinical features of patients infected with novel coronavirus in wuhan a familial cluster of pneumonia associated with the novel coronavirus indicating person-to-person transmission: a study of a family cluster pathological findings of covid- associated with acute respiratory distress syndrome covid- -the search for effective therapy covid- -navigating the uncharted interventions for treatment of covid- : a protocol for a living systematic review with network meta-analysis including individual patient data (the living project) preferred reporting items for systematic reviews and meta-analyses: the prisma statement the prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration cochrane handbook for systematic reviews of interventions version . (updated cochrane covid- study register a real-time dashboard of clinical trials for covid- rob : a revised tool for assessing risk of bias in randomised trials guideline for good clinical practice (ich-gcp) evidence at a glance: error matrix approach for overviewing available evidence thresholds for statistical and clinical significance in systematic reviews with meta-analytic methods version . . copenhagen: the nordic cochrane centre, the cochrane collaboration statacorp. stata statistical software: release . college station, tx: statacorp llc cochrane handbook for systematic reviews of interventions. version . . [updated the cochrane collaboration being sceptical about meta-analyses: a bayesian perspective on magnesium trials in myocardial infarction trial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis trial sequential analysis reveals insufficient information size and potentially false positive results in many meta-analyses apparently conclusive meta-analyses may be inconclusive -trial sequential analysis adjustment of random error risk due to repetitive testing of accumulating data in apparently conclusive neonatal meta-analyses can trial sequential monitoring boundaries reduce spurious inferences from meta-analyses? estimating required information size by quantifying diversity in random-effects model meta-analyses user manual for trial sequential analysis (tsa) interpreting meta-analysis according to the adequacy of sample size. an example using isoniazid chemoprophylaxis for tuberculosis in purified protein derivative negative hivinfected individuals false-positive findings in cochrane meta-analyses with and without application of trial sequential analysis: an empirical review remdesivir for the treatment of covid- -preliminary report a trial of lopinavir-ritonavir in adults hospitalized with severe covid- favipiravir versus arbidol for covid- : a randomized clinical trial a pilot study of hydroxychloroquine in treatment of patients with common coronavirus disease- (covid- ) efficacy of hydroxychloroquine in patients with covid- : results of a randomized clinical trial efficacy and safety of interferon beta- a in treatment of severe covid- : a randomized clinical trial. medrxiv remdesivir for or days in patients with severe covid- triple combination of interferon beta- b, lopinavir-ritonavir, and ribavirin in the treatment of patients admitted to hospital with covid- : an open-label, randomised, phase trial effect of convalescent plasma therapy on time to clinical improvement in patients with severe and life-threatening covid- : a randomized clinical trial efficacy and safety of lopinavir/ritonavir or arbidol in adult patients with mild/ moderate covid- : an exploratory randomized controlled trial clinical outcomes and plasma concentrations of baloxavir marboxil and favipiravir in covid- patients: an exploratory randomized hydroxychloroquine in patients with mainly mild to moderate coronavirus disease : open label, randomised controlled trial remdesivir in adults with severe covid- : a randomised, double-blind, placebo-controlled, multicentre trial high-flow nasal-oxygenation-assisted fibreoptic tracheal intubation in critically ill patients with covid- pneumonia: a prospective randomised controlled trial a novel protein drug, novaferon, as the potential antiviral drug for covid- . medrxiv a randomized, single-blind, group sequential, active-controlled study to evaluate the clinical efficacy and safety of α-lipotic acid for critically ill patients with coronavirus disease effect of dexamethasone in hospitalized patients with covid- -preliminary report efficacy and safety of chloroquine or hydroxychloroquine in moderate type of covid- : a prospective open-label randomized controlled study effect of colchicine vs standard care on cardiac and inflammatory biomarkers and clinical outcomes in patients hospitalized with coronavirus disease : the grecco- randomized clinical trial effect of high vs low doses of chloroquine diphosphate as adjunctive therapy for patients hospitalized with severe acute respiratory syndrome coronavirus (sars-cov- ) infection convalescent plasma for covid- . a randomized clinical trial dexamethasone in hospitalized patients with covid- -preliminary report hydroxychloroquine in nonhospitalized adults with early covid- : a randomized trial hydroxychloroquine with or without azithromycin in mildto-moderate covid- hydroxychloroquine for early treatment of adults with mild covid- : a randomized-controlled trial glucocovid: a controlled trial of methylprednisolone in adults hospitalized with covid- pneumonia intravenous immunoglobulin (ivig) significantly reduces respiratory morbidity in covid- pneumonia: a prospective randomized trial effect of hydroxychloroquine in hospitalized patients with covid- : preliminary results from a multi-centre, randomized, controlled trial. medrxiv a multicenter, randomized, open-label, controlled trial to evaluate the efficacy and tolerability of hydroxychloroquine and a retrospective study in adult patients with mild to moderate coronavirus disease (covid- ) antiviral activity and safety of darunavir/cobicistat for the treatment of covid- comparative effectiveness and safety of ribavirin plus interferon-alpha, lopinavir/ritonavir plus interferon-alpha and ribavirin plus lopinavir/ritonavir plus interferonalpha in patients with mild to moderate novel coronavirus pneumonia: results of a randomized, openlabeled prospective study the comparison of the effectiveness of lincocin® and azitro® in the treatment of covid- -associated pneumonia: a prospective study pulmonary radiological change of covid- patients with mtc-mdp treatment. medrxiv effect and safety of combination of interferon alpha- b and gamma or interferon alpha- b for negativization of sars-cov- viral rna. preliminary results of a randomized controlled clinical trial. medrxiv ruxolitinib in treatment of severe coronavirus disease (covid- ): a multicenter, single-blind, randomized controlled trial febuxostat therapy in outpatients with suspected covid- : a clinical trial safety of hydroxychloroquine among outpatient clinical trial participants for covid- . medrxiv statement from the chief investigators of the randomised evaluation of covid- therapy (recov-ery) trial on lopinavir-ritonavir the adaptive designs consort extension (ace) statement: a checklist with explanation and elaboration guideline for reporting randomised trials that use an adaptive design statistical review of effect of dexamethasone in hospitalized patients with covid- -preliminary report association between risk-of-bias assessments and results of randomized trials in cochrane reviews: the robes meta-epidemiologic study the thresholds for statistical and clinical significance-a five-step procedure for evaluation of intervention effects in randomised clinical trials world health organization. coronavirus disease (covid- ) outbreak hydroxychloroquine or chloroquine with or without a macrolide for treatment of covid- : a multinational registry analysis drug treatments for covid- : living systematic review and network meta-analysis pharmacologic treatments for coronavirus disease (covid- ): a review living mapping and living systematic review of covid- studies effectiveness and safety of antiviral or antibody treatments for coronavirus: a rapid review we report this systematic review based on the preferred reporting items for systematic reviews and meta-analysis (prisma) guidelines (s text) [ , ] . the updated methodology used in this living systematic review is described in detail in the cochrane handbook of systematic reviews of interventions [ ] and our protocol [ ] , which was registered in the prospero database (id: crd ) prior to the systematic literature search. key: cord- -ho m nqn authors: nguyen, van thu; rivière, philippe; ripoll, pierre; barnier, julien; vuillemot, romain; ferrand, gabriel; cohen-boulkia, sarah; ravaud, philippe; boutron, isabelle; alawadhi, solaf; amer-yahia, sihem; Ávila, camila; bafeta, aïda; baudry, julia; bollig, claudia; bonnet, hillary; bouet, marinette; cabanac, guillaume; chaimani, anna; chavalarias, david; chen, yaolong; chevance, astrid; cohen-boulakia, sarah; coquery, emmanuel; conil, francoise; davidson, mauricia; de nale, laura; devane, declan; diard, elise; doreau, bastien; evrenoglou, theodoros; fabri, alice; feron, gilles; fezeu, leopold; fouet, mathilde; el chall, lina ghosn; graña, carolina; grasselli, giacomo; grolleau, françois; hacid, mohand-said; haddy, loubna; hansen, camilla; hohlfeld, ameer; hróbjartsson, asbjørn; julia, chantal; mavridis, dimitris; meerpohl, joerg j.; meyer, brice; naidoo, nivantha; thu, van nguyen; oikonomidi, theodora; pienaar, elizabeth; quirke, fiona; rada, gabriel; riveros, carolina; sauvant, marie; schmucker, christine; toumani, farouk; tovey, david; xia, jun; yu, xuan; zoletic, emina; zweigenbaum, pierre title: research response to covid- needed better coordination and collaboration: a living mapping of registered trials date: - - journal: j clin epidemiol doi: . /j.jclinepi. . . sha: doc_id: cord_uid: ho m nqn background researchers worldwide are actively engaging in research activities to search for preventive and therapeutic interventions against covid- . our aim was to describe the planning of randomized controlled trials (rcts) in terms of timing related to the course of the covid- epidemic and research question evaluated. method we performed a living mapping of rcts registered in the who international clinical trials registry platform. we systematically search the platform every week for all rcts evaluating preventive interventions and treatments for covid- and created a publicly available interactive mapping tool at https://covid-nma.com to visualize all trials registered. results by august , , , trials for covid- were registered worldwide. overall, the median ([q -q ]; range) delay between the first case recorded in each country and the first rct registered was days ([ - ]; - ). for the countries with the highest number of trials registered, most trials were registered after the peak of the epidemic (from % trials in italy to % in the united states). most trials evaluated treatments ( , trials; %); only ( %) evaluated preventive strategies and post-acute period intervention. a total of trials were planned to assess different regimens of hydroxychloroquine with an expected sample size of , patients. conclusion this living mapping analysis showed that covid- trials have relatively small sample size with certain redundancy in research questions. most trials were registered when the first peak of the pandemic have passed. researchers worldwide are actively engaging in research activities to search for preventive and therapeutic interventions against covid- . our aim was to describe the planning of randomized controlled trials (rcts) in terms of timing related to the course of the covid- epidemic and research question evaluated. we performed a living mapping of rcts registered in the who international clinical trials registry platform. we systematically search the platform every week for all rcts evaluating preventive interventions and treatments for covid- and created a publicly available interactive mapping tool at https://covid-nma.com to visualize all trials registered. by august , , , trials for covid- were registered worldwide. overall, the median ([q -q ]; range) delay between the first case recorded in each country and the first rct -there is a notable redundancy in research questions. -we have created a living mapping that visualizes all clinical trials of covid- . -the living mapping supports researchers and decision makers in identifying research gaps, thus planning research of high priority. -research community needs a better coordination in research planning to ensure that all potential treatments for covid- are evaluated with robust methodology. -the living mapping provides a tool to monitor status of research, enhance research collaboration and interaction in medical and scientific community to avoid research waste. j o u r n a l p r e -p r o o f in december , an outbreak of pneumonia caused by a novel coronavirus started in wuhan, hubei province in china. the disease was later determined to be sars-cov- infection, or covid- ( ). in early march , the disease had spread to more than countries and territories ( ). on march , , the world health organization (who) declared the outbreak a pandemic ( ) . to respond to this emergency, researchers all over the world began to actively engage in research activities to develop and evaluate preventive and therapeutic agents for covid- . given this unprecedented context, we aimed to inform decision makers and researchers in near real-time about current research efforts, research gaps and overlap. a mapping of all research efforts is imperative to support researchers and decision makers to monitor status of research response to the epidemic and integrate emerging evidence in research planning timely to ensure that all potential treatments are evaluated, whilst avoiding waste in resources invested. for this purpose, we performed a living mapping of all registered randomized controlled trials (rcts) investigating interventions to prevent and treat covid- . this living mapping is updated every week and results are publicly available at https://covid-nma.com/. this paper describes the planning of rcts in terms of timing related to the course of the pandemic and research questions. this mapping is part of the covid-nma project, which also includes living systematic reviews and living network meta-analyses of studies of covid- . the protocol of this project is available at https://zenodo.org/record/ #.xwlasubui x. our data are obtained from the who international clinical trials registry platform (ictrp) (https://www.who.int/ictrp/en/), an international registry that assembles information on clinical trials registered in primary registries ( ) . who ictrp has created a database dedicated to all clinical trials evaluating interventions to prevent and treat covid- . the database is updated weekly and is publicly available. whenever the database is updated, we use php programming language to identify studies that are newly registered in the database. two researchers (vn and gf) systematically search the platform every week to identify new eligible rcts for data extraction. all rcts assessing the efficacy and safety of interventions for preventing or treating covid- and patients in the postacute period are included. we exclude observational studies, case series, non-randomized or single arm studies (i.e., diagnostic tests studies). we also exclude studies ) evaluating interventions to reduce psychological distress caused by the covid- outbreak or ) assessing herbs, homeopathy therapy, traditional chinese medicine (tcm) (with only tcm in two groups, or tcm plus standard of care). a standardized data collection form is used to collect data describing the rcts. several data items are available from the who ictrp database, such as registration number, countries where trials are conducted, recruitment status, inclusion and exclusion criteria, primary outcomes, and sample size. a team of trained data collectors independently retrieve other information from j o u r n a l p r e -p r o o f the trial registration such as study aim, number of arms, type of participants, and information related to experimental treatments and comparators (i.e., treatment name, treatment type). two researchers (vn and gf) verify the quality of the data and ensure the consistency of data entered in the database. we classify study aims as evaluation of prevention interventions, covid- treatments and post-acute period interventions. in rcts evaluating preventive interventions, participants are classified as healthy volunteers, health workers, and high-risk patients. patients in rcts assessing covid- treatments are classified by disease severity (i.e., mild, moderate, severe and critical). clinical criteria for classifying disease severity are provided in appendix . the full list of treatment types is provided in appendix . when the database of who ictrp is updated every week, we use php programming language to identify rcts with changes in recruitment status (e.g., from not recruiting to recruiting) and update our database accordingly. the covid- database maintained by our world in data (https://ourworldindata.org/coronavirus-source-data) was used to visualize the evolution of the pandemic over time. the database is updated daily and includes the number of confirmed cases, deaths, and testing data. we considered only data related to the number of confirmed cases and deaths. we created an online interactive mapping tool to visualize the data of trials registered. the interactive mapping was developed with d .js ( ) as an observable notebook ( ) . the projection for the map used was implemented in javascript ( ). we also used time series plotting to visualize the evolution of covid- research over time. this visualization was performed in r v . . (the r foundation statistical computing, vienna, austria). up to august j o u r n a l p r e -p r o o f in certain countries, the sample size is relatively small for trials evaluating covid- treatments ( timing of research response to the evolution of the pandemic in europe, spain registered only / trials ( %) before the peak (i.e., march , . in france, the first trial was registered only days before the peak (i.e., april , ). eight trials ( %) registered before the peak in france accounted for % of the total number of patients to be recruited in all trials. in the united kingdom, / ( %) trials were registered before the peak (i.e., april , ), representing % of the total number of patients to be recruited in all trials. in italy, no trial was registered before the peak on march , . furthermore, trial results will probably be published after the epidemic has passed, and countries where the trials were conducted might not have the direct benefits to improve clinical practice at the time of the epidemic ( - ). the planning of trials in response to covid- has a notable redundancy in research questions. in march , hydroxychloroquine received tremendous attention after the results of an observational study in france were published that generated a huge debate ( ) . following this publication, the us president highlighted this treatment as being a "game changer", despite the regulatory and health authority should provide timely guidance to clinicians to avoid off-label drug uses based on anecdotal evidence which might cause difficulties to trial planning and recruitment ( , ) . this study highlights the importance of clinical trial registries, an underused resource, to monitor the state of research for improving the organization of research efforts ( ) ( ) ( ) . our interactive living mapping of covid- research was designed to help decision makers use data from clinical registries for an up-to-date picture of all research questions being investigated so as to prioritize research and avoid waste in research ( ) . furthermore, this interactive mapping tool might also enhance collaboration in research to reduce redundancy and competition in trial organization ( , ) . in this analysis, we visualized trial registration over time by using the registration date rather than the actual starting date of recruitment because these data were not available on the ictrp database. additionally, investigators might not regularly update the status of recruitment on trial registries. for example, the trial chictr was reported as "not recruiting" on the registry, but the results of the trial were published ( ) . furthermore, the structure of reporting is heterogeneous across the primary registries, which affects the quality of reporting ( ) . investigators might register one trial in more than one registry under different titles or investigator names, such duplicates are almost systematically detected by the ictrp while a very few may remain undetected. lastly, we did not assess the risk of bias for each trial registered as information in trial registration is inadequate to enable a comprehensive assessment. we have created a living mapping tool to keep track of the evolution of research on covid- for supporting decision makers in prioritizing and planning research. this mapping analysis showed that many covid- trials were registered after the first peak had passed and a need to j o u r n a l p r e -p r o o f improve the organization of research efforts to avoid research redundancy. visualizing ongoing research can enhance the collaboration and interaction between research communities that can go beyond the covid- crisis. funding: this study received funding from the agence nationale de la recherche (anr). the funder had no role in the design, analysis and reporting of this study. statement regarding cluster of pneumonia cases in wuhan director-general's opening remarks at the media briefing on covid- - who. who announces covid- outbreak a pandemic discrimination of changes in osteoporosis outcomes d³ data-driven documents can there be a more patient-centred approach to determining clinically important effect sizes for randomized treatment trials using stated preference discrete choice modelling to evaluate the introduction of varicella vaccination statement from the chief investigators of the randomised evaluation of covid- therapy (recovery) trial on hydroxychloroquine remdesivir for the treatment of covid- -preliminary report a trial of lopinavir-ritonavir in adults hospitalized with severe covid- efficacy and safety of chloroquine or hydroxychloroquine in moderate type of covid- : a prospective open-label randomized controlled study efficacy of hydroxychloroquine in patients with covid- : results of a randomized clinical trial hydroxychloroquine and azithromycin as a treatment of covid- : results of an open-label non-randomized clinical trial fda. coronavirus (covid- ) update: daily roundup when clinical trials compete: prioritising study recruitment covid- : the inside story of the recovery trial covid- : recovery trial will evaluate "antiviral antibody cocktail advancing preparedness for clinical research during infectious disease epidemics covid- clinical trials: learning from exceptions in the research chaos future of evidence ecosystem series: . introduction evidence synthesis ecosystem needs dramatic change impact of searching clinical trial registries in systematic reviews of pharmaceutical treatments: methodological systematic review and reanalysis of meta-analyses clinical trials registries are underused in the pregnancy and childbirth literature: a systematic review of the top journals how to increase value and reduce waste when research priorities are set against pandemic research exceptionalism clinical outcomes and plasma concentrations of baloxavir marboxil and favipiravir in covid- patients: an exploratory randomized the quality of registration of clinical trials: still a problem data sharing: the dataset will be publicly available after the paper is published. key: cord- -rpwccvqi authors: rojek, amanda m; moran, james; horby, peter w title: core minimal datasets to advance clinical research for priority epidemic diseases date: - - journal: clin infect dis doi: . /cid/ciz sha: doc_id: cord_uid: rpwccvqi the ebola virus disease outbreak in west africa has prompted significant progress in responding to the clinical needs of patients affected by emerging infectious disease outbreaks. among the noteworthy successes of vaccine trials, and the commendable efforts to implement clinical treatment trials during ebola outbreaks, we should also focus on strengthening the collection and curation of epidemiological and observational data that can improve the conception and design of clinical research. during the currently ongoing ebola virus disease (evd) outbreak in the democratic republic of congo, a clinical trial of potential treatments has commenced. this is a significant step toward improving outcomes for patients with the disease. ebola virus disease constitutes but one of the priority diseases that the world health organization (who), in their blueprint for action to prevent epidemics, suggests poses a severe public health risk and for which there are insufficient countermeasures [ ] . the purpose of this priority list is to identify high-threat pathogens for which there is a need to prioritize and advance the development of diagnostics, vaccines, and therapeutics. any diagnostics, drugs, or vaccines that are developed as a result of this and other initiatives, such as the coalition for epidemic preparedness innovation, will need to be fully evaluated in diagnostic evaluation studies or phase ii and iii clinical trials. however, due to the very nature of the epidemic-prone infectious diseases that appear in the who list of priority diseases, evaluation in clinical studies is challenging, not least because the epidemiology is unpredictable but also because the pathogenesis and natural history of many of these diseases are not well defined. for example, during the influenza a(h n )pdm pandemic, case fatality rate (cfr) estimates varied widely from to per laboratory-confirmed infections, with a heterogeneity of . % (using i estimate) [ ] . a therapeutic trial designed with patient survival as a primary outcome measure would have grossly misjudged the required sample size if the trial was designed using the wrong cfr. therapeutic trials for the prevention of congenital zika syndrome will be hindered by the absence of consistently used criteria to define the outcome of congenital malformations [ ] . for middle east respiratory syndrome coronavirus, a lack of systematic biological sampling means that disease pathophysiology and factors associated with more severe disease and viral clearance (a commonly used secondary outcome measure) are not well understood [ ] . the need for well-defined core minimal datasets for emerging infectious diseases is not a new observation. a decade ago sheila bird and jeremy farrar [ ] noted the need to define a core minimal dataset for human cases of avian influenza a/h n , yet there remains no systematic examination of the completeness of the core data needed to design and conduct trials for highpriority pathogens. table identifies some key domains that could contribute to a core minimal dataset that informs clinical trial design for each priority pathogen. the benefit of this approach, when complemented by scoring or assessment of the available information, is that it allows for initial bench-marking and triaging of unmet data needs in order to prioritize further data gathering activities. importantly, a harmonized data collection initiative can also prospectively embed data-sharing agreements into data-collection protocols. this will allow valuable clinical information to be readily available to stakeholders, while identifying and protecting the interests of those collecting data in regions where outbreaks occur. accumulation and curation of the data will depend on a variety of sources and methodology types, but it is critical that high-quality clinical data are highlighted as an integral component. often lost to competing priorities for clinicians during outbreaks, standardized data collection regarding the presentation and natural history of disease, biomarkers of disease severity, and response to supportive care can be sporadic or missing. while these data have their most important benefits in improving patient management (through better recognition of disease complications and informing supportive care) and public health control, patient-based data are also used to determine key parameters for clinical trials, such as the inclusion criteria, the nature and rate of clinically relevant outcomes, and potential confounders. we suggest that adoption of clinical case registries (such as those used for rare cancers) provides a feasible option to produce standardized clinical data that have multiple clinical, public health, and research benefits [ ] . compared with expensive and lengthy countermeasure development pipelines, improving the scale, relevance, and quality of observational data is likely to be an efficient and cost-effective strategy to improve global preparedness against epidemic and pandemic infections. disclaimer. the funders had no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the paper for publication. temporal and geographical profile of previous outbreaks this is required for logistical planning, to ensure that local teams are sufficiently trained in research practices (such as good clinical practice) and trial-specific equipment is available. an agreed-upon case definition clinical characteristics of the disease are used to define enrollment criteria. analysis of strength of evidence for factors associated with increased disease severity or fatality stratification (or other statistical adjustment) on the basis of severity is often required when interpreting the clinical trial outcome. best available descriptions of the type and rate of clinical outcomes clinical outcomes will function as a trial outcome measures. understanding the natural course of illness will also help differentiate disease course from adverse events from treatment. assessment of confidence in estimates of clinical outcomes heterogeneity in patient outcomes between or within outbreaks creates uncertainty for power calculations and will affect selection of a statistical design for a trial. spurious heterogeneity may occur due to random error in small cohorts, or represent ascertainment, lead-time, measurement, or follow-up bias. real heterogeneity can occur due to improvements in care over an outbreak, pathogen evolution, or changes in host susceptibility and vulnerability but should be adjusted for. analysis of known or suspected covariates of outcome highlights possible confounders that will alter outcome independently of treatment and that will require adjustment if unequally distributed between treatment and control arms. the mean time from onset of symptoms to outcome allows for an estimation of the feasibility and logistics of medical intervention. agreed-upon standards of care for patient treatment determines if there is standardized supportive therapy to be adopted in all arms of a trial. this is especially important for multicenter research the performance characteristics of the favored diagnostic method determines whether a trial will be performed on an itt basis or following laboratory confirmation. mean time for laboratory diagnosis determines whether a trial will be performed on an itt basis or following laboratory confirmation. community priorities and expectations for trials determines the priorities of affected communities in terms of access to trials, acceptable methodology, and acceptability of treatments or vaccines. abbreviation: itt, intention-to-treat. annual review of diseases prioritized under the research and development blueprint informal consultation case fatality risk of influenza a (h n pdm ): a systematic review clinical trials of therapeutics for the prevention of congenital zika virus disease: challenges and potential solutions development of medical countermeasures to middle east respiratory syndrome coronavirus minimum dataset needed for confirmed human h n cases a systematic review and meta-analysis of patient data from the west africa ( - ) ebola virus disease epidemic key: cord- -x b qphd authors: hopper, lydia m.; jacobson, sarah l.; howard, lauren h. title: problem solving flexibility across early development date: - - journal: j exp child psychol doi: . /j.jecp. . sha: doc_id: cord_uid: x b qphd cognitive flexibility allows individuals to adapt to novel situations. however, this ability appears to develop slowly over the first few years of life, mediated by task complexity and opacity. we used a physically simple novel task, previously tested with nonhuman primates, to explore the development of flexible problem solving in -, -, and -year-old children from a developmental and comparative perspective. the task goal was to remove barriers (straws) from a clear tube to release a ball. the location of the ball, and therefore the number of straws necessary to retrieve it, varied across two test phases (four of five straws and two of five straws, respectively). in test phase , all children retrieved the ball in trial and . % used the most efficient method (removing only straws below the ball). across phase trials, -year-olds were significantly more efficient than -year-olds, and solve latency decreased for all age groups. test phase altered the location of the ball, allowing us to explore whether children could flexibly adopt a more efficient solution when their original (now inefficient) solution remained available. in phase , significantly more -year-olds than -year-olds were efficient; the older children showed greater competency with the task and were more flexible to changing task demands than the younger children. interestingly, no age group was as flexible in phase as previously tested nonhuman primates, potentially related to their relatively reduced task exploration in phase . therefore, this causally clear task revealed changes in cognitive flexibility across both early childhood and species. flexibility allows individuals to nimbly react to novel situations, playing an important role in adaptive responses to environmental changes and finding optimum solutions to problems. humans are particularly adept at flexible thinking, potentially due to complexity in their environment and social relationship structure (gökçen, petrides, hudry, frederickson, & smillie, ) . such flexibility is important because it is linked to our innovative ability and tool use (keen, ; neldner, mushin, & nielsen, ) . previous research has shown that, as compared with younger children, older children can more flexibly react to environmental or task changes (but see gopnik et al., ) . for example, children over years of age can quickly alter sorting techniques on the same objects when given different verbal cues (e.g., ''find the ones that look like a ____" vs. ''find the one that is the same kind as ____"; deák & bauer, ) or game rules (e.g., ''sort the red ones" vs. ''sort the small ones"; frye, zelazo, & palfai, ; zelazo, frye, & rapus, ) , whereas children under age often perseverate on the first cue or rule with which they are provided, making switch errors on - % of trials . indeed, children's flexibility appears to develop slowly over the first few years of life and may be mediated by a child's understanding of the task (karmiloff- smith, ; spensley & taylor, ) . these findings, and others, suggest that cognitive flexibility might be linked to children's biological age (zelazo, muller, frye, & marcovitch, ) , although some evidence finds that children as a whole may also be more flexible than adults in certain situations (lucas, bridgers, griffiths, & gopnik, ) . in addition to biological maturation across one's lifespan, we can explore cognitive flexibility from a comparative perspective, for example, by studying nonhuman primates (sneve et al., ) . although there appears to be much variability across and within primate species with regard to their flexible or conservative responses to novel tasks (reviewed in brosnan & hopper, ) , nonhuman primates do exhibit cognitive flexibility in relation to both physical and social understanding (e.g., amici, call, watzek, brosnan, & aureli, ; pope et al., ) . in certain cases, and as compared with adult humans, nonhuman primates (e.g., macaques, capuchin monkeys) have been found to be significantly better at quickly and flexibly altering their behavior in response to changing task demands (e.g., avdagic, jensen, altschul, & terrace, ; stoet & snyder, ; watzek, pope, & brosnan, ) . thus, an exploration both across human development and across species might prove to be particularly insightful for understanding the ontogeny and evolutionary development of cognitive flexibility. research with humans and nonhuman primates has revealed that other important factors, including task complexity, opacity, and cognitive demands, influence cognitive flexibility. zelazo, carter, reznick, and frye ( ) proposed that individuals' ability to evaluate their own success in a task (i.e., error detection and correction) is a key component of problem solving as related to executive function. as such, tasks that make error detection easier are more likely to elicit adequate task switching from children. for example, when -year-olds are asked to repeat game rules before task switching, they are much more likely to succeed than when they are asked to simply complete the task without this verbal reminder (kirkham, cruess, & diamond, ) . whereas understanding the rules of a task can enhance children's success, causally clear tasks are also more efficiently solved by children and nonhuman primates (jacobson & hopper, ) . for example, -year-olds are more successful at an inhibition task if the actions required of them are obviously causal (e.g., pull a lever to get an object) as opposed to unclear or arbitrary (e.g., answer a phone to get an object) (mcguigan & núñez, ) . in this way, causal understanding may allow for solutions to be found and errors to be identified as well as flexibility in response to changes in task demands (hopper, kurtycz, ross, & bonnie, ; jacobson & hopper, ) . research has suggested that differences in response flexibility also likely depend on the cognitive demands inherent in the task. for example, less demanding lookingtime paradigms often show much earlier evidence for cognitive flexibility than tasks that require children to act on objects (e.g., smith, thelen, titzer, & mclin, ) . given this, (davis, schapiro, lambeth, wood, & whiten, ) proposed that perseveration is likely mediated by response prepotency (how familiar an action is) and working memory load (how demanding the task is). here, we sought to explore how children's cognitive flexibility (i.e., set shifting; ionescu, ) changes across early development and how their strategies compare with those of nonhuman primates (chimpanzees and gorillas tested previously using a comparable task and testing protocol). specifically, research with nonhuman primates has shown that they are less likely to adopt a novel solution after learning one successful one (hrubesch, preuschoft, & van schaik, ) but that this is mediated by task transparency, such that individuals tested with a causally clear task appear to be more flexible (jacobson & hopper, ) . therefore, unlike many previous studies on early cognitive flexibility, we used a paradigm that was both physically and causally clear and asked children to switch strategies without a large memory demand (e.g., remembering an abstract rule; zelazo et al., ) . specifically, we used a novel puzzle that was a clear vertical tube with five paper straws threaded through it at equal intervals. a small ball was placed in the tube such that it rested on a straw, and to retrieve the ball children needed to pull out all the straws below the ball so that it could fall down the tube and out the bottom. each time children retrieved the ball, they could exchange it with the researcher for a sticker. thus, this clear task relied on participants' basic understanding of gravity and support, did not necessitate the use of arbitrary actions, and did not require participants to retain information across trials. in the first configuration of the task, four straws were below the ball and one was above it. this was to test children's spontaneous understanding of the causal mechanics of the task and to verify that it was causally clear to children. research shows that even infants appear to have a basic understanding of gravity (baillargeon & hanko-summers, ; needham & baillargeon, ) , looking longer at and interacting more with objects if they appear to magically float in space when their support is removed (e.g., stahl & feigenson, ) . infants also appear to understand that a solid barrier will stop an object from falling or rolling in a downward trajectory even when that object is behind an opaque occluder (spelke, breinlinger, macomber, & jacobson, ) . although -year-olds struggle with more cognitively demanding physical adaptation of these looking-time studies, by years of age children are able to track a falling object behind an occluder, correctly select a door to open, and reach for the fallen object (berthier, deblois, poirier, novak, & clifton, ) . thus, with this first configuration of our task, we could test whether children would remove only straws below the ball and ignore the straw above it or whether they would ''blindly" pull out all straws, in turn revealing whether they understood the task rules (without explanation or guidance). furthermore, by testing -, -, and -year-olds, we could observe whether their understanding and success differed by age. to examine whether the children could adopt a new solution strategy after repeated experience with the task in the first configuration, we subsequently presented a new configuration where only two straws were below the ball and three straws were above it. in this new configuration, the most efficient solution was to remove two straws instead of four, although the previously efficient strategy remained viable (albeit a less efficient solution). in this way, our paradigm allowed us to test individuals' flexibility in the face of possible conservatism and the interplay between causal understanding and cognitive flexibility across children of different ages and across (primate) species. our previous research with nonhuman primates using the same task revealed that chimpanzees and gorillas showed flexible problem solving when task demands changed, likely due to the apes' causal understanding of the task (jacobson & hopper, ) . therefore, we predicted that if children understood the task mechanics, they would respond flexibly when task demands changed. specifically, we predicted that if children solved the task using the most efficient strategy in the first task configuration, they would also be able to adopt a new efficient strategy when the configuration was changed. however, we also predicted that the younger children may be less likely to master the task (i.e., understand the solution and so be less likely to use efficient responses) and, accordingly, may be less flexible than the older children (if task understanding relates to flexibility in response patterns). thus, with our study, we wanted to see how young children responded to changing task demands, how their efficiency and flexibility differed with age, and how their responses compared with those of nonhuman primates. we tested children representing three age groups: -year-olds (m = . months, range = . - . ; girls), -year-olds (m = . months, range = . - . ; girls), and -yearolds (m = . months, range = . - . ; girls). from parental reporting, we determined that % of participants were caucasian, % were african american, % were hispanic, % were asian american, and % were multiracial ( % of parents opted out of answering questions concerning their child's race/ethnicity). in addition to the children described above, we tested children who were not included in the final sample due to refusing to participate in the given tasks (n = ), failure to obtain video-recording consent (n = ), or experimenter error (n = ). this study received approval from the franklin & marshall institutional review board. after entering the testing room, children sat in a chair placed directly in front of experimenter and the testing apparatus. the apparatus (modeled from the design previously used to test problem solving flexibility in apes; jacobson & hopper, ) was a clear pvc tube (approximately . cm in diameter and . cm long) affixed to a stand with equally spaced holes for up to five straws to be slotted through the tube (fig. ). experimenter sat on a small chair perpendicular to children. parents sat in a chair opposite from the apparatus, facing children and the experimenters, and were asked to remain quiet and to not intervene in the task or guide children's responses. sessions were videorecorded for later coding, with the camera located diagonal to the testing area (focused on experimenter , the testing apparatus, and children's hands. testing comprised a familiarization phase followed by two experimental phases. in the familiarization phase, experimenter introduced children to the test apparatus, saying, ''this is my toy. look what this toy can do. when i put a ball in, it comes out the end." the experimenter then proceeded through familiarization trials to acquaint children with the general mechanics of the tube. during these trials, the experimenter dropped a ball into the tube to show how a ball could fall down the apparatus when no obstructions (straws) were present. with each trial, experimenter verbally prompted children to retrieve the ball that came out of the tube (''can you get the ball?") and told children that they would be rewarded with a sticker by experimenter when they obtained the ball (''every time you give the ball to our friend [experimenter ], you get a sticker!"). the sticker provided an incentive for children to quickly retrieve the ball by removing the straw obstructions (in the previous study with chimpanzees and gorillas using this task, a food reward was used in place of the ball here, which was inherently rewarding; see jacobson & hopper, ) . in test phase , participants completed trials whereby they were asked to retrieve the ball from the test apparatus. for each trial, experimenter baited the test apparatus out of view of children (behind a by -cm tri-fold display board). experimenter inserted a straw through each of the five holes in the shaft of the tube, with the ball placed into the apparatus such that there were four straws below the ball and one straw above it (configuration in fig. ). in this configuration, children needed to remove the four straws below the ball to obtain it; the fifth straw could also be removed, but doing so was causally irrelevant to obtaining the ball. the experimenter ensured that the straws were aligned equally to avoid any visual cuing that might encourage participants to select specific straws (e.g., one straw sticking out from the apparatus farther than another straw). then, the experimenter removed the tri-fold board so that children could view the tube. a test trial began as soon as the experimenter verbally prompted children to interact with the apparatus (''can you get the ball?"). participants were then given a chance to remove any of the straws (below or above the ball) in whatever manner they wished. if children were hesitant to initiate an interaction with the straws or the apparatus, the experimenter further prompted them with encouraging but noninformative cues (e.g., ''it's okay, you can come up and touch it"; ''the ball is stuck. how do you think you could get it out?"; ''if you get the ball, you'll get a sticker!"). throughout the trial, the experimenter retained a neutral facial expression and a neutral tone of voice to prevent children from receiving any external cues that would interfere with their interaction with the apparatus. similar to the familiarization trials, participants received a sticker whenever they acquired the ball and handed it to experimenter . after completing trials with the apparatus in configuration , phase commenced. in test phase , participants completed test trials with the new task configuration. these trials were run as in phase , changing only the apparatus configuration; in phase , the experimenter baited the apparatus such that there were only two straws below the ball and three straws above it (configuration in fig. ) . thus, the total number of straws that could be removed (five) was the same across test phases, but the number of straws that needed to be removed to obtain the ball differed (four in phase and two in phase ). importantly, the experimenter never highlighted the change in task configuration, or the new location of the ball at the start of the trial, either verbally or via pointing. as with phase , the experimenter made statements only to encourage children's engagement (e.g., ''can you get the ball?"). in phase , children completed trials. there were two reasons why we ran fewer trials in this second phase. first, our primary interest was assessing children's ability to switch response strategies when the task configuration changed. for this, we were predominantly interested in assessing their the configuration both in phase (four straws below the ball and one straw above it) and in phase (two straws below the ball and three straws above it) and a photograph of one participant completing a trial in phase . in either task configuration, participants were free to pull out as many straws us they chose and in any order. thus, although the most efficient strategy would be to only remove straws below the ball, in either configuration children could also adopt inefficient strategies and remove straws both above and below the ball. importantly, the efficient solution for phase (removing straws - ) remained viable in phase , although a different solution was the most efficient one (removing straws and ). responses in the first trial post-configuration change (i.e., to see whether they adopted a new solution and whether they adopted the most efficient solution possible with their first response in phase ). second, although we also wanted to evaluate children's repeated responses in phase with multiple trials (to see whether their responses increased in efficiency over time if they did not make a strategy switch with their first trial of phase ), we did not want to give children too many opportunities to interact with the task because we wanted to test their spontaneous responses. this is in contrast to phase , where we wanted to assess their spontaneous understanding of the task (trial ) and also wanted to give them repeated experience with the task across multiple trials both to assess their exploration of the task and to generate a modal response (''remove four straws") that would be more likely to be conserved and potentially harder to deviate from in phase (in the sense of jacobson & hopper, ) . a trained researcher coded all the test trials from video. a second independent research assistant coded % of participants' trials, with the two coders agreeing on approximately . % of total behavioral scores. when there was a coding disagreement, we used the primary coder's scoring for a given trial. for each trial, the coder recorded four elements and associated information: the total number of straws participants removed, the order in which participants removed the straws, the length of each trial (i.e., latency to remove straws), and any comments participants made during the first trial of test phase when they were presented with a new configuration of the task (i.e., the ''switch trial"). for each trial, we coded for the total number of straws removed by children (out of a possible five for each trial) and the order in which children removed each straw. if participants retrieved the reward by pulling only the straws below the ball (four straws for test phase or two straws for test phase ), we coded the trial as ''efficient," but if participants pulled one or more straws above the ball (thereby pulling straws that were not causally necessary to receive the reward), we coded the trial was as ''inefficient" (as per jacobson & hopper, ) . we coded the time at which participants removed each straw within a given trial. thus, we could calculate the latency for participants to complete each of their trials. the start time for each trial began as soon as the experimenter uttered the introductory prompt (''can you get the ball?") and ended once children indicated they were done removing straws by explicitly stating such or moving to experimenter to retrieve a sticker for ball retrieval (typically as soon as the ball fell from the tube). during the first trial of phase (the ''switch trial" when the configuration of the apparatus was altered), we transcribed any verbal comments participants made that might indicate that they noticed a change and/or were seeking information related to the change (i.e., ''why" questions; legare, sobel, & callanan, ) . for instance, some participants would recognize the change of the tube arrangement and say, ''why is the ball all the way down there?" or ''how did you do that?" we provide a descriptive summary of these in the results section. to explore participants' spontaneous understanding of the task and their flexibility in response to changing task demands, we analyzed four key aspects of the coded data using r version . . (core, ): (a) participants' spontaneous understanding of the task, (b) any apparent learning across trials, (c) participants' flexibility and efficiency across and within phases, and (d) the verbal responses participants made, if any. for clarity, the specific analytical approaches that we used for each analysis are reported within that section of the results. for all pairwise comparisons, a bonferroni correction was applied (i.e., a . ). with a . and an effect size of . , our power analysis revealed a value of . . (to achieve a power value of . , we would need to have included children per condition, but with the current covid- pandemic, additional testing was not feasible.) we plotted all data using the ggplot package (wickham, ) and beeswarm package (eklund, ; see also wilkinson, ) in r version . . (core, ). all children tested were able to retrieve the ball from the apparatus in their first trial of phase . furthermore, of the -year-olds ( . %), of the -year-olds ( . %), and of the year-olds ( . %) used the most efficient method to do so in trial of phase (i.e., they removed only the lower four of the five straws from the tube) (fig. ) . we compared children's spontaneous ability to solve the task across the three age groups. to do so, we used first trial efficiency (efficient or inefficient) as our outcome variable and participant id as a random factor. given the binary response variable, we analyzed our data using a binomial generalized linear mixed model (glmm) in r version . . (core, ). we fit this model using the laplace approximation method via the ''glmer" function in the lme package (bates, maechler, & bolker, ) to test the relative effect of our predictor variable age group (family = ''binomial"). this revealed that there was no significant difference across the three age groups of children in their likelihood to use the most efficient method in their first trial of phase {z = . , p = . , % confidence interval (ci) [À . , . ]}. not only were the three age groups equally likely to use an efficient response with their first trial, but of those children who responded efficiently in the first trial of phase , there was strong consistency in the action sequence (i.e., straw removal order) that they used: . % of the -year-olds who responded efficiently used the , , , action sequence, as did . % of the -year-olds and . % of the -year-olds who responded efficiently (i.e., they sequentially removed the straw directly below the ball) (fig. ). on average across all their trials in phase , -year-olds used an efficient action sequence in . % (sd = . ) of trials, whereas -year-olds were efficient in . % (sd = . ) of trials and -yearolds were efficient in . % (sd = . ) of trials (fig. ) . using a glmm, we explored the proportion of all the participants' trials in phase that were efficient (family = ''poisson") and used independent t tests, using the ''t.test" function to compare children's efficiency across age groups. this revealed that there was a significant effect of age on the percentage of trials in which children made efficient responses in phase (z = . , p = . , % ci [ . , . ]). specifically, -year-olds made significantly more efficient responses than -year-olds, t( . ) = À . , p = . , % ci [À . , À . ], but there was no significant difference in the percentages of trials in which -and -year-olds made efficient responses, t( . ) = À . , p = . , % ci [À . , . ] or in the percentages of trials in which and -year-olds made efficient responses, t( . ) = À . , p = . , % ci [À . , . ] . when first presented with the task in phase , the average latency for -year-olds to complete their first trial was . s (sd = . ). the -year-olds were significantly quicker to complete their first trial than the -year-olds (average latency = . s, sd = . ), t( . ) = . , p = . , % ci [ . , . ] and the -year-olds (average latency = . s, sd = . ), t( . ) = . , p = . , % ci [ . , . ]. there was no significant difference, however, between the -and -year-olds in the time it took them to complete their first trial, t( . ) = . , p = . , % ci [À . , . ]. to test whether children's trial completion times became quicker across trials, as a proxy for learning, we correlated participants' trial latency with trial number using the ''rmcorr" function (bakdash & marusich, ) . this takes into account repeated samples from participants to determine whether their trial latency decreased over time. this revealed that there was a significant negative correlation between the trial completion latency and trial number for all three age groups, such that children became quicker to complete trials across the trials in phase : -year-olds (r = À. , p < . ), -year-olds (r = À. , p < . ), and -year-olds (r = À. , p = . ). the weaker negative relationship between trial number and latency for -year-olds is likely because they completed their first trial faster than the younger children and there is likely a limit to how quickly any children can complete a trial, creating a floor effect. indeed, -year-olds' average trial completion latency delta from trial to trial was only . s (average trial latency = . s, sd = . ), whereas the delta for -year-olds was . s (average trial latency = . s, sd = . ) and for -yearolds was . s (average trial latency = . s, sd = . ). to evaluate children's cognitive flexibility, we assessed their efficiency in the first trial of phase when they were presented with the new task configuration (fig. ). only ( . %) of the children used the same action sequence (straw removal order) in the first trial of phase as they had used in their last trial of phase . specifically, -year-olds and -year-olds used the , , , , action sequence in both trials, whereas -year-old used the , , , action sequence in the last trial of phase and the first trial of phase . thus, the majority of children ( . %) used a different action sequence across these trials. for all children and action sequences used, in the first trial of phase , of the year-olds ( . %), of the -year-olds ( . %), and of the -year-olds ( . %) used the (newly available) most efficient method (i.e., they removed only the lower two of five straws from the tube), highlighting their recognition of the changed task demands. as with test phase , we used the ''glmer" function in the lme package (family = ''binomial") to compare the numbers of children across the three age groups whose first trial in phase was efficient, and we used independent t tests using the ''t.test" function for post hoc pairwise comparisons across age groups. our analyses revealed that there was a significant effect of age on children's efficiency in the first trial of phase (z = . , p = . , % ci [ . , . ]). in spite of this, after correcting for multiple comparisons, post hoc pairwise comparisons revealed no significant difference across age groups when comparing the numbers of children whose responses in the first trial of phase responses were efficient: -year-olds versus year-olds, t( . ) = À . , p = . , % ci [À . , À . ]; -year-olds versus -year-olds, considering all trials that children completed in phase , on average children removed significantly fewer straws per trial in phase than they did in phase , highlighting their understanding of the changed task demands. this was true for all three age groups of children tested: -year-olds, t( . ) = . , p = . , % ci [ . , . ]; -year-olds, t( . ) = . , p < . , % ci [ . , . ]; -year-olds, t( . ) = . , p < . , % ci [ . , . ]. although children removed fewer straws in phase as compared with phase , did they consistently remove the fewest possible number (i.e., two straws)? on average across all trials in phase , -year-olds used an efficient action sequence in . % (sd = . ) of their trials, whereas -year-olds used an efficient solution in . % (sd = . ) of trials and -year-olds were efficient in . % (sd = . ) of trials. using a glmm (family = ''poisson"), we found that there was a significant effect of age on the proportion of phase trials in which children made efficient responses (z = . , p = . , % ci [ . , . ]). specifically, -year-olds made significantly more efficient responses than -year-olds, t( . ) = À . , p = . , % ci [À . , À . ], but there was no significant difference in the proportions of trials in which -and -year-olds made efficient responses, t( . ) = À . , p = . , % ci [À . , . ], or in the proportions of trials in which -and -year-olds made efficient responses, t( . ) = À . , p = . , % ci [À . , À . ]. to further explore children's causal understanding of the task and their ability to flexibly shift strategies across the phases in response to the change in task configuration, we compared children's latency to complete trials across the two phases as a proxy for flexibility (i.e., removing two straws should take less time than removing four straws). across all children tested, they were significantly faster to complete trials in phase (average trial completion latency = . s, sd = . ) compared with phase (average trial completion latency = . s, sd = . ), r( ) = À. , p < . . there was also a significant effect of age on children's latency to complete a trial. within phase , the average latency for -year-olds to complete a trial was . s (sd = . ), whereas the average trial completion latencies for -and -year-olds were . s (sd = . ) and . s (sd = . ), respectively. the -year-olds completed trials significantly faster than both the -year-olds, t( . ) = . , p < . , % ci [ . , . ] and -year-olds, t( . ) = . , p = . , % ci [ . , . ]. in contrast, there was no significant difference between the -year-olds' and -year-olds' trial completion latency, t( . ) = . , p = . , % ci [À . , . ]. in addition to comparing children's understanding of the task and flexibility across ages, we were also interested in how consistently proficient each child was. to examine this, we compared children's efficiency in phase with their efficiency in phase . we found that, for all three age groups, children showed intra-individual consistency in their efficiency across phases; that is, the proportion of trials that children solved efficiently in phase was significantly correlated with the proportion of trials that children solved efficiently in phase : pearson's product-moment correlation, -year-olds, t( ) = . , p < . , % ci [. , . ]; -year-olds, t( ) = . , p = . , % ci [. , . ]; -year-olds, t( ) = . , p < . , % ci [. , . ]. as reflected by children's responses in the first trial of phase , the action sequence most commonly used by children across all trials in phase was repeatedly removing the straw directly below the reward (i.e., , , , ) ( figs. and ) . this action sequence represented . % of -year-olds' trials, . % of -year-olds' trials, and . % of -year-olds' trials in phase . in addition, and as can be seen in fig. , the modal inefficient action sequence for all three age groups in phase was , , , , (i.e., pulling out all the straws from top to bottom). not only was there consistency across children in their modal action sequence ( , , , ) in phase , there was also intra-individual consistency such that some children perseverated in their response phenotype and used an action sequence in multiple successive trials. indeed, children ( -yearold, -year-olds, and -year-olds) used the same efficient action sequence for every response they made in phase , and -year-olds used the same inefficient sequence in each of their trials. therefore, we explored children's conservatism in this regard. for each child, we calculated the longest run of consecutive trials in which the child used the same action sequence in phase , where consecutive trials was the shortest possible run length and consecutive trials was the longest possible run length. in addition, we coded whether each run was efficient or inefficient, and we calculated each child's average efficient run length as a proportion of total possible responses. from this, we found that -year-olds made significantly longer efficient runs (average proportion of trials = . , sd = . ) than -year-olds (average = . , sd = . ), t( . ) = À . , p = . , % ci [À. , À. ] (fig. ) . however, there was no significant difference in the average efficient run length made by and -year-olds (average = . , sd = . ), t( . ) = À . , p = . , % ci [À. , . ] or in the average efficient run length made by -and -year-olds, t( . ) = À . , p = . , % ci [À. , À. ]. in spite of the aforementioned conservatism shown by some children to perseverate on an action sequence across multiple trials, there was variation in the action sequences used by the children across trials. thus, even after discovering the , , , solution, children sampled other action sequences. collectively, children used different action sequences in phase ( were possible) (fig. ) . they used ( . %) of the possible action sequences that were efficient, removing straws , , , and first (e.g., , , , and , , , ), but used only ( . %) of the possible action sequences that were inefficient in which they pulled out the irrelevant straw before releasing the reward (e.g., , , , , and , , , , ). in addition, ( . %) of the different action sequences that -year-olds used were efficient ones, whereas ( . %) of the action sequences used by year-olds and ( . %) of the action sequences used by -year-olds were efficient. to determine the diversity of action sequences children used, we calculated the diversity index of their responses (shannon & weaver, ) . if participants repeatedly used the same action sequence (i.e., developed a habit), their diversity index would be lower than those who did not. we used wilcoxon tests (''wilcox.test") to compare participants' ''h-index" diversity index across age groups and across test phases. children's individual h-index scores in phase ranged from . to . , where an index score of means that only one action sequence was used and an index score of . would mean that a different action sequence was used for each of the trials, although this never occurred, as indicated by children's maximum score of . . in phase , there was no significant difference in children's h-index scores across the three age groups: -year-olds versus -year-olds, t( . ) = . , p = . , % ci [À. , . ]; -year-olds versus -year-olds, t( . ) = À . , p = . , % ci [À. , . ]; -year-olds versus -year-olds, t( . ) = . , p = . , % ci [À. , . ]. in spite of this, we found differences across the three age groups in the relationship between their phase h-index score and the proportion of trials in which they used an efficient action sequence. specifically, for -year-olds, there was a significant negative correlation between their h-index score and their proportion of trials that were efficient {pearson's product-moment correlation: t( ) = À . , p < . , % ci [À. , À. ]}, and this was also the case for -year-olds, t( ) = À . , p = . , % ci [À. , À. ], but not -year-olds, t( ) = À . , p = . , % ci [À. , . ] . in contrast to phase , children needed to complete only trials in phase , so there was a higher probability that they would use the same response in all trials. indeed, whereas only children ( . %) used the same response across all trials in phase (described above), children ( . %) used the same response across all trials in phase , and of these children used an efficient response for every trial (no -year-olds used the same inefficient action sequence repeatedly across trials). reflecting this intra-individual consistency, there was also inter-individual consistency in the specific action sequence that children used in phase . as in phase , the action sequence most commonly used by children in phase was repeatedly removing the straw directly below the reward (i.e., , ) (fig. ) . this action sequence represented . % of -year-olds' trials, . % of -year-olds' trials, and . % of -year-olds' trials, and as in phase the modal inefficient response for -and -year-olds was to remove all the straws from top to bottom (i.e., , , , , ) (fig. ) . in spite of children's preference for the , action sequence in phase , they still explored other solution phenotypes, including the alternative efficient action sequence ( , ) and an additional different inefficient action sequences (fig. ) . in addition to the efficient strategies, -year-olds used inefficient action sequences ( . % of their action sequences were efficient), whereas -year-olds used inefficient sequences and -year-olds used , meaning that . % and . % of the action sequences they used were efficient, respectively. children's individual h-index scores in phase ranged from . to . , where an index score of means that only action sequence was used and an index score of . would mean that a different action sequence was used for each of the trials. as in phase , in phase there was no significant difference in children's h-index scores across the three age groups: -year-olds versus -year-olds, t( . ) = . , p = . , % ci [À. , . ]; -year-olds versus -year-olds, t( . ) = . , p = . , % ci [À. , . ]; -year-olds versus -year-olds, t ( . ) = . , p = . , % ci [À. , . ]. however, as with their responses in phase , we found that the older children's diversity of responses (h-index score) was negatively correlated with efficiency. specifically, for -and -year-olds, there was a significant negative correlation between their hindex score and their proportion of trials in phase that were efficient { -year-olds: t( ) = À . , p = . , % ci [À. , À. ]; -year-olds: t( ) = À . , p = . , % ci [À. , À. ]}, but this was not the case for -year-olds, t( ) = À . , p = . , % ci [À. , . ]. during the first trial of phase , when children were first presented with the novel configuration of the task (i.e., the ''switch trial"), none of the -year-olds made any verbal comment in relation to the task. however, . % of the -year-olds did, as did . % of the -year-olds. most -year-olds' comments reflected the change in task configuration (e.g., ''why did it go down to this one?"; ''it's not up there anymore"), whereas other comments highlighted the configuration change but also flagged the experimenter's agency in causing that change (e.g., ''why did you put two?"; ''how did you do that?"). like -year-olds, -year-olds' comments referred to the change of task configuration (e.g., ''there's only two straws"; ''hey, it's now down there") and the experimenter's causation of the change (e.g., ''how did you do that?"), but -year-olds also commented on how this change affected their own behavior and success (e.g., ''i only had to get two"; ''that was super fast-that's because there were only two straws"). both -and -year-olds commented on the configuration change in . % of their first trials in phase in which they made an efficient response (i.e., removing only the bottom two straws). in the trials in which children made an inefficient response (i.e., removing three or more straws), . % of -year-olds commented on the change, whereas a quarter ( . %) of -yearolds commented on the change. for both age groups, there was no significant difference in the numbers of efficient first trial responses in which children made a comment on the task configuration (fisher's exact test: -year-olds, p = . ; -year-olds, p = . ). in our study, we explored -, -, and -year-old children's ability to flexibly switch between response patterns as task demands changed. as jacobson and hopper ( ) found previously for nonhuman primates, all the children easily mastered the task and retrieved the ball from the tube. however, -year-olds were consistently more efficient than the younger children in terms of both the time it took them to complete trials and the number of straws they removed. this developmental trajectory in children's responses reflects previous research showing that children's problem solving and tool making skills increase with age (gönül, takmaz, hohenberger, & corballis, ) . indeed, the cognitive complexity and control theory proposes that ''executive function can be understood in terms of agerelated increases in the maximum complexity of the rules children can formulate and use when solving problems" (zelazo et al., , p. ) . in spite of this, there was no significant difference across the three age groups of children in their likelihood of using the newly available efficient solution when it was presented in the first trial of phase . we also identified intra-individual consistency in children's success such that their efficiency in phase correlated with their efficiency in phase . given the general success of children in all age groups, it is likely that this causally clear task facilitated children's success and flexibility (in the sense of davis et al., ) , as has been found in chimpanzees and gorillas tested using the same task (jacobson & hopper, ) . our aim was to provide a task that was accessible for children in all three age groups to allow us to make meaningful across-age comparisons (as well as comparisons with nonhuman primates' responses). supporting our goal, all children spontaneously solved the task and there was no difference across the three age groups in their initial understanding of the task, as evidenced by their comparable likelihoods to use the most efficient method in their first trial of phase ( . % of children used an efficient solution in the first trial of phase ). however, -year-olds showed sustained efficiency across trials in phase ; significantly more of their trials were solved via the efficient method than those of -year-olds. although the -year-olds were not more likely to spontaneously use the most efficient solution in their first trial of phase than the younger children, they were more likely to stick with it and were significantly quicker to complete their first trial, suggesting enhanced physical dexterity, potentially in combination with a better understanding of the task demands. however, and in spite of -year-olds' greater use of efficient solutions, within phase all three age groups showed an improvement in task proficiency over time, as demonstrated by the negative correlation between trial latency and trial number. although the vast majority of children spontaneously used an efficient solution when first presented with the task, when we changed the task demands and introduced the possibility of a new efficient solution in phase , only . % of children spontaneously used the most efficient solution with their first attempt (removing only two straws). as with the first trial of phase , however, there was no effect of age on children's likelihood to use an efficient solution for the first trial of phase . thus, nearly half of the children, irrespective of age, did not switch strategies in the first trial of phase . in certain situations, humans react in remarkably fixed ways even when their environment does not necessitate such rigidity (bilalić , mcleod, & gobet, ; gopnik, griffiths, & lucas, ) . for example, adults often sit in the same seat during classes or meetings even without seat assignment and when there are no repercussions for moving (costa, ) . the limited flexibility we observed cannot be explained by pure conservatism (in the sense of hrubesch et al., ) ; of the children who used an inefficient solution for the first trial of phase , only ( . %) were inefficient because they used the exact same action sequence as they had used in the previous trial (i.e., the final trial of phase ). we also note that a large subset of -and -year-olds remarked on the change in task configuration. although children often verbally seek out information to understand causal elements of their environment (legare et al., ) , our experimenters were instructed not to answer children's questions and were not useful as informants (and, anecdotally, children almost never explicitly asked for help). therefore, it is possible that children were describing the changes they observed to help themselves make sense of the changes and respond to the new task demands (winsler, fernyhough, & montero, ) . we had predicted that greater exploration of the task (i.e., using a range of action sequences) would protect children against conservatism and allow them to more flexibly respond when task demands changed. paradoxically, although -year-olds were the most efficient in their responses, they were also the most conservative; fully . % of their trials in phase were solved using the same action sequence ( , , , ), and we saw similar patterns in their responses in phase (when they preferentially used the , action sequence). indeed, the older children's preferred responses (whether efficient or inefficient) were to remove straws sequentially rather than in a random pattern (although they did this on occasion). reflecting this, of the children who used the same action sequence for every trial in phase , were -year-olds (only -year-old used the same action sequence in every trial in phase ). indeed, for -and -year-olds, but not -year-olds, there was a significant negative correlation between their h-index score and proportion of trials that were efficient. thus, for the older children, decreased diversity was associated with increased efficiency, reflecting the results of their likelihood to display longer runs of efficient action sequences (this was also seen in their phase responses). in this way, the younger children's greater exploration did not benefit them either within phases or in their flexibility across phases. although conservatism is often seen as a sign of reduced cognitive flexibility, because the older children struck on and then stuck with an efficient solution from the start, they were able to sustain their efficiency (i.e., ''if it ain't broke, don't fix it"). furthermore, the older children's apparent flexibility in switching strategies from phase to phase might not reflect a switch but rather a continuation of the same strategy (''pull the straw below the ball"). without further controls, it is difficult to discern whether this is an insightful solution or a rigid response-sticking with the first reinforced pattern used-but we would argue the former given that the younger children also used this solution early on but did not stick with it. the increased exploration in the younger children aligns with some of the ideas outlined in the overlapping waves theory, which states that children do not simply progress from ignorance to full comprehension across development but rather proceed through cognitive waves involving data collection, mapping, strengthening, and refinement when attempting to effectively solve problems (e.g., chen, siegler, & daehler, ) . however, although this exploration might be viewed favorably with respect to cognitive flexibility, it is actually counter to maximizing efficiency. in phase , there were fewer possible action sequences that were efficient as compared with phase , and so we might expect that children with a strong causal understanding of the task, and a desire to be efficient, would use fewer different action sequences in phase than in phase . indeed, this is what we saw with year-olds. the -year-olds used fewer action sequences in phase ( ) than in phase ( ), whereas the -year-olds used a comparable number of action sequences in both phases ( vs. ). although our a priori goal for this task was to remove the ball by removing as few straws as possible, we never explicitly shared this goal with the children. therefore, for the younger children, rather than maximizing efficiency, play and exploration might have been stronger drives, which can be advantageous (greve & thomsen, ) . a drive to play might explain why the younger children used more action sequences in phase , although this could also be explained by a reduced understanding of the task mechanics or could be related to young children's tendencies to be more exploratory when events are surprising (stahl & feigenson, ) . in addition to exploring ontogenetic changes in children's cognitive flexibility, we were also interested in comparing children's behavior in this task with that of chimpanzees and gorillas tested previously with the same task under comparable protocols. when first presented with the task, all children spontaneously retrieved the ball and . % used the most efficient method with their first attempt. as noted above, the apes we tested previously were equally successful, with . % of them using the most efficient method when first presented with the same task (jacobson & hopper, ) . however, in spite of the seeming similarities across species, there were differences in the way in which the children and apes solved the task. for example, whereas the children used different efficient action sequences collectively in phase , the apes deployed . furthermore, the children used fewer action sequences on average across the first trials of phase as compared with the apes (see fig. s in the online supplementary material). the increased exploration by the apes may be due to differences in experimental protocol (children completed all trials within a single session, whereas apes completed trials over one or more sessions; see jacobson & hopper, , for details) . however, it is notable that whereas the -and -year-old children never used more than different action sequences each, the -year-old children used up to seven and eight different sequences each, revealing exploration rates more similar to that of the apes. a greater percentage of the apes were flexible in adopting a more efficient response in the first trial of phase as compared with the children even when comparing apes with the oldest child age group. as discussed, within phase and across all three age groups, the children predominantly solved the task by repeatedly removing the straw directly below the ball (i.e., , , , ). this was also the predominant strategy used by the chimpanzees tested previously, but not by the gorillas, whose preferred strategy was to remove straws sequentially from the bottom up (i.e., , , , ) (cf. fig. here with fig. in jacobson & hopper, ) . the strategy of consistently moving the straw that the ball rests on could potentially represent a simple association that was learned by the children rather than a holistic understanding of the task mechanism, but what explains these apparent species differences is not clear at this time. the observed species differences may be a result of methodological elements between this study and that of jacobson and hopper ( ) . namely, we gave the children trials in phase before changing the task configuration, whereas the apes received more than trials spread over multiple sessions before the task was changed. the apes' increased experience with the task may have afforded them greater experience with the task, which may have allowed them to be more flexible or simply gave them more opportunities to explore alternative action sequences. indeed, this might be why the apes' average run length was shorter than that of the children (average proportion of apes' first trials that were runs = . , sd = . ; cf. with fig. here) ; however, unlike the children, the apes never performed a run of inefficient action sequences in their first trials of phase (see table in jacobson & hopper, ) . future work is needed to tease apart the influences of experience, causal understanding, and conservatism on the apparent species differences we identified. from our results, we propose that causal understanding of a task not only promotes problem solving but also reduces the likelihood of conservative perseveration (see also jacobson & hopper, ) . however, humans are inherently social, and although we may sometimes solve problems by ourselves, we also often seek out information from others. in a landmark study, bonawitz et al. ( ) found that children were much less flexible when they were directly trained on how to use a certain object. termed the ''double-edged sword of pedagogy," children who observed someone interacting with the object were more likely to explore and learn its multiple functions, whereas those who were directly given instructions did not stray from the singular function they were taught. this effect has now been seen in a number of other contexts, such as children learning to flexibly solve new math problems (loehr, fyfe, & rittle-johnson, ) , and may explain children's proclivity for overimitation (lyons, young, & keil, ; over & carpenter, ; whiten, mcguigan, marshall-pescini, & hopper, ). thus, although direct social instruction can help children to quickly learn how to complete a task, it might also unnecessarily cause behavioral perseveration. conversely, nonhuman primates appear to be less influenced by social norms as compared with children (e.g., haun, rekers, & tomasello, ; horner & whiten, ; but see hopper, schapiro, lambeth, & brosnan, ) . future research could explore the role of individuals' causal understanding and their reliance on social information on cognitive flexibility (e.g., burdett, mcguigan, harrison, & whiten, ) from both a comparative perspective and an ontogenetic perspective (e.g., horner & whiten, ; pope, fagot, meguerditchian, washburn, & hopkins, ; stengelin, hepach, & haun, ; wood, kendal, & flynn, ) . here, we found that although all three age groups of children were successful in solving the task, year-olds were more efficient and more flexible in their approach to solving the task and responding to changing task demands than -year-olds. we previously tested apes on the same task and concluded that their ability to alter the solution strategy they used when we changed the task configuration was likely linked to their causal understanding of the task (jacobson & hopper, ) . unfortunately, procedural differences in testing protocols across species prevents us from making too many inferences about the apparent species differences we observed or what might drive these differences. however, we note that research using different tests of cognitive flexibility has also identified differences across human and nonhuman primates' responses to matched tasks (e.g., avdagic et al., ; pope et al., ; watzek et al., ) , although typically such research has tested adult humans, not young children as we did. future work exploring the interplay among social information, causal understanding, and cognitive flexibility is needed. social inhibition and behavioural flexibility when the context changes: a comparison across six primate species rapid cognitive flexibility of rhesus macaques performing psychophysical task-switching is the top object adequately supported by the bottom object? young infants' understanding of support relations repeated measures correlation lme : linear mixed-effects models using s classes where's the ball? two-and three-year-olds reason about unseen events inflexibility of experts-reality or myth? quantifying the einstellung effect in chess masters the double-edged sword of pedagogy: instruction limits spontaneous exploration and discovery psychological limits on animal innovation the interaction of social and perceivable causal factors in shaping 'over-imitation across the great divide: bridging the gap between understanding of toddlers' and older children's thinking. monographs of the society for research in child development territorial behavior in public settings behavioral conservatism is linked to complexity of behavior in chimpanzees (pan troglodytes): implications for cognition and cumulative culture the effects of task comprehension on preschoolers and adults categorization choices beeswarm: the bee swarm plot, an alternative to stripchart theory of mind and rule-based reasoning sub-threshold autism traits: the role of trait emotional intelligence and cognitive flexibility the cognitive ontogeny of tool making in children: the role of inhibition and hierarchical structuring when younger learners can be better (or at least more open-minded) than older ones changes in cognitive flexibility and hypothesis search across human life history from childhood to adolescence to adulthood evolutionary advantages of free play during childhood children conform to the behavior of peers; other great apes stick with what they know captive chimpanzee foraging in a social setting: a test of problem solving, flexibility, and spatial discounting chimpanzees' socially maintained food preferences indicate both conservatism and conformity causal knowledge and imitation/emulation switching in chimpanzees (pan troglodytes) and children skill mastery inhibits adoption of observed alternative solutions among chimpanzees (pan troglodytes) exploring the nature of cognitive flexibility hardly habitual: chimpanzees and gorillas show flexibility in their motor responses when presented with a causally-clear task the development of problem solving in young children: a critical cognitive skill constraints on representational change: evidence from children's drawing helping children apply their knowledge to their behavior on a dimensionswitching task sorting between theories of perseveration: performance in conflict tasks requires memory, attention and inhibition causal learning is collaborative: examining explanation in social contexts wait for it delaying instruction improves mathematics problem solving: a classroom study when children are better (or at least more open-minded) learners than adults: developmental differences in learning the forms of causal relationships the hidden structure of overimitation executive functioning by - -month-old children: effects of inhibition, working memory demands and narrative in a novel detour-reaching task intuitions about support in . -month-old infants young children's tool innovation across culture: affordance visibility matters the social side of imitation enhanced cognitive flexibility in the seminomadic himba optional-switch cognitive flexibility in primates: chimpanzees' (pan troglodytes) intermediate susceptibility to cognitive set r: a language and environment for statistical computing the mathematical theory of communication knowing in the context of acting: the task dynamics of the a-not-b error high-expanding regions in primate cortical brain evolution support supramodal cognitive flexibility origins of knowledge the development of cognitive flexibility: evidence from children's drawings observing the unexpected enhances infants' learning and exploration expectancy violations promote learning in young children cross-cultural variation in how much, but not whether, children overimitate task-switching in human and nonhuman primates: understanding rule encoding and control from behavior to single neurons capuchin and rhesus monkeys but not humans show cognitive flexibility in an optional-switch task emulation, imitation, over-imitation and the scope of culture for child and chimpanzee ggplot : elegant graphics for data analysis dot plots. the american statistician private speech, executive functioning, and the development of verbal selfregulation copy me or copy you? the effect of prior experience on social learning early development of executive function: a problem-solving framework an age-related dissociation between knowing rules and using them the development of executive function in early childhood. monographs of the society for research in child development a portion of this study was funded by the franklin & marshall college hackman scholars program award to lhh, and lmh received support from the lincoln park zoo's women's board. in addition, we thank the following individuals for their assistance with data collection, coding, and methods preparation: lauren hein, natalie hutchins, enya meade, you jin park, ayla saferstein, tian tian, ellen verry, fiona waters, and peiru yu. supplementary data to this article can be found online at https://doi.org/ . /j.jecp. . . key: cord- -jh klbg authors: sivapalan, pradeesh; ulrik, charlotte suppli; bojesen, rasmus dahlin; lapperre, therese sophie; eklöf, josefin viktoria; håkansson, kjell erik julius; browatzki, andrea; tidemansen, casper; wilcke, jon torgny; janner, julie; gottlieb, vibeke; meteran, howraman; porsbjerg, celeste; madsen, birgitte lindegaard; moberg, mia; pedersen, lars; benfield, thomas lars; lundgren, jens dilling; knop, filip krag; biering-sørensen, tor; ghanizada, muzhda; sonne, tine peick; bødtger, uffe christian steinholtz; jensen, sidse graff; rasmussen, daniel bech; brøndum, eva; tupper, oliver djurhuus; sørensen, susanne wiemann; alstrup, gitte; laursen, christian borbjerg; møller, ulla weinrich; sverrild, asger; jensen, jens-ulrik stæhr title: proactive prophylaxis with azithromycin and hydroxychloroquine in hospitalised patients with covid- (propac-covid): a structured summary of a study protocol for a randomised controlled trial date: - - journal: trials doi: . /s - - - sha: doc_id: cord_uid: jh klbg objectives: the aim of this randomised gcp-controlled trial is to clarify whether combination therapy with the antibiotic azithromycin and hydroxychloroquine via anti-inflammation/immune modulation, antiviral efficacy and pre-emptive treatment of supra-infections can shorten hospitalisation duration for patients with covid- (measured as "days alive and out of hospital" as the primary outcome), reduce the risk of non- invasive ventilation, treatment in the intensive care unit and death. trial design: this is a multi-centre, randomised, placebo-controlled, -arm ratio : , parallel group double-blind study. participants: participants are recruited at the trial sites/hospitals, where the study will take place in denmark: aalborg, bispebjerg, gentofte, herlev, hillerød, hvidovre, odense and slagelse hospitals. inclusion criteria: • patient admitted to danish emergency departments, respiratory medicine departments or internal medicine departments • age≥ years • hospitalized ≤ hours • positive covid- test / diagnosis during the hospitalization (confirmed). • men or non-fertile women. fertile women* must not be pregnant, i.e. negative pregnancy test must be available at inclusion • informed consent signed by the patient *defined as after menarche and until postmenopausal (no menstruation for months) exclusion criteria: • at the time of recruitment, the patient uses > lo /min (equivalent to % fio if measured) • known intolerance/allergy to azithromycin or hydroxychloroquine or hypersensitivity to quinine or -aminoquinoline derivatives • neurogenic hearing loss • psoriasis • retinopathy • maculopathy • visual field changes • breastfeeding • severe liver diseases other than amoebiasis (inr> . spontaneously) • severe gastrointestinal, neurological and hematological disorders (investigator-assessed) • egfr < ml/min/ . m • clinically significant cardiac conduction disorders/arrhythmias or prolonged qtc interval (qtc (f) of> / ms). • myasthenia gravis • treatment with digoxin* • glucose- -phosphate dehydrogenase deficiency • porphyria • hypoglycaemia (blood glucose at any time since hospitalization of < . mmol/l) • severe mental illness which significantly impedes cooperation • severe linguistic problems that significantly hinder cooperation • treatment with ergot alkaloids *the patient must not be treated with digoxin for the duration of the intervention. for atrial fibrillation/flutter, select according to the cardiovascular national treatment guide (nbv): calcium antagonist, beta blocker, direct current (dc) conversion or amiodarone. in case of urgent need for digoxin treatment (contraindication for the aforementioned equal alternatives), the test drug should be paused, and ecg should be taken daily. intervention and comparator: control group: the control group will receive the standard treatment + placebo for both types of intervention medication at all times. if part or all the intervention therapy being investigated becomes standard treatment during the study, this may also be offered to the control group. intervention group: the patients in the intervention group will also receive standard care. immediately after randomisation to the intervention group, the patient will begin treatment with: azithromycin: day - : mg x day - : mg x if the patient is unable to take the medication orally by themselves, the medication will, if possible, be administered by either stomach-feeding tube, or alternatively, temporary be changed to clarithromycin mg x (this only in agreement with either study coordinator pradeesh sivapalan or principal investigator jens-ulrik stæhr jensen). this will also be done in the control group if necessary. the patient will switch back to azithromycin when possible. hydroxychloroquine: furthermore, the patient will be treated with hydroxychloroquine as follows: day - : mg x main outcomes: • number of days alive and discharged from hospital within days (summarises both whether the patient is alive and discharged from hospital) ("days alive and out of hospital") randomisation: the sponsor (chronic obstructive pulmonary disease trial network, cop:trin) generates a randomisation sequence. randomisation will be in blocks of unknown size and the final allocation will be via an encrypted website (redcap). there will be stratification for age (> years vs. <= years), site of recruitment and whether the patient has any of the following chronic lung diseases: copd, asthma, bronchiectasis, interstitial lung disease (yes vs. no). blinding (masking): participants and study personnel will both be blinded, i.e. neither will know which group the participant is allocated to. numbers to be randomised (sample size): this study requires patients randomised : with in each group. trial status: protocol version . , from april , . recruitment is ongoing (first patient recruited april , ; final patient expected to be recruited october , ). trial registration: clinicaltrials.gov identifier: nct (registered march , ) full protocol: the full protocol is attached as an additional file, accessible from the trials website (additional file ). in the interest in expediting dissemination of this material, the familiar formatting has been eliminated; this letter serves as a summary of the key elements of the full protocol. the study protocol has been reported in accordance with the standard protocol items: recommendations for clinical interventional trials (spirit) guidelines (additional file ). the drug is approved and marketed in denmark for the prevention and treatment of malaria, for the treatment of rheumatoid arthritis, discoid and systemic lupus erythematosus and juvenile idiopathic arthritis. manufacturing, packaging and labelling of imp: the trial drugs (imp) are manufactured by glostrup pharmacy by pharmacist kristian Østergaard nielsen. placebo capsules are thus made similar to the intervention medicine. the drug (and placebo) are labelled, according to appendix . glostrup pharmacy has a key for blinding. therefore, it will always be possible to unblind a patient if indicated. investigator must notify sponsor on grounds if a patient is unblinded. medical professionals dispense the imp daily during hospitalization, except for patients preferring to handle their medication themselves. in the latter case, patient will fill in a medication diary (provided at randomisation). for all patients who are discharged during the intervention period, a medication diary will be provided. the medication diary is subsequently collected. exactly the amount of medication or placebo that the patient is required to take during the study periode will be provided, but for patients who do not take all the medication (protocol deviation), the remaining medication will be collected by the study staff. the patient will receive a follow-up phone call to check whether they have adhered to the medication schedule according to the trial protocol. proactive prophylaxis with azithromycin and chloroquine in hospitalised patients with covid- (propac-covid) . hypothesis: in patients with acute hospital admission, a positive test for -ncov and symptoms of covid- disease, treatment with virus-modifying agent hydroxychloroquine as well as virusimmunomodulatory and antibacterial drug azithromycin can lead to shorter hospitalisation and fewer admissions to the intensive care unit. the aim of this randomised gcp-controlled trial is to clarify whether combination therapy with macrolide azithromycin and hydroxychloroquine via anti-inflammation/immune modulation, antiviral efficacy and pre-emptive treatment of supra-infections can shorten hospitalisation duration (measured as "days alive and out of hospital"as the primary outcome), reduce the risk of non-invasive ventilation, treatment in the intensive care unit and death. in the ongoing coronavirus pandemic, covid- , with its origin in wuhan, china, there is still sparse data on the course, risk of various complications, and the best possible treatment of patients admitted to hospital to ensure best possible survival and reducing length of stay at hospital. the most frequent symptoms are fever (> %) and cough ( - %), together with radiologically fingdings of "ground-glass infiltrates" or "patchy infiltrates" in the patients with the most severy ill patients ( %), compatible with severe viral pneumonitis ( , ) . the length of hospitalisation is observed to be relatively long, - days ( ) , which in itself is a problem as hospitals can quickly reach the maximum capacity for hospitalisation and the proportion of patients who become critically ill have, based on the observations reported so far, had a mortality rate of> % ( ), and overall mortality for admitted patients in china with covid- infection is apparently unusually high for viral respiratory tract infections with an estimate of % ( ). only specific data on patients with chronic obstructive pulmonary disease (copd) have been reported in a few studies, but the risk of in-hospital death appears to be very high (or . [ % ci . - . ]) ( ) . proactive prophylaxis with azithromycin and chloroquine in hospitalised patients with covid- (propac-covid) despite the rapid spread of the disease globally, there is no solid data yet to recommend any specific treatments, and by that, symptomatic, organ supportive therapy is recommended, and in case of progression to severe acute respiratory failure mechanical ventilation ( ) . a high incidence of bacterial super-infections has been reported in patients with covid- who died ( %) compared to survivors ( %) (p < . ), and likewise, an incidence of septic shock of % and %, respectively ( ) . thus, there is an urgent need for treatments that can improve the course of the diseases in the individual patients, including positive impact on risk for hospital admission, duration of hospitalisation, risk of secondary infections and death. azithromycin is a macrolide antibiotic that has shown convincing efficacy in several studies in recent years to reduce hospitalisation-related exacerbations in copd ( , ) , and to reduce exacerbationrate in asthma ( ) and non-cystic fibrosis bronchiectasis ( ) . at the same time, it has been shown that azithromycin has a distinct effect by down-regulating airway inflammation by reducing cxcl , tnf-alpha, il- and il- p ( ) . furthermore, a strong association has also been reported between survival from acute respiratory distress syndrome (ards) and administration of azithromycin (hr for days of death for all causes: . [ % ci . - . ] in a well-conducted study ( ) . furthermore, it has been consistently observed in several recent publications that azithromycin itself appears to have an antiviral effect on a number of several viruses causing respiratory tract infections, such as respiratory syncytial virus (rsv) ( ) , rhinovirus ( ) and zika virus ( ) . hydroxychloroquine has been marketed since , originally developed for prophylaxis and treatment of malaria, but has for years also been used an anti-inflammatory agent for rheumatic diseases. large daily doses (up to mg a day) of hydroxychloroquine are prescribed over many years to patients with arthritis such as systemic lupus erythematosus and rheumatoid arthritis for anti-inflammatory purposes, which is generally well-tolerated ( ) . but in addition to these effects, it is well described that the drug has an antiviral effect especially against flavivirus, retrovirus and coronavirus by inhibiting a number of low-ph-dependent steps in virus replication, as well as by inhibiting the ph-dependent endosomal mediated viral uptake in cells ( ) . the drug is well tolerated even with high dosage, for up to five years and there is no signal for birth defects with usage of the drug summarised by savarino et al. ( ) . cell studies with primate cells infected with the coronavirus that induced sars- (formerly called sars) have shown that chloroquine, in a dose-dependent manner, inhibits the ability of the corona virus to infect cells and to spread among cells ( ) . thus, several researchers and health care professionals have, during the present sars-cov- pandemic, have proposed studies examining hydroxychloroquine/chloroquine as treatment for patients with covid- disease ( , ) . proactive prophylaxis with azithromycin and chloroquine in hospitalised patients with covid- (propac-covid) the study will clarify whether treatment with azithromycin in combination with hydroxychloroquine for days from the time of hospital admission with diagnosed covid- disease may reduce the length of hospitalisation, the risk of admission to the intensive care unit, treatment with non-invasive ventilation and death. the study will also clarify whether this treatment can reduce the need for oxygen supplementation (time for breathing on its own without oxygen supplementation) or for regular long-term oxygen therapy oxygen supplementation ("home oxygen"). if the treatment also improves the course of covid- disease in patients with pre-existing lung disease, a very large number of patients could benefit from the treatment immediately. the study originates from the danish national non-commercial lung research network cop:trin (chronic obstructive pulmonary disease: trial network). . design: randomised, good-clinical-practice-monitored, placebo-controlled, double-blind study. see point . inclusion criteria: interactions should be taken into account if the patient is taking other medications. for azithromycin, this includes antacids, ergotamine derivatives, colchicine and cyclosporine. for hydroxychloroquine, these include antidiabetic agents, tricyclic antidepressants, antipsychotics, halofantrine, cyclosporine, mefloquine, antiepileptic drugs, praziquantel and agalsidase. as there is no specific treatment for covid- , standard assessment and treatment is based on organ supportive therapy such as oxygen therapy (central), fluid therapy, antibiotic therapy for secondary infections. if the disease progress to severe acute respiratory failure, the patients will often require referral to an intensive care unit for mechanical ventilation. in addition, danish national guidelines for handling of in-hospital covid- patients can be obtained from www.lungemedicin.dk or from the open access journal european respiratory clinical journal (at present in press). the sponsor generates a randomisation sequence. randomisation will be in blocks of unknown size and the final allocation will be via an encrypted website (redcap), where also inclusion and exclusion criteria are required to be filled in correctly in order to randomise a patient. allocation will be stratified for age (> years vs. <= years), site of recruitment and whether the patient has any of the following chronic lung diseases: copd, asthma, bronchiectasis, interstitial lung disease (yes vs. no). the primary daily project management is carried out by the project manager. in addition, a project group (investigators), consisting of doctors from the departments involved, is trained to assist the project manager with the recruitment, sampling and follow-up of patients. all medical decisions regarding patients will be taken by a physician. data is collected in case report forms (crf) for each individual patient. prior to consent to participate in the trial, we will only assess the specific information needed to assess inclusion and exclusion criteria. no other information will be accessed. it is the attending physician who asks if patients are interested in hearing more about the trial. if yes, an investigator is contacted, who will inform the patient about the trial. as part of the study, all patients will be regularly monitored for oxygen saturation, heart rate, blood pressure, respiratory rate and temperature during hospitalisation. the following information will also be obtained: all this information is passed on to the investigator. the information from the medical records is required to calculate demographic data, medication data and outcomes in the trial. no information that is not required according to the protocol will be obtained. case report form is archived at the departments involved for years. a separate database is created in redcap (www.projectredcap.org) for data management. *the blood samples include haemoglobin (hb), leukocytes + differential count, thrombocytes, creactive protein (crp), na+, k+, albumin, creatinine, urea, amylase, alkalic phosphatase, beta- -microglubulin, fibrinogen, glucose, tsh, inr, billirubin, d-dimer, aptt, calcium, triglycerides, ferritin and lactate dehydrogenase (ldh). these blood tests will also be recommended daily for covid patients outside studies in the recommendation of the danish lung medicine association. **when screening for the study, any ecg from within the last days can be used. ***a follow-up ecg can be recorded during any remaining days of the hospital admission. ****only in patients with copd. neither patients nor study staff will know which group the patient is allocated to. the medicine will be marked neutral, e.g. "azithromycin group a" and "azithromycin group b" and the same for hydroxychloroquine. note: if the patient is receiving azithromycin prophylaxis, common practice is followed: the prophylaxis is paused and then restarted as usual. control group: the control group will receive the standard treatment + placebo for both types of intervention imp at all times. if part or all the intervention therapy being investigated becomes standard treatment during the study, this may also be offered to the control group. intervention group: the patients in the intervention group will also receive standard care. immediately after randomisation to the intervention group, the patient will begin treatment with : azithromycin: if the patient is unable to take the medication orally by themselves, the medication will, if possible, be administered by either stomach-probe, or alternatively, temporary be changed to clarithromycin mg x (this only in agreement with either study coordinator pradeesh sivapalan or principal investigator jens-ulrik staehr jensen). this will also be done in the control group if necessary. the patient will switch back to azithromycin when possible. hydroxychloroquine: furthermore, the patient will be treated with hydroxychloroquine as follows: follow-up is done on days , , and days, by accessing the electronic medical record system. the specific information obtained, and its purpose can be found in section . . . the summary of product characteristics for azithromycin suggests mg/day for three days or mg/day for one day and then mg daily for four days. however, other clinical studies have found a positive effect of a daily dose of mg for prolonged periods as prophylactic treatment. mortality among hospitalised patients with covid- is quite high and the median time for hospitalisation is - days, so it seems reasonable to give patients prophylactic dose for days. the dosing of hydroxychloroquine follows the summary of product characteristics. regarding other treatment with antibiotics: if antibiotic therapy is deemed indicated to the patient due to e.g. pneumonia or if it becomes standard therapy, piperacillin-tazobactam should be given as an empirical treatment at a dose adjusted to renal function. in case of penicillin allergy, cefuroxime is also given at a dose appropriate to renal function, weight and age. when positive microbiology is available, immediately switch to targeted treatment. if specific suspicion of atypical pneumonia is raised, ciprofloxacin is administered at a dose of corresponding to kidney function and concomitant examination for atypical pneumonia will be performed. if negative, ciprofloxacin is discontinued. if positive, ciprofloxacin treatment is continued for the duration of treatment corresponding to the microorganism detected. if there is a specific need for treatment with macrolide and where other options are not available (e.g. allergy to fluoroquinolones, or when there is an estimated need for combination treatment of e.g. legionella pneumonia), consult with an investigator, and in this case it may be decided to discontinue azithromycin (active) or azithromycin placebo. in this case, treatment stops without unblinding. furthermore, ecg recordings during the treatment period will be analysed with focus on qtc. at qtc (f)> / ms for respectively women and men, imp will be discontinued for safety reasons (but the patient remains in the study). primary endpoint: this requires patients randomised : with in each group. this is a fixed sample size. it is assumed that most patients complete the intervention. however, for interim analysis, the data and safety monitoring board (dsmb) may recommend the steering committee to expand sample size. frequency and depth are determined by the gcp units. initiation visits and the first monitoring visits to all centres will be conducted off-site, i.e. without a physical meeting, due to the sars-cov- pandemic. consent sheets will be scanned into an online system (redcap or journal system) that can be accessed by gcp monitors. after recruiting half the sample size (approximately patients), an interim analysis will be performed focusing on safety. an external data and safety monitoring board is appointed. the interim analysis will be prepared and presented by physician josefin eklöf. the groups will be presented as "group a" and "group b" and dsmb will only be unblinded if they ask the steering committee for the study on this. as part of ususal care, blood samples are taken daily from the time of inclusion and as long as the patient is admitted. blood samples include haemoglobin (hb), leukocytes + differential count, thrombocytes, c-reactive protein (crp), na+, k+, albumin, creatinine, urea, amylase, alkalic phosphatase, beta- -microglobulin, fibrinogen, glucose, tsh, d-dimer, aptt, calcium, triglycerides, ferritin, bilirubin, alat, inr, and lactate dehydrogenase (ldh), see table . these blood samples are analysed at the hospitals. in addition, supplemental blood tests and material obtained with nasal swaps will be performed according to the sub-study protocols. material from this will be included in a research biobank for the trial, and after completion of the trial in another regional biobank. the trial is expected to end in february , and the material will then be transferred to the regional biobank. for the present trial, the results of blood tests are collected from the patient record. the treating physician may at any time discontinue intervention with imp if, in clinical and/or paraclinical assessment, it is deemed contraindicated. serious side effects to regular blood sampling (venous puncture) are rare. frequent ( - %) can be seen transient discoloration of skin around the insertion site. chest x-rays correspond to a radiation dose of approx. . millisievert (msv). this should be compared with the average background radiation in denmark of approx. msv per year. there are no documented adverse effects of the radiation dose received by chest x-rays in the literature. therefore, we believe that the study is not associated with any risks or side effects. • with pigment change requires careful dosing and careful control. when using rheumatologic doses, eye examination by an ophthalmologist is recommended before starting treatment and with follow-up to check for any eye manifestations that may arise. annual monitoring after years of treatment is recommended, however in risk patients initially annual control, see also chloroquine derivatives (inflammatory rheumatic diseases), side effects. • with cardiomyopathy can be fatal. • with macular degeneration is seen and may be irreversible. • with reversible corneal changes with oedema and blemishes can cause blurred vision or photophobia. • with blurred vision accommodation is dose-dependent and reversible. • in malaria treatment and prophylaxis, fewer and milder side effects occur. **prolonged qt interval has been seen in patients with particular risk factors for it. ***acute generalized exanthematous pustulosis must be distinguished from psoriasis. psoriasis exacerbation may occur. may be associated with fever and hyperleukocytosis. an adverse reaction (ar) is defined as any adverse and undesirable reaction to a trial drug regardless of dose. an adverse event (ae) is defined as any adverse event in a patient or subject in a clinical trial following treatment with a drug, without necessarily linking this treatment to the adverse event. since the trial drugs are well known and have been used for many years, we will only record side effects not mentioned in the respective drug summary of the trial drug. a severe adverse reaction or event (sar/sae) is defined as an event or adverse event that, regardless of dose, results in death, is life-threatening, results in hospitalization or prolongs hospitalization, results in significant or persistent disability or incapacity, or leading to a congenital anomaly or malformation. investigators must immediately (= within hours) report serious incidents and serious adverse reactions (saes and sars) to the sponsor regardless of whether they are described in the respective product summary. this allows the sponsor to assess the benefits and risks along the way in the study. events and adverse events recorded during the period from the patient have received the first dose of trial medication up to and including day . recording and reporting of all events and adverse events will end when the trial drug is stopped. a high degree of comorbidity and death is seen in this patient group and therefore it is also expected that prolonged admissions, re-admissions, niv, respirator treatment and death will occur in this patient group. therefore, these parameters will not be considered as a sae. all incidents and registered side effects are reported at the end of the trial in a final report to the danish medicines agency. all serious suspected adverse reactions must be reported annually together with a report on the safety of the subjects and sent to the the danish medicines agency (lmst) and the danish national committee on health research ethics (vek). the product summary of the trial drugs is used to assess whether a serious adverse event is unexpected and thus possibly a suspected unexpected serious adverse reactions (susar). in the event of a fatal or life-threatening susar, this must be registered and reported to lmst and vek within days of the sponsor becoming aware of it. no later than days after the report, the sponsor must provide lmst and vek with all relevant information about the sponsors and investigators' follow-up on the event. all other susars are reported to lmst and vek within days of the sponsor becoming aware of them. the report must be followed up by a detailed written report, and in both the immediate report and the subsequent report, the investigator must identify the subjects with a personal code number. when reporting deaths, the investigator must provide any additional information that the sponsor may request. the research project is (investigator) initiated by cop:trin. funding has been obtained from the novo nordisk foundation of dkk . million for sponsor, remuneration of auxiliary personnel, payment of laboratory tests and equipment, as well as for manufaction of imp treatment and placebo. the sponsors and investigators are not financially linked to private companies, foundations, etc. in this research project. medical expenses are covered, if not obtained from other sources, by the section for respiratory medicine research, gentofte hospital. to the extent possible, the section for respiratory medicine research, gentofte hospital supports follow-up for endpoints and otherwise by appointment. patients are not paid for participation in the trial. the consent gives the primary investigator, monitor and any control authority direct access to obtain information in the patient's medical record, etc., including electronic record, in order to see information about the subject's health conditions which are necessary as part of the implementation of the research project and for control purposes, including self-monitoring, quality-control etc. the project group that has designed and conducted this study has the right to data and the right (and duty) to publish based on data. project management manages data and invites members of the study group to publications. all sites that recruit patients are entitled to at least one authorship on the primary publication, and for every patients recruited, the site is entitled to an extra authorship. sites that have not participated in the design of the study are entitled to a maximum of authorships. it is the opinion of the steering committee that knowledge sharing creates more and better scientific results. requests for knowledge sharing from other groups may be submitted to project management (jens-ulrik jensen, charlotte ulrik, pradeesh sivapalan) who will evaluate primarily and who, if the project is found suitable, will discuss it with the cop:trin steering committee. project management has the first right to undertake sub-studies but may well assign projects to other contributors. in that case, the following considerations will be significant in the assessment: ) participation in the design phase of this rct and at what level, and ) number of patients recruited at a site. if the hypothesis to be investigated is not planned to be examined by our group, we will allow the use of our data if the steering committee finds the project scientifically sound and, if appropriate, a collaboration with members of the cop:trin steering committee will be proposed. however, it should be emphasized that data is used for a specific purpose, not for future purposes in general. this becomes conditional by the steering committee for data to be used in a sound way to test hypotheses with relevant scientific content. information regarding subjects are processed and stored in accordance with the data protection regulation (gdpr), the data protection act and the health act and the project is properly notified in accordance with applicable rules and laws to the appropriate authorities. all project results will be sought published in scientific contexts, including international journals. this will happen regardless of whether the result is positive, negative or inconclusive. the study is conducted in accordance with the declaration of helsinki and is carried out in accordance with the rules of the personal data act and the health act. the study has been registered at the danish data protection agency. recruitment and inclusion will take place as previously described (section . . ). participation requires a signed statement of consent. patients can withdraw their participation consent and withdraw from the research project at any time without this having any effect on their right to current or future treatment. furthermore, the patient is entitled to bring a bystander to the information interview and is entitled to reflection time before any declaration of consent is signed. the important objective of the study is to investigate whether pro-active and pre-emptive treatment against covid- can reduce the length of hospitalisation and the risk of intensive care and improve the survival of patients -an area that has so far been poorly researched and where the need for evidence-based guideline for handling and processing is large and very urgent. potential disadvantages and side effects are described in the separate section . among other things, it appears that the likelihood of serious adverse reactions to both treatment and examinations is rare. in addition, the treating physician can always discontinue treatment if it is considered contraindicated. placebo is given patients allocated to the control group as no specific standard medical treatment is available. the experimental method and statistical analyses have been carefully considered in order to be able to disseminate and apply relevant and secure research results to clinical practice. based on the above considerations, we believe that the experiment is sound ethically sound and can be conducted without exposing the test participant to unjustifiable risks. at each trial centre, screening of patients admitted with a positive sars-cov- test is performed. patients are assessed against the inclusion and exclusion criteria of the attending physician who receives the patient's consent to contact the investigator. the investigator then contacts the patient for recruitment to the study. disclosure of information about the study and obtaining informed consent may also be undertaken by other healthcare professionals. this includes research assistants (medical students), clinical nursing specialists and project nurses (see below for specific requirements). these are all separately trained in the task and have the opportunity to call a physician should any medical issues arise. they can also contact the coordinating investigator as well as a hotline team for the trial should any questions arise about informed consent. this hotline is available hours a day. all patients are offered a consultation with a physician if they ask for it. for project nurses, the following applies specifically: i) must have at least years of seniority. these requirements are verified by primary investigator from each site that afterwards creates a document for these individuals from which the above specific requirements are verified. this document is dated and signed. if a patient is considered suitable, the person will be invited to participate in the project. participation in the trial is voluntary. informed consent is obtained from the participants of the trial acc. to executive order no. of september on information and consent for participation in health science research projects and on notification and supervision of health science research projects. the first contact with the the potential participant in the trial will be at admission to one of the participating departments. participant information is provided both orally and in writing, and the patient is informed that they are entitled to hours of reflection time before consent is given for participation in the trial. participants who wish to do so themselves after the period of reflection time may give consent in connection with the information meeting. the right to a bystander is ensured by the patient being able to bring a bystander, however, subject to covid isolation rules. if no bystanders come to the first call, they are ensured afterwards to a bystander, when the patient is out of isolation. it is ensured that the conversations are undisturbed by using the patient's isolation room. if the doctor carries a "pager" or telephone, these are handed in prior to the call. the trial participant will be provided with the document "the research subject's rights in a health science research project", which contains information about confidentiality, access to documents and access to complaints. the subjects are protected under the personal data processing act. the trial has been reported to the regional science ethics committee, the danish medicines agency and the danish data protection agency. it must be ensured at all times that subjects have consented to participate in clinical trials. if an isolated subject with covid- can sign consent declaration via electronic tool, this can be used instead of consent with signature. this can be, for example, a mobile phone, an ipad, a laptop with secure identification, for example by an easy id (or other solution that meets the oces standard). if the above described solution is not possible, the following solutions can be used as temporary documentation for the consent: • copy of signed consent declaration -e.g. using camera: the subject can sign the consent form as usual. since the signed form must not leave the isolation room, the signature can be documented in the form of a photograph of the signed form, for example through a window. • if the test subject cannot sign the consent declaration himself, e.g. due to problems with having electronic equipment in the room, or obtaining documentation for the consent out of the room, the witness can sign on behalf of the subject: if the subject verbally consents, a witness can on behalf of the subject sign the consent form. for both of the above solutions, documentation (photo and witness signature) will be filed in the investigator's section of the trial master file (tmf). furthermore, it is ensured that the data protection regulation and the data protection act are complied with, although documentation of the consent is temporarily different than it usually is. if the situation is normalised, the correct signed consent form must be obtained from the subject as soon as possible. regular monitoring and quality control of the study will be carried out. if the physician responsible for the study deems it necessary, the physician may during treatment take the subject out of the study. the physician may also terminate the study at any time if there is a medical justification (such as the development of allergies to medicines), a safety risk or a requirement from the authorities. the test subject may also withdraw their informed consent and withdraw from the investigation at any time, as mentioned in the above paragraph. patients who participate in these studies and who believe they have suffered injury can seek compensation through the patient compensation (http://patienterstatningen.dk/) cf. danish law. common ( - %) powerlessness, decreased appetite. vomiting, taste disorders. decreased lymphocyte count. decreased serum bicarbonate. arthralgia. headaches, paraesthesia, dizziness. skin itching, rash uncommon ( . - %) pain. hepatitis, oral candidiasis. dyspnea, pneumonia, oedema. eosinophilia, leukopenia, neutropenia. elevated serum bicarbonate, hyperchloremia, hyperglycaemia, hyperkalaemia, hypernatremia, hypokalaemia, hyponatraemia. arthritis, back pain. nervousness, somnolence. facial oedema liver impact. agitation. acute generalized exanthematous pustulosis*, allergic reactions*, angioedema*, hypersensitivity. not known fulminant hepatitis, hepatotoxicity, hepatic insufficiency, pancreatitis, pseudomembranous colitis. arrhythmias, extended qt interval, hypotension, torsades de pointes tachycardia. haemolytic anaemia, thrombocytopenia dress -drug reaction with eosinophilia and systemic symptoms, erythema multiforme, stevens-johnson syndrome, toxic epidermal necrolysis. anaphylactic reaction. acute renal failure *in case of allergic reactions, including acute generalized exanthematous pustulosis and dress, azithromycin should be discontinued. hydroxychloroquine: very common (> %) abdominal pain common ( - %) eating refusals. diarrhoea, vomiting. emotional lability, headaches. skin itching, rash. accommodation difficulty aggravation of porphyria, hypoglycaemia. myopathy. extrapyramidal genes, cramps, neuropathy, palsy, psychosis, suicidal behaviour. acute generalized exanthematous pustulosis***, exfoliative dermatitis, erythema multiforme, photosensitivity, stevens-johnson syndrome, toxic epidermal necrolysis. allergic reactions (including dress syndrome proactive prophylaxis with azithromycin and chloroquine in hospitalised patients with covid- clinical characteristics of coronavirus disease in china clinical course and risk factors for mortality of adult inpatients with covid- in wuhan, china: a retrospective cohort study clinical characteristics of refractory covid- pneumonia in wuhan, china clinical course and outcomes of critically ill patients with sars-cov- pneumonia in wuhan, china: a single-centered, retrospective, observational study care for critically ill patients with covid- azithromycin maintenance treatment in patients with frequent exacerbations of chronic obstructive pulmonary disease (columbus): a randomised, double-blind, placebo-controlled trial azithromycin for prevention of exacerbations of copd effect of azithromycin on asthma exacerbations and quality of life in adults with persistent uncontrolled asthma (amazes): a randomised, double-blind, placebo-controlled trial proactive prophylaxis with azithromycin and chloroquine in hospitalised patients with covid- (propac-covid) effect of azithromycin maintenance treatment on infectious exacerbations among patients with non-cystic fibrosis bronchiectasis: the bat randomized controlled trial randomised, double-blind, placebo-controlled trial with azithromycin selects for anti-inflammatory microbial metabolites in the emphysematous lung adjunctive therapy with azithromycin for moderate and severe acute respiratory distress syndrome: a retrospective, propensity score-matching analysis of prospectively collected data at a single center azithromycin attenuates airway inflammation in a mouse model of viral bronchiolitis azithromycin induces anti-viral responses in bronchial epithelial cells azithromycin protects against zika virus infection by upregulating virus-induced type i and iii interferon responses effects of chloroquine on viral infections: an old drug against today's diseases? the role of antimalarial agents in the treatment of sle and lupus nephritis chloroquine is a potent inhibitor of sars coronavirus infection and spread chloroquine for the novel coronavirus sars-cov- breakthrough: chloroquine phosphate has shown apparent efficacy in treatment of covid- associated pneumonia in clinical studies key: cord- -ip i bdk authors: jeon, juyeun; kim, hyeikyoung; yu, kyung-sang title: the impact of covid- on the conduct of clinical trials for medical products in korea date: - - journal: j korean med sci doi: . /jkms. . .e sha: doc_id: cord_uid: ip i bdk background: the number of clinical trials conducted in korea continues to increase and an increasing proportion focus on severe and rare incurable diseases. after the start of the severe acute respiratory syndrome, coronavirus disease (covid- ), korea centers for disease control and prevention (kcdc) developed guidelines to prevent the spread of infection. this study evaluated the impact of covid- and the kcdc guideline on the conduct of clinical research in korea. the purpose was to develop recommendations on how to minimize the risk of infection while enabling subjects to take part in the trials if no better alternative treatment options were available. methods: the impact on subject's scheduled visits and major milestones of clinical trials in korea were measured by conducting a survey among clinical project manager (cpms) working at global clinical research organization. the policy on monitor's access to hospital and site initiation meetings was investigated through correspondence with clinical trial center of hospitals. the top pharmaceutical companies' official press and public clinical trial registry database were used to analyze companies' trial strategy during the pandemic and covid- clinical research status, respectively. results: of cpms, % reported that trial subjects' scheduled visits had been affected in their project. monitors' access to hospital for source data verification was restricted at all sites in february . accordingly, % of cpms reported that the covid- epidemic had an effect on study major milestones and data cleaning and database lock accounted for > % of milestones affected. in addition, % sites advised not to have site initiation meetings and % pharmaceutical companies suspended recruitment or new study start-up due to the pandemic. on the other hands, the number of covid- related clinical trials increased rapidly in korea and worldwide, with investigator-initiated trials accounting for % and % of all trials locally and globally, respectively. most trials were phase and were in the recruitment stage. conclusion: the covid- and the kcdc guideline influenced all parties involved in clinical trials in korea. in order to ensure the safety and well-being of trial subjects during the pandemic, new approaches are required for clinical trials to respond to the impact actively. method of non-contact is developed to replace and supplement the face-to-face contact and alternatives to reduce the travel is introduced to decrease the risk of infection for all trial participants in whole trial process. the relevant regulations should be developed and the guidelines for foreign countries need to be adopted in accordance with the situation in korea. covid- trial is rapidly increasing worldwide and continuous support of health authorities, regulation, and facilities is required for developing the treatments with protecting all trial participants. in terms of the clinical trial share, korea ranks eighth globally and is in third place in a single country clinical trial basis. the number of approved clinical trials for medical product conducted in korea increased from cases in to in . the ministry of food and drug safety (mfds) has reported an increasing trend of trials for severe and rare incurable diseases. anti-cancer clinical trials account for . % of all clinical trials, and have been the most common indication of clinical trial for consecutive years. the conduct of clinical trials consists of a series of processes and multiple parties are involved in the overall trial process. accordingly, face to face contact, meetings and travel to designated places are required (fig. ) . coronavirus disease (covid- ) has spread globally since the first outbreak in december from wuhan in hubei province, china. the first case in korea was confirmed on january , , in an individual who entered korea from wuhan. compared to previous respiratory syndrome, middle east respiratory syndrome (mers) epidemic that occurred in - , covid- had greater community spread. the first case not linked with epidemiologic evidence, identified on february , , accelerated increase in the number of confirmed cases due to community transmission. the korea centers for disease control and prevention (kcdc) raised the disease alert to the highest level on february , to heighten the response (fig. ) . as there was no available treatment approved for covid - in korea yet, the government emphasized prevention, to avoid the spread of infection. this strategy influenced not only personal lifestyles but also the conduct of clinical trials of medical products. there were no exceptions made to compliance with kcdc guideline when conducting clinical trials, because of the concern about infection during the operation of trial. the guideline called on all citizens to maintain social distancing, refrain from visiting crowded places and reduce gatherings as well as outdoor activities. conducting clinical trials results in an increased number of subject visits to hospitals, face to face contact and meetings. there was thus a conflict between being compliant with the kcdc guideline firmly and clinical trials activities, including encouraging subjects' continuous participation. new approaches were necessary in clinical trials to eliminate the risk of infection by complying with the guideline and enable subjects to continue to participate in trials if no better alternative treatment options were available, for protecting the subjects' safety and well-being. the impact of the covid- pandemic has not been addressed on clinical trials in korea. the study evaluated the impact of the covid- epidemic and the kcdc disease control guideline on the conduct of clinical research in korea, on subjects, investigators, monitor, pharmaceutical companies, institutional review boards (irbs) and regulatory authorities (ras), in order to suggest recommendations for conducting clinical trials during the pandemic. to measure the impact of covid- , the study selected individual indicators for each involved party in clinical trials. to evaluate the impact on subjects' study visits and study major milestones, a questionnaire was administered. the survey was distributed to total clinical project manager (cpms) who were working at global clinical research organization and responsible for trials performed in korea, according to method of simple random sampling from february , to march , . the questionnaire included questions on whether covid- had an effect on trial subjects' scheduled study visits and whether there was any major milestone influenced or expected to be delayed by covid- . the milestones could be described by the cpms and in the process of data analysis, they were classified as site initiation, close out, first patients in/last patient in/last patient out, data cleaning and database lock. if a cpm reported that there was an impact on the milestone however did not provide details, they were contacted to try to obtain further information. if response was not received, the impact was classified as not specified. the impact of the policy by trial sites on monitor's access to monitoring room and site initiation was investigated by contacting clinical trial center in hospitals. majority of hospitals ( %) were selected in seoul and gyeonggi-do where clinical trials were actively performed in korea and gyeongsangbuk-do and daegu where high infection rate was shown. they were contacted for the week of february , to march , , when the confirmed cases increased dramatically and the week of april to , , when the daily confirmed cases decreased less than in korea. the response on the access of monitor was classified as total closure, partially available and available, depending on the degree of access that monitors were permitted. the official press statement of the top pharmaceutical companies on the conduct of their clinical trials during covid- pandemic were reviewed in april and their decision were classified. , , of health were analyzed, during the week of june to , , according to the used investigational products, study status, phase and initiating parties, using the pivot table function. , for investigational products-based analysis, drug intervention studies for covid- from both database were included. the used drugs in each arm of individual trial were classified and if or more products were being administered in the same arm as a combination, each product was counted separately. of cpms, survey was retrieved from total ( %). of respondents, cpms provided valid answer on the question about the impact of covid- on subject study visits in korea and others left it blank. of cpms, % reported that they had an impact of covid- in their project, % reported there was no impact on the subject visits, % reported that they had no planned subject's visit during the survey period so the impact could not be estimated, and % reported that they didn't have active subjects in korea at the time of survey, because recruitment was not initiated, or all subject visits were already completed (fig. ) . the confirmed reason was that hospitals started restricting patient's physical visit from local cluster, daegu and gyeongsangbuk-do, with the highest infection rates, . % in korea, while seoul had a . % infection rate at the time of the survey. the number of outpatients visiting hospitals was lower during the covid- epidemic that in the corresponding period in and there were some trial subjects who requested to postpone hospitals visits because of concern about covid- . of total sites contacted, during the week of february , to march , , all sites ( %) restricted external visitor's access to the hospital, including that of monitors. four of the sites ( %) allowed only urgent monitoring to comply with the kcdc guideline. for the week of april to , , as the number of daily confirmed cases decreased, sites started opening the monitoring rooms again. of sites, ( %) partially opened the room with only limited seats available to maintain social distancing, ( %) made the room available, and ( %) remained closed the room but planned to reopen on may , (fig. ) . even the sites that had reopened the room, applied the guidance for preventing spread of infection. monitors who had visited countries with a high incidence of covid- or lived with someone who had visited a high incidence country within days were still restricted from visiting the monitoring room. some sites requested non-contact communication with site staff. of cpms, a total cpms ( %) answered the question of survey about impact on study milestone by covid- in korea. of cpms, % reported that they expected the impact in their project. % reported that study milestone was not affected, and % reported that the study had been completed so the question was not applicable. data cleaning and database lock were the most common milestone affected (> %), as a result of restriction of monitor's access to the sites which resulted in delayed source data verification (fig. ) . the site initiation meeting is an important milestone that marks the trial start at the site. some trial sites prohibited visits from subjects living in area with high infection rates, and subjects who were exposed to infectious agents were required to be quarantined according to the kcdc guideline, so some subjects were unable to attend scheduled study visits. although clinical trial drugs could be delivered in accordance with exceptional regulation permit, appropriate examination should be conducted for subject's safety by investigators. through temporally allowed remote assessment by mfds for trial subjects, it is required for investigator to evaluate the subjects' condition even during their isolation whether it is suitable to prescribe same investigational products repeatedly and whether laboratory tests could be omitted. as a final decision-making authority on continuation and suspension of their clinical trials, the limitation and reluctance of subject to visit hospitals and medical profession's burden for covid- control are the factors to consider when deciding whether to proceed with trials, based on risk and benefit assessments. , ra of united kingdom, medicines and healthcare products regulatory agency, advised for the early phase healthy volunteer trials, where there is no therapeutic benefit to the volunteer but taking part in the trial dose pose a risk of infection. to avoid adding to demands on healthcare system during this unprecedented crisis with covid- epidemic and to reduce the possibility of infection of trial participants, ( %) of the top pharmaceutical companies declared to hold recruitment or activation of the new trials at least partially. five ( %) companies continued ongoing trials in close discussion with the trial investigators in all the participating countries. seven ( %) pharmaceutical companies did not mention a change in their conduct of trials on their official website or in press release ( fig. ) . multiple countries are turning to pharmaceutical companies to develop coronavirus treatments, so accordingly, the number of covid- related clinical trials is growing very rapidly worldwide. a notable feature of the global covid- treatment clinical study is that the existing licensed treatment or new drug candidate material is being evaluated to determine whether they can be repurposed for treating in korea, the mfds approved a total of covid- clinical trials, in march and in each of the following months. two trials ( %) were for vaccine development and ( %) were for treatment development. a total covid- studies ( %) were recruiting patients or preparing for enrollment. of the trials, investigator-initiated trials and sponsor-initiated trials accounted for almost the same percentage, trial ( %) and trials ( %) each. among the sponsor-initiated trials, phase occupied highest portion ( %) (fig. ) . remdesivir was the most commonly used investigational drug ( trials), followed by hydroxychloroquine and nafamostat mesylate with trials. kyungpook national university hospital, located in daegu, and seoul medical center in seoul, which operated a professional negative-pressure isolation ward took part in the largest number of covid- clinical trials in korea, with trials each. globally, a total drug or biological agent intervention clinical trials were registered for covid- . in terms of study status, almost half ( %) were recruitment phase. regarding initiating parties, trials ( %) were investigator-initiated, ( %) were sponsor initiated and ( %) were sponsored by the us national institutes of health or us federal reserve, as of june , . the majority of studies were phase trials ( %) and treatment development trials ( %) (fig. ) . hydroxychloroquine was the most commonly used investigational drug ( table ) . multiple sites, they recommended that a joint review committee be established to review each study protocol and make a joint decision. to prevent the spread of covid- during review meetings, it was suggested that the review process be modified by having non-contact review meetings, and that this be documented. in order to support covid- drug development, the mfds applied for expedited review of covid- trial protocols. a trial of remdesivir for treatment of covid- , sponsored by gilead sciences, inc was submitted to the mfds for ra approval on february , and approved on march , . the us food and drug administration suggested that sponsors and clinical investigators document the reason for any contingency measures implemented, how restrictions related to covid- led to changes in the study conduct, which trial participants were affected and how these participants were affected in the clinical study report. the european medicine agency guided for regularities to put priority on any clinical trial application for the treatment or prevention of covid- infection, and the approval of applications for substantial amendments to existing clinical trials if necessary as a result of the covid- pandemic. the covid- pandemic is having a global impact. clinical trials are one of the areas that have been affected. clinical trials are meaningful in that they provide patients with novel treatment options during the process, and if they lead to marketing of new treatments, they may be of benefit to many future patients. moreover, considering that clinical trials are required for developing covid- treatments, conditions should be established in which necessary clinical trials can be carried out smoothly even during the pandemic, while protecting all trial participants' safety. in korea, all parties involved in the conduct of trials have been affected by covid- pandemic. various guidelines for conducting trials during the covid- pandemic have been released both locally and globally for supporting the operation of trials. guidelines developed by foreign countries should be considered and adopted in accordance with the situation in korea if they are judged to be appropriate. twelve percent cpms responded that subject study visits were not performed as planned according to the protocol. although the proportion of trials not affected ( %) was higher, given that the provision of treatment is contingent on assuring the safety of subject, % is a sizeable proportion that cannot be overlooked. if the treatment that subjects receive through clinical trial participation cannot be replaced by the standard of care, action should be taken to maintain the treatment while complying with the kcdc guideline. under the current article ( ) of the pharmaceutical affairs act, the sale of medicines outside pharmacies is prohibited in korea, which makes it impossible to deliver medicines by courier. however, mfds released a guideline that allows investigational products to be delivered to subjects directly on a temporary basis, to enable them to receive the products while reducing the risk of sars-cov- infection that they could face on their way to hospital. the guideline stipulates that the products be stored according to protocol requirements during the delivery process by a certified carrier. the us fda guidance advises investigators to confirm that subjects voluntarily consent to receive investigational products via courier. the italy ra, decrees of the italian president of the council of ministers allow healthcare providers to provide an increased amount of investigational product, to cover an extended period of treatment if it is beneficial and safe for the subjects. for the products that are administered in a health care setting, the us fda permits utilize home nursing services or healthcare professionals to serve as alternative sites, in order to keep access to medical products from being cut off even if the products needs to be administered by professionals with specialized skills. the guidance of the european medicines agency (ema) states that it is acceptable for laboratory, imaging or other diagnostic tests to be conducted at a local laboratory or relevant clinical facility certified to perform such tests routinely. for the introduction of these foreign guidelines in korea, policies on the supply and demand of necessary personnel or the designation criteria of alternative institutions must first be in place. comparing monitoring room availability for source data verification in february and april , it is evident that it is affected by the number of covid- confirmed cases per day. given the possibility of worsening of the pandemic situation, monitoring rooms may be forced to close again and considering the limited seats available and the high demand on monitor to verify cumulative clinical trial data, measures should be introduced to verify clinical data efficiently with fewer site visits or the methods of obtaining clinical data that require less verification should be considered. risk-based monitoring can be recommended. it enables earlier detection of issues through focused centralized and remote monitoring activities, and targeted on-site visit only for key risk indicators. this can reduce the total number of on-site monitoring visits required. association of clinical research organizations has suggested that even after routine monitoring is resumed, it is not necessary to perform % source data verification if remote monitoring was closely performed during the closure of monitoring room. regulations in italy have allowed remote source data verification exceptionally, limited to covid- pandemic once these methods are described in a specific guideline by the sponsor and clinical research organization and approved by the personal data protection officer of the trial site. once the research is equipped with a mechanism for collecting data remotely and directly from the patient or vendor, by using methods such as electronic questionnaires, centralized laboratories or electronic image data, remote verification can supplement on-site monitoring. the increasing number of investigator-initiated trials and trials conducted under the auspices of us federal reserve and the us national institutes of health indicates that all parties are making concerted efforts to develop covid- treatments and vaccines. considering that the number of clinical trials of covid- related products is rapidly increasing and that most patients are being recruited, measures are needed to ensure an ongoing supply of clinical trial drugs. it is necessary to diversify supply and demand channels for raw materials and to establish regular import and export channels between countries. given that the studies target patients with sars-cov- infection, relevant regulations should be implemented and facilities should be supported to use non-contact methods in order to protect investigators and trial staff. as one of non-contact approaches, telephone consent process is allowed for patients during isolation. guidance on the management of clinical trials during the covid- pandemic by ema advises the presence of an impartial witness, if written consent by the trial participant is not possible. to enable trial facilities to maintain the non-contact between investigators and trial subjects, they should be well equipped for covid- trials with capabilities such as electronic informed consent or device to electronically scan and save the signed informed consent without bringing the form out of the subjects' ward. this study has some limitations. although a single indicator was used to identify the impact of the covid- pandemic on each party, the effects of more diverse areas needs to be assessed using a greater diversity of indicators. the analysis of covid- related trials was based on a search of clinicaltrials.gov, so the trials not registered in this database were not included in the analysis. we only consider the guidance from ras in the us, korea, european union and the united kingdom, so referring to the guidance of other ra could help to suggest more various measures. all ra guidelines put patient safety first, and mfds is making continuous efforts to conduct clinical trials that are beneficial to participants. health authorities and policymaker should refer to various guidelines and consider alternative measures to conduct trials during this pandemic situation, taking the scope of the application into consideration while prioritizing subject safety, so that the most appropriate method can be implemented and adopted in korea, in a timely manner. at this time, the impact of the covid- pandemic on clinical trials in korea has not been as marked as in some other countries. however, restrictions on subject study visits to hospital can directly impact on subject safety and monitor's restricted access to site caused a delayed safety review for subjects. therefore all the lessons learned and policies implemented internally and externally should be evaluated, in order to be prepared for persistence of the covid- pandemic or future pandemics. development of the required regulations, strategies, facilities, and platform could help korea not only adapt to the rapidly changed environments and protection of trial participants, but also emerge as a contributing country in the field of trials. in order to increase in the number of clinical trials for treatment of serious diseases, consideration should be given to developing regulations that will enable patients who have difficulty with visiting hospital physically even after pandemic is controlled, so that varying methods can be applied to patients with different clinical conditions. presentation of the results of approval of clinical trials for medicines in korea ich harmonized guideline integrated addendum to ich e (r ): guideline for good clinical practice ich e (r ) ich consensus guideline report on the epidemiological features of coronavirus disease (covid- ) outbreak in the republic of korea from daily status of middle east respiratory syndrome covid- response guideline th edition by central disaster management headquarters · central disease control headquarters korea pharmaceutical and bio-pharma manufacturers' association integrated drug information system ministry health and welfare. daily status of covid- considerations for clinical trials in a situation of covid- notice of training considerations for clinical trial workers according to corona situation united kingdom medicines and healthcare products regulatory agency. synopsis of mhra advice on management of clinical trials in relation to coronavirus analysis of domestic and international clinical trials guidance on conduct of clinical trials of medical products during covid- public health emergency guidance on the management of clinical trials during the covid- pandemic acro's considerations on monitoring during covid- clinical trials' management in italy during the covid- (coronavirus disease we express our sincere appreciation for all clinical project managers who conducted the survey and all clinical trial participants who contributed new drug developments. key: cord- -qp authors: kotsimbos, t.; humbert, m. title: pandemic treatments on trial: the bigger picture date: - - journal: eur respir j doi: . / . - sha: doc_id: cord_uid: qp the tension between immediately using any potentially useful novel therapy in covid and trialling all novel therapies as rigorously as possible is addressed from a bigger picture perspective. given the stated extremes above, there is a clear trade-off between "doing all that one can for individual patients with currently available information in a timely manner and despite significant uncertainty" and "group treatment of individuals to be enrolled in properly conducted but costly (effort, time and money) clinical trials for specific therapeutic approaches that will help inform the evidence-base for future patients". in many ways this is a false dichotomy as both are necessary frameworks with clearly defined validity and purpose, which should ideally sit comfortably next to each other. the lack of a clear transition zone between these two frames of reference necessitates ambidextrous frame-shifts. ideally, frame-shifting would be sensitive and responsive to the prevailing environment when health-care provision is at its best for any level of resource constraint. although this may be difficult to define in positive terms of agility, dynamic range and contextual fit across and within multiple frames, it is most easily identifiable by its absence. this absence is even more obvious when the "in-between" situation is excessively encroached upon as some sort of compensatory way forward. here, therapies are prescribed to patients for varying reasons that can only be partly rationalized on the basis of either individual or group interests. the resultant case series and uncontrolled observational studies therefore quickly transition from a reasonable, adaptive initial response to explore large efficacy and safety signals in poor clinical outcome settings to efforts that are maladaptive when excessive. at best, these efforts minimally progress either individual patient care decisions or the broader evidence-base necessary to inform integrating guidelines. at worst, unchecked biases may lead to false positive results whilst simultaneously diverting resources from potentially more useful strategies. that there has been an excess of small and uncontrolled trials in the setting of a covid pandemic which itself is very much an "n of " situation is therefore worthy of further reflection and exploration. whose needs are being met? how do we best explore and exploit our overall clinical management and scientific research approach during the covid pandemic? there is little argument with the upholding the public health principles and standards of supportive care in relatively mild community cases that are likely to recover without ill effect. but how do we minimize the risks of progression in more severe hospitalized cases, particularly older patients with risk factor co-morbidities? how do we maximize compassionate use of available, potentially useful but unproven therapies when no other alternatives exist? how do we minimize the risks of these very same therapies for individual patients? and how do we quickly establish and conduct difficult and costly randomized, controlled clinical trials for the most promising old and new therapies where there is a clear equipoise between possible benefits and potential risks? all these questions so far turn on what is hopefully an impartial assessment of what is in the best interests of the patient. however, there are also other potential factors at play, albeit very difficult to quantify. these include (in no specific order): i) the difficult research environment that would necessarily follow from health care systems that are overwhelmed in a pandemic setting; ii) specific resource and funding constraints in such systems; iii) widespread fear and anxiety at all levels of the community including healthcare providers; iv) clinician-researcher and other interested party belief systems and potential "savior" mentalities; v) potential pharma industry interests regarding expanding re-purposed drug possibilities; vi) potential biotech start-up/pharma interests regarding new agent fast tracking opportunities; vii) potential political and economic interests; and viii) potential media and socio-cultural factors. although it is impossible to quantify in any meaningful way the magnitude of these factors and how they may all interact with each other, an enlightened awareness of all these possibilities is an essential first step to both prevention and any countering response. it is also the quintessential function that underpins all drug evaluating regulatory agencies/processes that we discard or bypass at our peril. in the midst of a pandemic that has overwhelmed many health-care systems globally it is indeed very impressive that several, large collaborative clinical trials have already been established to meaningfully assess potential covid therapies including the world health organisation's "solidarity" trial (see below). ideally this list should be extended to all other therapies where there may be a potential benefit seemingly counterbalanced by potential risk. however, at all points decisions will need to be weighed and taken. which drugs to trial? which patients? what disease severity? what dose? what to measure?, and how?, and why? which endpoints? how to adjust for confounder variables? how many patients? additionally, analytic integrity is all. not enough design control and internal validity is threatened, too much and there will be a problem with external validity. even with well-controlled rcts for specific agents uncertainty can never be completely eliminated. the best we can hope for is to optimally quantify key uncertainties thereby elevating our certainty base for future decision-making. the more iterative this process can be, the better. notwithstanding this however, there will always be a tension between the ideal of increasingly covering all testing possibilities for completeness and the practicalities of getting on with the real-time demands of work to be done. paradoxically, the almost immediate establishment of a well-designed rct to test the combination antiviral treatment lopinavir-ritonavir in wuhan, china during the beginning of the pandemic exemplified this tension in a most unusual way when trial recruitment ceased early due to falling covid case numbers ( ). it is clearly not possible to immediately perform a well-controlled rct on all potentially useful therapies. hence, in trying to do everything one can for individual patients with severe disease who are likely to die it is compassionate and reasonable to empirically trial available products off-label where the existing information base suggests potential benefit outweighing any potential harm. this of course all turns on our shared understanding of "likely to die", "likely to benefit" and "potential for harm" at the very least. the more we use these therapies however in patients with less severe illness, where the uncontrolled benefit signal may be significantly "overcalled" by a positive natural history and the risks of harm may become increasingly unacceptable, the more it behoves us to go down the path of a properly controlled and conducted rct. excellence here is not the enemy of "good enough" as may be argued in the exceptional circumstances of a pandemic or indeed any uniquely uncertain area of clinical medicine but the standard that should always be aspired to. indeed, an enlightened view of excellence would expand its horizon to embrace "good enough" in the most excellent way possible for any specific situation. anything less than this is professionally and personally diminishing as it quickly contracts down to no-risk comfort zones and sets us down the murky path of "good enough for who" questions and excusable actions that are answerable to noone. under pandemic conditions of overwhelming health-care system stress and understandable urgencies to try untested therapies, the introduction of novel therapeutic practices may therefore take on a life of its own and become disproportionately uncoupled from any primary purpose. hence, randomly collected case series and uncontrolled smaller trials may proliferate without necessarily being in the best interests of either individual patients or specific patient cohorts. the example of hydroxychloroquine/chloroquine is a case in point where some early in vitro evidence of efficacy against sars-cov has been taken both "too far" and "not far enough". hence, it has been used empirically in a large, number of small and uncontrolled "trials" around the world despite its potential for cardiac toxicity, and in one larger, poorly controlled observational study no significant benefit and potential for significant harm signal ( ) was inconclusive thereby paving the way for rationalizing any position you may wish to take. more recently, the controversial results of a multinational registry study examining the risk/benefit ratio of hydroxychloroquine/chloroquine with or without macrolide for treatment of covid also weighed into the evidence debate only to now be retracted by the authors ( ) . add to this a heady mixture of high profile, celebrity supporters and trial by media and you have a recipe for compounding one crisis with another. also, very illuminating is the story of the antiviral remdisivir. from early, uncontrolled results that reported a clinical improvement in % of "analyzable" patients ( ) through to a negative rct study result involving patients which was stopped early due to reduced case numbers in china resulting in a trial with reduced power to detect any smaller clinically significant benefit with any certainty ( ) to further preliminary reports of potentially positive results in larger trials sponsored by the remdisivir patent holder ( ) . similar stories abound for many other therapies being used to manage hospitalized patients with more severe covid related illness including various respiratory support strategies and their combinations, a range of anticoagulation protocols, novel immunotherapeutic agents including the anti-il receptor monoclonal antibody tocilizumab, and various combination strategies and therapies ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) . at the other end of the spectrum are the much more inclusive, simplified and highly "capturing" rct approaches such as the who solidarity international clinical trial across multiple countries ( ) and the oxford university led recovery trial in the uk ( )both designed to compare different treatment options against usual of care for as many hospitalised covid patients as possible within their jurisdictions. with large number of enrolments helping to counter the large degree of individual case heterogeneity that comes with a simplified approach, the solidarity trial's international steering committee discontinued the trial's hydroxychloroquine and lopinavir/ritonavir arms in early july as interim trial results for both these treatment arms showed little or no reduction in the mortality of hospitalised covid patients ( ) . the large recovery trial in the uk (n> , as of th june ) almost simultaneously confirmed these findings for both hydroxychloroquine and lopinavir/ritonavir, and by contrast, revealed that low dose dexamethasone ( mg given once daily for up to ten days) reduces the risk of death by about one-third among patients receiving ventilation and by one-fifth in those requiring oxygen alone (with no benefit among those not requiring ventilatory support) ( , ) . other treatment arms in the recovery trial include azithromycin, tocilizumab and convalescent plasma ( ) . a closer look at what transpired during the influenza a h n ( ) pandemic -our most recent pandemic prior to covid , is very salient. patients were frequently treated for pandemic influenza viral pneumonia with drugs not specifically registered for this indication and rarely under circumstances of high-quality data capture. this resulted in experimental drugs being used largely on compassionate grounds with no real improvement in our understanding of the potential benefits and harms of specific treatments. however, further reflections on the above outcomes led some clinician-researchers to a unifying multicentre embrace of master protocols, adaptive trial platforms and overarching statistical plans. these are designed to build operational "checks and balances" into a modular study framework that remains congruent across time and space and promotes a culture of learning as you go. hence, although remap-cap was originally designed to study severe pneumonia requiring icu management in over sites across many countries, it was quickly adapted after the hin ( ) pandemic be pandemic influenza ready making it relatively easy to switch its focus to the current covid pandemic. a key feature of adaptive trial design is that researchers are required to add and remove treatment groups as the trial is running in order to minimize the diluting effects of futile treatments and maximize the signal to noise ratio of more promising treatments according to certain trigger thresholds. and so, no algorithmic approach can completely substitute for imaginatively insightful, thoughtfully rational and well-equilibrated ethical thinking. additionally, there is the increasing appreciation that although adaptive trial designs for treatments is a great start, they may not go far enough. given the extensive variation in clinical outcomes with covid infection in the population at large, we may well need to also integrate some well-chosen deeper phenotype and genotype factors into our adaptive trial designs and analyses. the benefits of any such extended approach will of course need to be weighed up against the potential for greatly increasing the logistic complexity of adaptive trials beyond what is practically manageable even with the potential help of enhanced machine learning protocols, and of course the inexorable law of diminishing returns. as always, appropriately managing individual patient case priorities is a central pillar of ethical clinical practice. this is true both within an exploratory n of framework and in exploiting n of many rcts. hence, although the uncontrolled, compassionate use of specific off-label therapies may be acceptable in one framework, a completely different framework is required to generate a quality evidence base for improved pandemic management practices now and in the future. additionally, within framework discussions matching rights and responsibilities is generally much more comfortable than between frameworks. and yet, individuals and the society they belong to are irrevocably intertwined and interdependent. negotiating the deep chasm between frameworks is a key challenge for treating clinicians so that we don't get lost in a multitude of poorly controlled studies but leap towards a hierarchy of larger clinical trials arranged according to likelihood of success based on early but uncertain signals in smaller numbers of patients. ethical leadership therefore requires us to be agile, proportionate and adaptive in aligning, balancing and contextualising patient needs across many dimensions both simultaneously and separately. during a pandemic these fundamentals are unchanged, although our responses now have to be both speedier and as complete as possible with a varying emphasis on either depending on the guiding framework and associated driving purpose. with this in mind, doing the most right thing, at the most right time, in the most right way for the one and the many is the ethicist's embrace of epistemological excellence and the flourishing of all. references: thoracic society of australia and new zealand h n influenza task force. influenza a/h n _ : australia and new zealand's winter of discontent clinical and epidemiological profile of patients with severe h n / pandemic influenza in australia and new zealand: an observational cohort study influenza epidemiology in patients admitted to sentinel australian hospitals in : the influenza complications alert network (flucan) the role of chest imaging in patient management during the covid- pandemic: a multinational consensus statement from the fleischner society systematic review of clinicaltrials.gov infectious disease trials treating influenza infection, from now and into the future. front immunol a trial of lopinavir-ritonavir in adults hospitalized with severe covid- observational study of hydroxychloroquine in hospitalized patients with covid- :nejm. oa hydroxychloroquine or chloroquine with or without a macrolide for treatment of covid- : a multinational registry analysis compassionate use of remdesivir for patients with severe covid- actt- study group members.remdesivir for the treatment of covid- -preliminary report asian critical care clinical trials group. intensive care management of coronavirus disease (covid- ): challenges and recommendations emerging pharmacotherapies for covid- tsiodras s; escmid study group for respiratory viruses. review of trials currently testing treatment and prevention of covid- how did we rapidly implement a convalescent plasma program? transfusion trials and tribulations: so many potential treatments, so few answers off-label use of tocilizumab for the treatment of sars-cov- pneumonia in successful recovery from severe covid- pneumonia after kidney transplantation: the interplay between immunosuppression and novel therapy including tocilizumab effective treatment of severe covid- patients with tocilizumab triple combination of interferon beta- b, lopinavir-ritonavir, and ribavirin in the treatment of patients admitted to hospital with covid- : an open-label, randomised, phase trial solidarity" clinical trial for covid- treatments: th why haven't others delivered more results? kai kupferschmidt effect of dexamethasone in hospitalized patients with covid- -preliminary report medrxiv nd key: cord- -nesvd a authors: singh, arjun gurmeet; chaturvedi, pankaj title: clinical trials during covid‐ date: - - journal: head neck doi: . /hed. sha: doc_id: cord_uid: nesvd a as this ever‐evolving pandemic lays itself, more of its impact is being understood. until recently, most guidelines were reported to aid in managing and treating suspected or confirmed cases. research institutions around the world are responding with a sense of confusion. some are continuing routinely, especially those who are overseeing clinical trials that could offer life‐saving therapies, particularly against the novel coronavirus. since research must continue even in the face of a shutdown, we aim to collate the currently available recommendations from various organizations and provide guidance to head and neck researchers across the world during these trying times. various challenges exist for researchers during the coronavirus pandemic that are resulting from restrictions at health care facilities, changes in availability of staff and unidentified possible interactions. some trial participants or investigators might also be required to self-isolate or be infected by the severe acute respiratory syndrome-like coronavirus- (sars-cov- ) causing a moral dilemma in keeping trials running. another issue specific to the head and neck region is the proximity to the areas through which the virus can be easily transmitted requiring further amendments to protocols and need for additional equipment. multi-institutional trial conduct can also be affected by the unavailability of investigational medicinal products (imps) due to the restrictions affecting supply chains across the world. all these factors might have an impact on the accrual, assessment, and provision of clinical trials. the impact of coronavirus pandemic needs to be considered on ongoing and new clinical trials. sponsors need to take into account the national guidelines and restrictive measures imposed including limitations of trial participants and staff confinements and their ability to perform visits, interviews and forms, and notification of adverse effects. funding agencies and sponsors should be informed regarding the impact the current situation might have on the trial protocol. currently, most of the trials are being run based on risk-benefit concerns and contingencies present locally with the highest priority given to the health and safety of the trial participants. in the event, the participant is unable to attend a scheduled visit, measures such as virtual or tele contact may be required to identify ensure adequate medical oversight. with all these limitations, it is of utmost importance to make provisions for data protection with adequate documentation. it is also important to note that international publishers have made clear that in the event of a public health emergency, information with immediate health implications would be disseminated without any restrictions. this document aims to collate the currently available recommendations from various organizations and provide guidance to head and neck researchers across the world during these trying times. regulatory bodies have currently paused approvals for non-covid- new trial registration unless requiring an expedited approval in the event of public health emergencies with few also advising to withhold enrollment. the study team should critically assess the feasibility of starting a clinical trial or enrolling new participants. the risk-benefit section of protocols should include the additional risks that the participant and trial workers could encounter due to covid- with adequate risk mitigation measures. most guidelines suggest a risk assessment by the sponsor weighed against the anticipated benefit for the participant and society before initiating enrollment. , in trials especially of the head and neck cancer, where the participants enrolled may be determined as being a risk group for covid- or in trial that involve increasing the risk to contract covid- such as chemotherapy related trials, should be carefully considered before starting or continuing such trials. ensuring safety of trial participants is paramount and decisions should be taken on an individual participant basis focusing on the potential benefit. decisions regarding further trial recruitment, continuing use of imps for enrolled participants and need to change the monitoring methods and site during an ongoing trial should be kept in mind. at any stage, it is very important for participants to be kept informed of changes to study and other plans that could impact their care. all guidelines suggest obtaining written informed consent before enrolling patients. if this is not possible due to covid- isolation measures or safety, electronic methods should be considered. if not possible electronically, a three way call phone or video communication with the investigator, patient, and an impartial witness and/or if requested, additional participants (eg, next in kin, spouse, etc.) have been suggested for verbal confirmation. if the signed consent form cannot be collected from the patient, an attested copy by the witness and investigator who participated in the call or a photograph of the signed informed consent with attestation of the investigator should suffice. the details of the above procedure should be included with the informed consent in the source notes for records. clinical investigators and sponsors should consult their institutional review boards (irbs) or institutional ethics committees (iecs) in deciding if the participants' safety and rights are best served and protected by participating in the ongoing study or by discontinuing the imp, intervention or even participation in the trial. most head and neck oncology trials are ongoing in an institutional setup where preemptive measures to contain the virus are already instituted. if the study is enrolling participants, provisions for investigator's safety should be available at the site. if not possible, the sponsor should choose to withhold further intervention or monitoring until such safety measures are in place for the investigator. most guidelines encourage sponsors and investigators to engage with their local irb/iec as early as possible as emergent changes are anticipated in protocols due to covid- . , , some sponsors may choose to redesign their studies to require less labor, at the same time in a way that salvages the study. the changes made to minimize the hazard or protect the life and well-being of participants and investigators may be implemented without irb approval before filing an amendment, but are required to be reported later. the amendment needs to include the specific protocol deviation and the reason for the deviation. for example, specific visits conducted by virtual contact rather than as specified in the protocol due to the current situation, generally would be acceptable. for efficacy assessments, protocol modifications are required for the collection of endpoints done via methods alternate to the ones specified in the protocol such as virtual assessments, delays and alternate collection of specimens. since trial participants may not be able to visit the site for the protocol specific visits and investigations, sponsors should evaluate if alternate measures such as virtual visits, alternate locations for assessment, including imaging centers and labs, could suffice when necessary, only after ensuring the safety of the participant. if an in-person visit is necessary, it is the sponsors' responsibility to assure the implementation of the monitoring safely. , this is important for trials that include participants who no longer have access to imps and who need additional safety monitoring (eg, for withdrawal of an active imp). . | should covid- screening be reported? some institutions might mandate screening all participants for covid- as a protocol. this does not need to be reported or amended in the protocol unless the testing data would be collected and used as a part of a new objective in the study. since the scheduled visits at the site may be significantly impacted, certain imps that require self-administration may be amenable for home delivery but should be documented. this may include infusions as well. where selfadministration is not possible and monitoring is required, alternate administration methods via home nurses or at smaller care facilities may be planned after obtaining approval from the local reviewing authorities. in all cases, it is important to maintain imps accountability before implementing any alternate methods. due to the changes in scheduled visits or patient's discontinuation, there may be missing information for the protocol specified procedures. nonetheless, it is important to capture all details in the case report form that explains the basis of this missing information. this information should be summarized in the study report. if changes in protocol, including missing data, will lead to amending the data collection and statistical analysis, the sponsor should consider informing the local review board before locking their databases. a pre-specified plan for analyses should be constituted to tackle the protocol deviations related to covid- . the need to apply new processes or to modify existing ones will vary based on the protocol and local situation. some considerations need to be made whether it is appropriate to delay assessments for an ongoing trial or to discontinue or withdraw participants if the study cannot be properly conducted under the existing protocols. it is important to keep the measures going that yield the most information for the least effort and resources. sponsors should describe the measures instituted in view of the ongoing pandemic that may include: • contingency measures to manage study conduct during disruption. • document and list all participants affected by and describe how the participation has altered. • analyze and discuss the impact of the contingency measures on safety and efficacy. q&a on the impact of covid- on research funded or supported by nihr c for de and clinical trial conduct during the covid- pandemic how to cite this article: singh ag, chaturvedi p. clinical trials during covid- the authors declare no conflicts of interest. all authors have made a significant contribution to this article from concept to implementation and publication. arjun gurmeet singh https://orcid.org/ - - - pankaj chaturvedi https://orcid.org/ - - - key: cord- - cah lg authors: arabi, yaseen m.; asiri, ayed y.; assiri, abdullah m.; aziz jokhdar, hani a.; alothman, adel; balkhy, hanan h.; aljohani, sameera; al harbi, shmeylan; kojan, suleiman; al jeraisy, majed; deeb, ahmad m.; memish, ziad a.; ghazal, sameeh; al faraj, sarah; al-hameed, fahad; alsaedi, asim; mandourah, yasser; al mekhlafi, ghaleb a.; sherbeeni, nisreen murad; elzein, fatehi elnour; almotairi, abdullah; al bshabshe, ali; kharaba, ayman; jose, jesna; al harthy, abdulrahman; al sulaiman, mohammed; mady, ahmed; fowler, robert a.; hayden, frederick g.; al-dawood, abdulaziz; abdelzaher, mohamed; bajhmom, wail; hussein, mohamed a. title: treatment of middle east respiratory syndrome with a combination of lopinavir/ritonavir and interferon-β b (miracle trial): statistical analysis plan for a recursive two-stage group sequential randomized controlled trial date: - - journal: trials doi: . /s - - -x sha: doc_id: cord_uid: cah lg abstract: the miracle trial (mers-cov infection treated with a combination of lopinavir/ritonavir and interferon-β b) investigates the efficacy of a combination therapy of lopinavir/ritonavir and recombinant interferon-β b provided with standard supportive care, compared to placebo provided with standard supportive care, in hospitalized patients with laboratory-confirmed mers. the miracle trial is designed as a recursive, two-stage, group sequential, multicenter, placebo-controlled, double-blind randomized controlled trial. the aim of this article is to describe the statistical analysis plan for the miracle trial. the primary outcome is -day mortality. the primary analysis will follow the intention-to-treat principle. the miracle trial is the first randomized controlled trial for mers treatment. trial registration: clinicaltrials.gov, nct . registered on july . middle east respiratory syndrome (mers) is a viral respiratory disease caused by the middle east respiratory syndrome coronavirus (mers-cov). mers cases continue to occur and are often associated with respiratory and multiorgan failure [ ] . there is no antiviral treatment with proven efficacy at present [ , ] . the miracle trial (mers-cov infection treated with a combination of lopinavir/ritonavir and interferon-β b) is the first randomized controlled trial for mers treatment. the study protocol has been previously published [ ] . there are several challenges in a trial for treatment of a disease like mers: ( ) there is not enough information on the effect size of the lopinavir/ritonavir and interferon-β b provided with standard supportive care compared to placebo provided with standard supportive care to conduct adequate planning for the study sample size; ( ) mers is a sporadic, unpredictable, and rare disease, which makes it difficult to plan a separate pilot study to collect the necessary information needed for the planning of the main trial. to overcome these challenges, we designed the miracle trial as a recursive two-stage adaptive trial, which is a relatively new method for group sequential trials [ ] [ ] [ ] [ ] . the approach is based on the conditional error principle, which allows for flexible and continuous adjustment of the trial parameters using data observed during prior stages without inflation of the type i error [ ] . another advantage of this method is the flexibility in setting the timing and the number of needed interim analyses. such flexibility is necessary in a situation where recruitment rate is unpredictable and a sudden flux in recruitment of patients could happen at any time. finally, the design takes advantage of the accumulated information throughout the trial from every single recruited patient as opposed to a traditional two-study approach (pilot followed by the main trial). in this article, we describe the miracle trial statistical analysis plan (sap) in advance of trial completion. we identify the procedures to be followed for the primary and secondary analyses for the trial. the sap was written by the study steering committee members led by the principal investigator, who remains blinded to both group allocation and to study results until after completing patient recruitment, patient follow-up, and completion and locking of the database. the final study report will follow the guidelines of the consolidated standards of reporting trials (consort) for reporting randomized controlled trials [ , ] . the trial is being conducted according to the standard requirements of good clinical practice e [ ] . the sap was developed in accordance with the international council for harmonisation guidelines (e statistical principles for clinical trials and e clinical study reports guidelines) [ , ] and with the guidelines for the content of statistical analysis plans in clinical trials [ ] . the miracle trial is a recursive, two-stage, group sequential, multicenter, randomized, placebo-controlled, double-blind trial. the trial includes hospitalized patients who are years old or older with laboratoryconfirmed mers in addition to evidence of acute organ dysfunction that is judged related to mers. inclusion and exclusion criteria have been detailed in a previously published protocol manuscript [ ] . patients are randomized to receive lopinavir/ritonavir and recombinant interferon-β b or placebo. randomization is stratified according to center and according to whether the patients require mechanical ventilation (invasive or noninvasive) at the time of enrollment, as mechanical ventilation is a major, but pragmatic, surrogate for severity of illness. the study interventions continue for days or until hospital discharge. patients are followed up daily until day or hospital discharge and then at day . a consort flow diagram of the trial progress will be constructed (fig. ). the number of randomized patients to each group will be reported as well as the number of randomized patients who received the interventions. we will also report the number of screened patients (defined as all hospitalized patients with mers) who met the eligibility criteria but were not enrolled and the reasons for non-enrollment. the intention-to-treat population consists of all enrolled patients whether or not they received the allocated intervention, and will be used for the primary analysis. a per-protocol analysis will be conducted for patients who received the allocated interventions (defined by any dose of the study intervention). baseline characteristics will be presented for the two study groups (additional file : table s ) including age, sex, and body mass index, the presence of co-infections, nosocomial versus community-acquired mers infection, acute physiology and chronic health evaluation (apa-che) ii scores, sequential organ failure assessment scores, and the karnofsky performance status scale score [ ] . we will report comorbidities and the interventions received before randomization for the patients in each group. we will report baseline laboratory values (international normalization ratio, platelet count, hemoglobin, white blood cell count, lymphocyte count, liver enzymes, glucose, serum amylase, blood urea nitrogen, creatinine, creatine kinase, lactate) and respiratory and vital parameters in addition to the location of the patient at time of randomization. for each group we will report the time of hospital admission to randomization and the time of randomization to the first dose received of the study drugs. we will report the received study intervention and its duration for each group, in addition to the missing or incomplete doses and protocol violations (additional file : table s and table s ). we will compare any use of vasopressors, renal replacement therapy, neuromuscular blockade, mechanical ventilation, extracorporeal membrane oxygenation (ecmo), nitric oxide, prone ventilation, and tracheostomy. we will also compare the use of intravenous immunoglobulin, antiviral therapy, antibiotics, corticosteroids, and statins (additional file : table s ). the primary outcome is -day mortality (additional file : table s ). the primary outcome is defined as all-cause mortality after enrollment in the trial within days, as either an inpatient or outpatient. secondary outcomes and subgroups are defined as presented in table and additional file : table s , and s ). in addition, we will compare the physiological parameters among patients treated in the treatment group and the control group. all analyses will be performed using sas . with specially written code for the analysis of the primary supplemental oxygen-free days number of days within the first days after enrollment when patients do not receive of supplemental oxygen. patients who die within days will be assigned the value " " renal replacement therapy-free days number of days within the first days after enrollment when patients do not receive of renal replacement therapy. patients who die within days will be assigned the value " " vasopressor-free days number of days within the first days after enrollment when patients do not receive of vasopressors. patients who die within days will be assigned the value " " invasive or non-invasive mechanical ventilation-free days number of days within the first days after enrollment when patients do not receive of mechanical ventilation. patients who die within days will be assigned the value " " organ support-free days number of days within the first days after enrollment when patients do not receive of invasive mechanical ventilation, renal replacement therapy and vasopressor. patients who die within days will be assigned the value " " extracorporeal circulation support-free days number of days within the first days in which patients are not receiving extracorporeal circulation support. patients who die within days will be assigned the value " " icu-free days number of days in which patients are not being cared for in the icu during the first days after enrollment. patients who die within days will be assigned the value " " post-randomization hospital length of stay number of days between randomization and discharge from the hospital. because of the competing risk effect of death on length of stay, length of stay will be also reported for survivors alone the number and percentage of reported serious adverse events any time during the study period. these saes include: acute pancreatitis, severe elevation of alanine aminotransferase (alt) to more than five-fold the upper normal limit, anaphylaxis, bleeding diathesis and others the number and percentage of adverse events graded using the common terminology criteria for adverse events, at any time within days after enrollment. the adverse drug reactions include: allergic reactions, gastrointestinal, general nervous system and others. see also table s functional outcomes karnofsky score karnofsky performance status scale for functional impairment, which is a scale from (indicating "normal," no complaints; no evidence of disease) to (indicating death) at day outcome that accounts for the recursive design, as described in chang [ ] . a detailed interim analysis plan is reported in the mir-acle protocol [ ] . the trial is designed as a recursive, two-stage, group sequential randomized trial. the first interim analysis will be conducted when subjects ( per group) have completed days of follow-up. this is about . % of the total sample size needed for the classical design (a classic two-group design requires a total of subjects ( subjects per group) to have an % power at a significance level of % using a one-sided z test for difference in proportion to detect % absolute risk reduction in days mortality among subjects receiving treatment ( %) compared to a control group ( %)). a data and safety monitoring board (dsmb) will be convened to review the unblinded data (efficacy and safety) and advise on continuation or termination of the trial. the determination of the stopping boundaries in the first two-stage design was calculated using the conditional power method based on the summing stagewise p values. at the first interim analysis, the dsmb will determine whether the trial should be terminated for futility or not using the following boundaries and their corresponding decisions ( table ) . we will summarize and report the demographics and baseline clinical characteristics using descriptive statistics. as appropriate, the chi-square test or fisher's exact test will be used to compare the categorical variables, which will be reported as numbers and percentages. student's t test or the mann-whitney u test will be used as appropriate to compare the continuous variables, which will be reported as means and standard deviations or as medians and interquartile ranges. all adverse events will be grouped using common terminology criteria for adverse events (ctcae) version of the national institutes of health (nih) (additional file : table s ). adverse events will be grouped into aggregate groups and reported for the entire study period (additional file : table s ). all results will be summarized in terms of frequency and percentage and will be compared across study arms using fisher's exact test. all results will be declared statistically significant with a p value < . . let k be the number of stages of the current clinical trial needed to complete the trial and i ∈ { , } be the index for the two-stage design in the k th stage. let r ki and r ki be the proportions of days mortality in the standard of care and treatment group respectively. then the z test statistic for the difference in proportion can be calculated as follows: where n ki is the sample size per group for the i th twostage design of the k th stage. in the interim analysis (i.e., at each i = of the k th two-stage), the primary outcome will be evaluated, and the trial sample size will be reestimated for the subsequent stage based on the observed effect size using the following formula assuming a conditional power of % (pc = . ) to decide if the trial should continue: here α k, is the precalculated rejection boundary for efficacy at the second stage of the two-stage design at the k th stage, and p k, is the raw table probability corresponding to the z ki statistic. at the first interim analysis, should the data suggest that another stage of the twostage steps is required, we will recalculate the conditional error and new boundaries will be calculated for k = . let β k + , , α k + , be the rejection boundaries for futility and efficacy for the first (i = ) of the twostage step of the k th + stage. then the conditional error is: efficacy stopping boundary (α ) . stop trial for efficacy at the second stage or recalculate based on conditional power at first interim analysis α is the maximum probability threshold under which the trial will be terminated early for efficacy. β is the maximum probability threshold above which the trial will be terminated for futility. α is the maximum probability threshold (the sum of the stage-wise p-values), above which the study will be declared as met its endpoint where a(p , ) is the type i error, which is set to . . the new α k + , boundary for the k th + stage for pre chosen β k + , , α k + , will be calculated as follows: at the end of the trial, the treatment will be declared efficacious if the calculated stage-wise ordered p value p k, is less than α k, . the adjusted p value will be obtained using backward recursion as follows: where k is the total number of two-stage stages, and t is the sum of stage-wise raw p values. finally, the adjusted overall % one-sided confidence interval will be calculated by: where δ i, and δ k ; are the stage-wise and the last stage of the kth two-stage design confidence interval bound. the last stage confidence bound δ k ; can be found by solving the following equation numerically for δ k ; : where n k , and n k , are the sample sizes for the first and second stage of the last k th two-stage design, and p k , , p k , are the stage-wise adjusted p values. in order to stay consistent with the method that was used in calculating the boundaries for the trial, we will not account for stratification in the primary outcome analysis. in general, this approach is acceptable and it preserves both type i and type ii errors as long as the weighted average of the effect size stays close to the hypothesized effect size [ ] . furthermore, as long as the sample size re-estimation at the interim analysis was based on the weighted average of the effect size, the overall power of the trial will be preserved. with the exception of the analysis of the primary outcome, all other analyses will be tested using regular statistical methods and will be two-sided. a secondary adjusted analysis will be conducted using multiple logistic regression analysis, in which death within days will be modeled as the dependent variable, and a set of baseline variables that are strongly believed to affect the outcome of mers will be included as independent variables. those variables will include at minimum the following: age, community-acquired versus hospital-acquired infection, mechanical ventilation, center, and sequential organ failure assessment score. ninety-day median survival time will be summarized and reported using kaplan-meier curves and will be compared between the study groups using the log-rank test (additional file : figure s ). analysis of secondary outcomes will be compared in the intention-to-treat cohort only. subgroup analyses will be conducted if patient numbers permit (e.g., no fewer than five patients in subgroups of interest) in a priori defined subgroups (additional file : table s ). multivariable logistic regression will be used to report the results of tests of interactions for these subgroups. all missing data will be reviewed and characterized in terms of their pattern (e.g., missing completely at random, missing at random, etc.). for missing completely at random, all analyses will be based on a list-wise deletion approach where only observations with complete values will be considered for analysis. for variables with values missing at random, multiple imputation techniques will be utilized to impute the missing values, as suggested by rubin [ ] . to adjust for multiple testing, we will use the false discovery rate (fdr) as described by benjamini and hochberg [ ] . in this procedure all hypothesis tests will be sorted in ascending order based on their calculated p value. all hypothesis tests below an index k will be rejected, where k is calculated as follows: where i = m, … , , m is the total number of tested hypotheses, and q = . . additional details about the sap are available in additional file . the miracle trial investigates the efficacy of a combination therapy of lopinavir/ritonavir and recombinant interferon-β b provided with standard supportive care, compared to placebo provided with standard supportive care, in hospitalized patients with laboratory-confirmed mers. the first patient was enrolled in november . at present, sites are actively screening for eligible patients. the recruitment rate in the miracle trial has been slow mainly related to the decline in the number of mers cases in saudi arabia. due to the uncertainty of the efficacy level of the treatment and the recruitment rate, the trial is designed to be a recursive, two-stage, group sequential randomized trial [ ] . several methods could be utilized to build an adaptive trial. however, most of these methods would require one to specify a priori the time and type of adjustments that need to take place in the trial. for a disease such as mers there are many factors that could limit the ability to specify a priori those elements; thus, the recursive two-stage design is a natural choice. this type of design provides enough flexibility to introduce different adjustments while learning from the observed data without inflating the type i error. reporting of the sap to the miracle trial in advance of trial completion will enhance evaluation of the clinical data and support confidence in the final results and the conclusion. prior specification of the statistical methods and outcomes analysis will facilitate unbiased analyses of these important clinical data. recruitment started in november and is currently ongoing. supplementary information accompanies this paper at https://doi.org/ . /s - - -x. additional file : table s . baseline characteristics of intention-to-treat (itt) population. table s . summary of interventions and cointerventions. table s . primary outcome: -day mortality. table s . secondary outcomes. table s . subgroup analyses. table s . classification of adverse events in the miracle trial (mers-cov infection treated with a combination of lopinavir/ritonavir and interferon-β b) using the nih common terminology criteria for adverse events (ctcae), version . . table s . summary of adverse events by severity. table s . summary of protocol violations. middle east respiratory syndrome saudi critical care trials group. ribavirin and interferon therapy for critically ill patients with the middle east respiratory syndrome: a multicenter observational study treatment of middle east respiratory syndrome with a combination of lopinavir-ritonavir and interferon-beta b (miracle trial): study protocol for a randomized controlled trial adaptive design theory and implementation using sas and r benefit-risk evaluation of multi-stage adaptive designs draft guidance for industry on enrichment strategies for clinical trials to support approval of human drugs and biological products; availability flexible interim analyses in clinical trials using multistage adaptive test designs modification of the sample-size and the schedule of interim analyses in survival trials based on data inspections explanation and elaboration: updated guidelines for reporting parallel group randomised trials statement: updated guidelines for reporting parallel group randomised trials harmonisation of technical requirements for registration of pharmaceuticals for human use: good clinical practice (gcp) guideline the international council for harmonisation of technical requirements for pharmaceuticals for human use (ich): statistical principles for clinical trials international conference on harmonisation of technical requirements for registration of pharmaceuticals for human use: e -structure and content of clinical study reports guidelines for the content of statistical analysis plans in clinical trials robustness of an odds-ratio test in a stratified group sequential trial with a binary outcome measure multiple imputation after + years controlling the false discovery rate: a practical and powerful approach to multiple testing publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations we would like to acknowledge the following data safety monitoring board (dsmb) chair the miracle trial is funded by king abdullah international medical research center, riyadh, kingdom of saudi arabia. the study sponsor does not have any role in the study design, collection, management, analysis or interpretation of the data, or in writing the report. the datasets generated and/or analyzed during the current study are available from the corresponding author on reasonable request. the miracle study is approved by the scientific committee and the institutional review board at the national guard health affairs, riyadh, saudi arabia (rc / ) and all participating sites and registered at the saudi food and drug authority (sfda), riyadh, saudi arabia. patients who meet the eligibility criteria or substitute decision-makers (for patients lacking decisionmaking capacity) of eligible patients will be approached to obtain informed consent for enrollment. not applicable. the authors declare that they have no competing interests. ya and fgh are unpaid consultants on antivirals active for mers for gilead sciences, sab biotherapeutics, and regeneron. key: cord- -gduhterq authors: spitzer, ernest; ren, ben; brugts, jasper j; daemen, joost; mcfadden, eugene; tijssen, jan gp; van mieghem, nicolas m title: cardiovascular clinical trials in a pandemic: immediate implications of coronavirus disease date: - - journal: card fail rev doi: . /cfr. . sha: doc_id: cord_uid: gduhterq the coronavirus disease (covid- ) pandemic started in wuhan, hubei province, china, in december , and by april , it had affected > . million people in countries and caused > , deaths. despite diverse societal measures to reduce transmission of the severe acute respiratory syndrome coronavirus , such as implementing social distancing, quarantine, curfews and total lockdowns, its control remains challenging. healthcare practitioners are at the frontline of defence against the virus, with increasing institutional and governmental supports. nevertheless, new or ongoing clinical trials, not related to the disease itself, remain important for the development of new therapies, and require interactions among patients, clinicians and research personnel, which is challenging, given isolation measures. in this article, the authors summarise the acute effects and consequences of the covid- pandemic on current cardiovascular trials. trials may not be able to attend hospitals for follow-up visits or to collect study medications. a careful and periodic risk assessment by sponsors and investigators is required to preserve the safety of trial participants (and employees) and the integrity of trials. in this article, we summarise the immediate implications of the covid- pandemic on ongoing cardiovascular trials. this review incorporates recent recommendations from the us food and drug administration (fda), the european medicines agency, the uk's medicines and healthcare products regulatory agency and australia's therapeutic goods administration, as well as personal views. [ ] [ ] [ ] [ ] [ ] [ ] planning, executing and reporting clinical trials designed for the approval of (or to extend indications for) drugs, biological products, devices and combinations thereof, are highly regulated activities. clinical trialists must observe national regulations, as well as international standards, such as those proposed by the international conference of harmonization, the international organization for standardization and the international medical device regulators forum. two general principles governing the execution of clinical trials are ensuring patient safety and clinical trial integrity. according to the who, patient safety is "the absence of preventable harm to a patient during the process of health care and reduction of risk of unnecessary harm associated with health care to an acceptable minimum." as defined by the fda, data integrity refers to "the completeness, consistency, and accuracy of data. complete, consistent and accurate data should be attributable, legible, contemporaneously recorded, original or a true copy and accurate (alcoa)." data are to be recorded exactly as intended, and when retrieved at a later time, should be the same as originally recorded. while patient safety is paramount, both should be prioritised for the successful execution of clinical trials. if data integrity is compromised, study results may no longer be interpretable, reliable or usable. a pandemic has the potential to directly impact all individuals and organisations involved in clinical research (figure ) . highly contagious and rapidly spreading viruses, such as sars-cov- , require comprehensive measures to avoid human-to-human spread. with the ongoing pandemic, the world has progressively witnessed a reduction of airline activity to almost zero with widespread travel bans, and limitation of private and public transportation, temporary closure of retail businesses, banning of public gatherings and the requirement to work from home. all these measures are designed to limit exposure to potential carriers of the virus. individual measures, such as meticulous hand hygiene, self-isolation and social distancing, are encouraged. public measures, such as quarantines, curfews or lockdowns, have been implemented. however, sectors, such as healthcare, food supply chains, law enforcement, governmental agencies and regulatory bodies, remain indispensable, with an increased workload challenging the capacity of local and national systems, as well as risking (if not sufficiently protected) the well-being of individuals. overall, the majority of people stay at home, work remotely and limit use of healthcare systems as much as possible. the clinical trial life cycle can be divided into trial design and registration, trial start-up, enrolment, follow-up, reporting and regulatory submission. changes in trial conduct should be documented and, if substantial, incorporated as protocol amendments (although not in an expedited manner unless impacting on patient safety); lesser changes may be captured as protocol deviations related to the pandemic. regulatory agencies offer a diverse range of flexibility in such procedures, and applicable guidance documents should be consulted to establish the most appropriate approach for a particular trial. [ ] [ ] [ ] , trial enrolment should be put on hold or stopped if there is significantly reduced feasibility (e.g. drug trials with infusions), when participants require intensive care post-treatment (e.g. surgical trials) or when the investigator is unavailable. when inclusion is delayed by the pandemic, it should be dealt with in a similar manner to other circumstances that lead to a low recruitment rate. if appropriate, and especially if foreseen by the protocol, a data and safety monitoring board may assess futility due to severe impact on data collection or outcomes. however, if stopping or putting on hold a trial puts participants at increased risk, efforts should be taken to continue with trial-related activities. enrolment in cardiovascular trials generally takes place at outpatient visits or during hospitalisation. trials in patient populations with acute presentations (e.g. st-elevation mi [stemi]) may identify potentially suitable trial candidates; however, the capacity to comply with study procedures needs to be assessed, as well as considerations related to patient safety during follow-up. it is also pertinent to consider that covid- may mimic some classical presentations, such as stemi; ecg changes are shown to reflect myocarditis, after angiography demonstrates non-obstructive disease. it is problematic when the trial design mandates a protocol-related treatment before angiography. furthermore, the analysis of outcomes may be rendered more difficult, and parallel analyses of the intention-to-treat and per protocol populations will be pertinent. participants in the follow-up phase (when they are generally at home) constitute a higher-risk population in the reduced capacity at investigational sites will impact on availability to perform study visits (or phone calls) to assess and confirm eligibility, enter data in electronic case report forms (ecrfs), to report (serious) adverse events and to follow the protocol in general. all protocol deviations should be noted, with those that are pandemic related clearly participants should be given the option to continue, suspend or withdraw participation. reporting of sever adverse events is expected to continue according to standard procedures and regulation. dsmbs may be appointed to determine feasibility of continuing trials based on overall conduct, patient safety and date integrity. all meetings should be remote, by means of teleconference/ video conferencing. steering committee calls, investigator meetings, endpoint adjudication committees and dsmbs should meet remotely with appropriate technology in place. consideration should be given to changes that limit the exposure of participants, investigators and staff to sars-cov ; changes in enrolment and testing. reporting should differentiate between pre-pandemic, peri-pandemic and post-pandemic, as well as covid- positivity/negativity. novel trial approaches that reduce physical contact remote site management and monitoring could be considered, if feasible (i.e. privacy issues and site workload are considered). consider virtual visits, telemedicine, electronic consent or teletrials. change site location outside the hospital. deliver medication to homes. clinical research organisations need to swiftly transition into home-based organisations and increase level of oversight to deliver urgent and ongoing responsibilities. remote systems need to be upgraded to allow adequate online execution and oversight. ongoing core laboratory activities during a pandemic core laboratories continue operations utilising virtual environments to analyse and review materials. remote analysts and supervisors utilise secure platforms with access to required validated analysis software and study datasets. continued ict support is pivotal. perform risk-bene t assessment. decide to continue with or without changes, to put on hold or stop trials. evaluate feasibility of protocol adherence or need for modi cation. evaluate capacity to continue trials based on human resources, logistics and drug distribution. limit enrolment. prioritise pandemic-related research, as evidence-based treatments are lacking. incorporate measures that limit physical contact between researchers and participants. consider novel approaches that allow remote data capture and remote monitoring. identified. most importantly, principal investigators must ensure that enrolled subjects fully comply with eligibility criteria and that all measures are taken to report adverse events in a timely fashion, given that these two are of paramount importance for patient safety. coordinating centres may require an increased level of monitoring of ecrfs and a degree of flexibility in terms of timing for data cleaning. , , , large, multicentre collaborative trials require the participation of a coordinating centre, either a contract or an academic research organisation. coordinating centres execute the study, or activities within, on behalf of the sponsor or manufacturer. a study team is composed of a project manager, clinical research associates and study monitors, data managers, biostatistician, quality assurance manager and safety reporting units, with or without a medical monitor. a pandemic prompts the need to work from home and cancel face-toface visits. where systems are upgraded to allow remote work and staff remain available for reception of materials, coordinating centres can continue to operate during a pandemic. site initiation and monitoring visits are cancelled, postponed or performed remotely using webbased technology (although source data verification can be postponed). remote monitoring is possible, but might not be feasible at all participating sites in a trial and increases the workload at the site. moreover, technical requirements, confidentiality issues, updated consents and the increased burden to site personnel could make it impractical. in line with this, quality assurance measures, such as site audits, are postponed unless serious non-compliance is identified. the participation of several committees in clinical trials ensures proper scientific and operational oversight, data integrity and quality, as well as patient safety. typically, the steering committee is composed of established investigators or key opinion leaders, and representatives from parties involved (e.g. coordinating centres, sponsor, grant givers). during the pandemic, office-based professionals work from home, and participation may be limited. nevertheless, given the oversight duties of the steering committee and data and safety monitoring boards, the frequency of meetings might need to be increased to address immediate pandemic-related needs. at the beginning of the pandemic, cardiovascular clinicians saw a reduction in patient load, as the population was advised to stay at home. unfortunately, this has resulted in late presentations of severe conditions (e.g. non-stemi or decompensated heart failure). however, as hospital resources are depleted, not only in materials but also in personnel, cardiovascular clinicians are required to perform pandemic-related tasks and to self-isolate, potentially limiting their availability for participation in committees. the same applies to members of clinical event committees and data and safety monitoring boards. potential exceptions are data managers and biostatisticians. in theory, this could reduce the availability of clinicians to participate in committee calls; however, in practice, this might not be the case. committed investigators tend to stretch time when required, as shown by chinese investigators who managed to report initial cohorts despite being at the centre of the pandemic. , all meetings are planned as teleconferences. cardiovascular trials, particularly interventional trials, rely heavily on imaging. for the purpose of an unbiased and consistent analysis, central laboratories are utilised. imaging modalities, such as echocardiography, ecg, cardiac mri, angiography assessments, intracoronary imaging and cardiac ct, are frequently used. thus, for a core laboratory to ensure timely delivery during a pandemic, conditions should allow analysts and supervisors to work remotely. data should reach core laboratories electronically, with secure and certified datatransfer providers. time windows for imaging follow-up might need to be adjusted and uploading activities may also be interrupted. analysing cardiovascular images might not be as efficient at home when compared with a well-equipped work environment. however, remote access through a secure connection to software and datasets, as well as databases, will allow continuity of activities. information and communication technology departments play a pivotal role in setting up and maintaining reliable infrastructure. a lack of remote access could force activities to stop during a pandemic. safety reporting should continue in line with national regulations and following standard procedures. [ ] [ ] [ ] , investigators should ensure timely capture of serious adverse events, a process that might involve extended use of telehealth. moreover, serious adverse events should be identified, where possible, as pandemic or non-pandemic related. the inability to deliver investigational drugs could pose additional risks to participants and warrants an increased level of safety monitoring. , ongoing trials lacking data and safety monitoring boards might need to revisit that decision on a per-case basis. data and safety monitoring boards may independently assess an ongoing trial that has been severely affected by the pandemic (e.g. incomplete data, incomplete follow-up) to help investigators and sponsors elucidate, without compromising the integrity of the trial, whether continuing the trial will yield interpretable data. a pandemic has a significant impact on the ability to adhere to protocol requirements (e.g. missed follow-up visits or tests). importantly, protocol deviations should be documented with an indication that they are pandemic-related following standard procedures. , - data collection could be challenging, but should not stop. when reporting the results of a trial, cohorts might need to be divided as pre-pandemic, peri-pandemic and post-pandemic. statistical analysis plans might need adaptions when considering the influence of the pandemic in the interpretability of results, especially when endpoints share characteristics with covid- related events. guidance on the interpretability of results when analysing data with missing values, unbalanced completeness or out-of-window assessments (e.g. echocardiograms, control angiograms, laboratory values) might also be required, depending on the duration of the pandemic. for multicentre trials, a per-site assessment might be required for outbreak areas versus non-outbreak areas. the interpretability of the overall evidence generated should be discussed with regulatory authorities. , - the use of vaccines, once available, might also require adequate documentation in study databases to avoid unbalanced usage. ethics committees (ecs; or institutional review boards [irbs]) and regulatory agencies experience a significant increase in activity during a pandemic. ecs/irbs face the burden of protocol amendments for ongoing trials, and prioritise activities related to the pandemic, including the review of covid- trial submissions. , - regulatory agencies play a critical role in protecting citizens from threats, including emerging infectious diseases, thus the importance of providing timely guidance, such as the regulatory documents that form the basis of this article. [ ] [ ] [ ] [ ] [ ] [ ] based on accumulating experience, the advice of ecs/irbs and regulatory agencies to sponsors and investigators could be critical to determine the continuation, modification or pause of trial activities. such recommendations are complex, given the uncertainties related to the pandemic duration. a pandemic has a significant impact on every component of cardiovascular clinical research. when facing a rapidly spreading disease with no effective treatment or vaccine, efforts should be focused on facilitating the day-to-day work of healthcare professionals with required personal protective equipment. pandemic-related investigations should be prioritised. nevertheless, sponsors and investigators should take all necessary actions to ensure patient (and employee) safety and to maintain trial integrity in ongoing, nonpandemic-related clinical trials, and capture pandemic-induced trial adjustments in focused amendments so that meaningful conclusions can be achieved when reporting results. a novel coronavirus from patients with pneumonia in china the novel chinese coronavirus ( -ncov) infections: challenges for fighting the storm clinical characteristics of coronavirus disease in china the reproductive number of covid- is higher compared to sars coronavirus severe acute respiratory syndrome -retrospect and lessons of outbreak in china emerging threats from zoonotic coronaviruses -from sars and mers to -ncov high contagiousness and rapid spread of severe acute respiratory syndrome coronavirus . emerg infect dis fda guidance on conduct of clinical trials of medical products during covid- public health emergency postmarketing adverse event reporting for medical products and dietary supplements during a pandemic guidance on the management of clinical trials during the covid- (coronavirus) pandemic. brussels: european medicines agency points to consider on implications of coronavirus disease (covid- ) on methodological aspects of ongoing clinical trials medicines and healthcare products regulatory agency. managing clinical trials during coronavirus (covid- ) covid- : guidance on clinical trials for institutions, hrecs, researchers and sponsors. canberra: doh data-integrity-and-compliance-with-current-good-manufacturing-practice-guidance-for-industry key: cord- - r rtixg authors: kharma, nadir; roehrig, stefan; shible, ahmed atef; elshafei, moustafa sayed; osman, dema; elsaid, ingi mohamed; mustafa, salma faisal; aldabi, asjad; smain, osamah a.m.; lance, marcus d. title: anticoagulation in critically ill patients on mechanical ventilation suffering from covid- disease, the anti-co trial: a structured summary of a study protocol for a randomised controlled trial date: - - journal: trials doi: . /s - - - sha: doc_id: cord_uid: r rtixg objectives: to assess the effect of anticoagulation with bivalirudin administered intravenously on gas-exchange in patients with covid- and respiratory failure using invasive mechanical ventilation. trial design: this is a single centre parallel group, superiority, randomized ( : allocation ratio) controlled trial. participants: all patients admitted to the hamad medical corporation -icu in qatar for covid- associated respiratory distress and in need of mechanical ventilation are screened for eligibility. inclusion criteria: all adult patients admitted to the icu who test positive for covid- by pcr-test and in need for mechanical ventilation are eligible for inclusion. upon crossing the limit of d-dimers ( . mg/l) these patients are routinely treated with an increased dose of anticoagulant according to our local protocol. this will be the start of randomization. exclusion criteria: pregnancy, allergic to the drug, inherited coagulation abnormalities, no informed consent. intervention and comparator: the intervention group will receive the anticoagulant bivalirudin intravenously with a target aptt of - sec for three days while the control group will stay on the standard treatment with low-molecular-weight heparins /unfractionated heparin subcutaneously (see scheme in additional file ). all other treatment will be unchanged and left to the attending physicians. main outcomes: as a surrogate parameter for clinical improvement and primary outcome we will use the pao /fio (p/f) ratio. randomisation: after inclusion, the patients will be randomized using a closed envelope method into the conventional treatment group, which uses the standard strategy and the experimental group which receives anticoagulation treatment with bivalirudin using an allocation ratio of : . blinding (masking): due to logistical and safety reasons (assessment of aptt to titrate the study drug) only the data-analyst will be blinded to the groups. numbers to be randomised (sample size): we performed a sample size calculation and assumed the data for p/f ratio (according to literature) is normally distributed and used the mean which would be: and sd is . we expect the treatment will improve this by %. in order to reach a power of % we would need patients per group (in total patients). taking approximately % of dropout into account we will include patients ( in each group). trial status: the local registration number is mrc- - with the protocol version number . the date of approval is th june . recruitment started on (th) june and is expected to end in november . trial registration: the protocol is registered before starting subject recruitment under the title: “anticoagulation in patients suffering from covid- disease. the anti-co trial” in clinicaltrials.org with the registration number: nct . registered on june . full protocol: the full protocol is attached as an additional file, accessible from the trials website (additional file ). in the interest in expediting dissemination of this material, the familiar formatting has been eliminated; this letter serves as a summary of the key elements of the full protocol. list abbreviations, acronyms and terms of reference used in the protocol; provide definitions for each as needed the pandemic of covid- , a newly upcoming viral disease caused by sars-cov- , puts the whole worlds health system under pressure. patients suffering from this disease mainly develop respiratory symptoms, which can lead to severe acute respiratory distress syndrome (ards) necessitating icu admission in - % of the cases admitted to hospital. in addition to these symptoms, patients show lymphopenia, cardiac symptoms and altered coagulation profiles. although those patients are treated in the icu the mortality there is more than % due to multiorgan failure [ , ]. one of the recent insights in this disease is its effect on the coagulation system. meanwhile we know that the coagulation system gets activated. furthermore, it seems that clot formation takes place in the pulmonary micro-vasculature which could contribute to the widely observed gasexchange impairment. therefore, many centers apply empiric anticoagulation for their covidpatients. at the moment, we at hmc, also apply an empirical anticoagulation protocol as our standard approach (see attachment ). however, this is a symptomatic treatment without good proof. bivalirudin is a well-known agent which is used in hmc for cases in which anticoagulation is needed, but a contraindication for heparin exists (i.e. hit syndrome). this drug has an interesting pharmacologic profile. it acts independent of antithrombin (at), the physiological compound which enhances heparin effects under normal circumstances. this lack of dependence on at, makes bivalirudin an attractive choice in light of the contradictory reports on the levels of at during covid infection. if at levels are decreased during the infection, heparin (as well as lmwh) cannot work efficiently, which would render our treatment less effective. luckily, bivalirudin acts without the support of at, so we could bypass this problem. in addition, bivalirudin has some fibrinolytic activities. it inhibits clot-bound thrombin which as a result destabilizes the clot rendering it more susceptible to thrombolysis. this property partially supports the dissolving of clots and could support re-opening the pulmonary microcirculation. [ , ] objectives: to prove the positive effect of anticoagulation with bivalirudin intravenously on gas-exchange in patients with covid- and respiratory failure on invasive mechanical ventilation. as a surrogate parameter for clinical improvement we will use the pao /fio ratio. clinically parameters (e.g. time to extubation, ventilator settings) all patients coming to admission to the icu for covid associated respiratory distress and in need for mechanical ventilation according to clinical decision are screened for eligibility. inclusion criteria: all adult patients admitted to the icu who are covid positive tested and in need for mechanical ventilation are eligible for inclusion. upon crossing the limit of d-dimers ( . mg/l) these patients would be standardly treated with an increased does of anticoagulant. this would be the start of randomization. exclusion criteria: pregnancy, allergic to the drug, inherited coagulation abnormalities, no informed consent. after inclusion the patients will be randomized using a closed envelope method into the conventional treatment group (see attached protocol), which uses the hmgh-approved strategy and the experimental group which receives anticoagulation treatment with bivalirudin (see attached hit-protocol). as a safety measure we will monitor and titrate all drugs according to the aptt and/or antixa (according to the protocols-see attachment). we will collect demographic data (age, sex, bmi), co-morbidities and clinical data (need for special intervention e.g. proning). next to the vital parameters (blood pressure, heart frequency and body temperature) the oxygenation parameters and etco will be collected. the standard coagulation parameters (aptt, inr, platelet count, antixa) will be collected. in addition, we monitor fibrinogen levels and d-dimers (both are associated with mortality) and at. kidney-function parameters (creatinine, urea), liver function tests and whole blood counts and immune-parameters like interleukin (il- ) will be recorded. cardiac enzymes will be collected (troponin, bnp) and echocardiography will be evaluated and reported. all of these parameters are reported routinely on daily base for clinical purpose. power-calculation: grasselli et al. found that the median pf ratio was with (iqr - ) in their cohort of patients. [ ] to perform sample size calculation, we assume data is normally distributed and set the median equal to mean (mean=median and sd = iqr/ . ). so, the mean would be: and sd is . we expect the treatment will improve this by %. in order to reach a power of % we would need patients per group (in total patients). taking approximately % of exclusion into account we will include patients ( in each group). data analysis: categorical data will be presented as number and percentage, while interval data will be presented by median and interquartile range (iqr). normally distributed data will be analyzed by using two-tailed unpaired students t-test. continuous variables with skewed distribution will be analyzed using mann-whitney u testing and dichotomous variables by means of fisher's exact test. a p-value < . is considered significant. all data analyses will be done using spss version v (ibm corp, armonk ny, usa). graphs will be constructed using graphpad prism (graphpad prism version . a for windows, graphpad software, san diego ca, usa). we will maintain privacy for the subject throughout data collection by carrying out data collection in private rooms (icu-rooms). data confidentiality will be maintained by the use of study ids rather than any identifying data. all data will be entered into a secure database, which is password protected with restricted access, only assigned by the research team. each subject will be assigned an alphanumeric study id, to ensure data confidentiality. the link between the identifier and the study code will be deleted at the end of the study and the anonymized data set will be kept for at least years after study completion per mrc policy. subjects can withdraw from the study at any point and this will not be held against them. they will be informed during the consent process of this and will be asked to contact the research team so that the investigator can withdraw them from the study. if a subject should withdraw, the data and samples collected will be destroyed, unless they have already been analyzed or processed or coded. this study is an interventional trial using anticoagulants. the anticoagulant used (bivalirudin) principally is registered and fda approved for the use as a direct thrombin inhibitor in adults undergoing percutaneous coronary interventions and in patients with heparin induced thrombocytopenia. as with all medication this medication might also show side effects. next to allergic reaction the most prominent could be bleeding. as a bleeding might be a severe side effect, data after the first patients will be reviewed by the dsmb. if the major bleeding rate exceeds the numbers reported in literature ( . %) we will be forced to stop the study prematurely due to safety concerns. although the research drug (bivalirudin) is an old drug which is approved and used widely we will follow and report all adverse effects to ensure safety for the patients. case-fatality rate and characteristics of patients dying in relation to covid- in italy baseline characteristics and outcomes of patients infected with sars-cov- admitted to icus of the lombardy region argatroban and bivalirudin for perioperative anticoagulation in cardiac surgery the direct thrombin inhibitors (argatroban, bivalirudin and lepirudin) and the indirect xa-inhibitor (danaparoid) increase fibrin network porosity and thus facilitate fibrinolysis the members of the research team act according to the gcp guidelines. all research is done under recognition of the helsinki declaration and under full adherence to the moph regulations in qatar. key: cord- -n acd hj authors: van der plas, j.l.; roestenberg, m.; cohen, a.f.; kamerling, i.m.c. title: how to expedite early phase sars‐cov‐ vaccine trials in pandemic setting – a practical perspective date: - - journal: br j clin pharmacol doi: . /bcp. sha: doc_id: cord_uid: n acd hj nan several bottlenecks can be identified in the vaccine development process [ ] . after preclinical studies and manufacturing processes, a rate-limiting step in vaccine development is the conduct of clinical trials. in addition, the majority of vaccine candidates fail in clinical development and never reach market authorization [ ] . only % of the vaccine candidates will successfully transition from phase i to phase ii and only % will successfully transition from phase ii to phase iii [ ] . these early phase clinical trials (classical phases i and ii) are essential to assess safety, tolerability and immunogenicity of the vaccine candidate, as well as preliminary information on its efficacy, before progressing further to large scale phase iii trials. in the current covid- pandemic it is more important than ever to identify and select the most promising vaccine candidates as early as possible and stop clinical development for failing candidates to avoid wasting valuable time and resources. here, we discuss several practical suggestions that could accelerate early phase vaccine trials in the covid- pandemic. approval of the competent authority and mec [ ] . legislation pertaining to the use and deliberate release of gmo's is not fully harmonized between countries. therefore, regulation by national authorities can differ substantially between countries. in addition, regulatory bodies often work independently from each other. obtaining the environmental permit for a clinical trial with a gmo-based vaccine can take up to several months, depending on the country-specific regulatory framework for the use of gmos [ ] . these timelines impose a significant hurdle for rapid clinical development of a vaccine against sars-cov- . however, temporary legal exceptions can be made to the application process when there is a clear and urgent need for human health develop a gmo for the purpose of a vaccine. in the netherlands, an accelerated licensing procedure has been implemented for gmo-based vaccine candidates. following discussions between researchers and regulators, an emergency regulation has been implemented in the netherlands by use of a ministerial decree. in short, the emergency regulation means that permits can be issued immediately, even before the clinical trial approval. this emergency regulation allows to process permit application through the regular licensing procedure, but drastically shortens the decision period of the application from days to maximum of days [ ] . the gmo application process can be further expedited if the applicant consults the competent authorities and advisory bodies in an early stage and the review board has previous experience with the vaccine platform. in the netherlands, the authorities can be consulted to provide a pre-advice (before the formal clinical trial application) to help streamline the application process. pro-active and transparent communication between applicant, sponsors and regulators are essential to complete the mandatory gmo applications in an expedited manner. iii. allow investigators to start preparing for trials before the formal clinical trial approval another possibility to expedite the start-up phase is to allow investigators to start recruitment and screening of potential participants before formal approval of the clinical trial. recruitment and screening of participants are time consuming activities in early phase clinical trials. significant time can be saved if potential participants can be identified, counseled and general health status assessed before the formal clinical trial approval. this can be achieved by a conditional approval of the clinical trial submission or by submitting a separate research protocol that solely aims to identify eligible participants for covid- vaccine trials. this will allow the investigator to maintain an ongoing pool of (pre)screened healthy participants that are ready to be enrolled in vaccine trials. it is imperative that participants are again counseled and consented for the final, approved, study protocol prior to enrollment in the clinical trial. however, the majority of recruitment and screening activities will than already be completed and will allow for a rapid start of the clinical trial. iv. centralization of facilities to perform immunogenicity assays for codid- vaccines another rate-limiting step in clinical trial start-up is the validation of immunogenicity assays. the relevant immune assays in vaccine trials will typically depend on the mechanism of action of the vaccine candidate and possible known correlates of protection. unfortunately, for sars-cov- such correlate of protection have not yet been identified. however, most current covid- vaccine trials use some form of virus neutralizing assay to assess immunogenicity. pathways, by allowing investigators and authorities to work in parallel rather than a sequential order and by identifying, validating and centralizing immunogenicity assays as soon as possible. early discussions with authorities and regulators about study design may also facilitate guided and rapid drug development. vaccine development in a pandemic setting requires flexibility of both investigators, developers and authorities. in these trying times we need to find practical solutions and make joint efforts to expedite vaccine development for covid- . risk in vaccine research and development quantified process mapping of vaccines: understanding the limitations in current response to emerging epidemic threats estimation of clinical trial success rates and related parameters accelerated procedure (fast track) review of coronavirus research files world health organization. draft landscape of covid candidate vaccines on the deliberate release into the environment of genetically modified organisms and repealing council directive / is it time to reform oversight of clinical gene therapy in the eu? research related to treatment and vaccine covid- accelerated through emergency regulation covid- : how ema fast tracks development support and approval of medicines and vaccines developing covid- vaccines at pandemic speed the authors would like to thank the central committee on research involving human subjects (ccmo), the netherlands, and the gene therapy office, the netherlands, for their openness and willingness to discuss clinical trial application procedures for covid- . key: cord- -oudvxb d authors: beane, joal d.; dedhia, priya h.; ejaz, aslam; contreras, carlo m.; cloyd, jordan m.; tsung, allan; pawlik, timothy m. title: conducting clinical trials in the time of a pandemic date: - - journal: ann surg doi: . /sla. sha: doc_id: cord_uid: oudvxb d nan mini abstract: the novel human coronavirus (sars-cov- ) has created formidable challenges to the conduct of clinical trials and research.herein we describethe unparalleled response to these challengesand the effort to advance clinical trials in the face of the pandemic. the covid- pandemic has created formidable challenges to the conduct of clinicaltrials and research.overcoming these challenges has requiredcreativity and resourcefulness to mount broad-sweeping changes. responses to mitigate the effects of the pandemichave included: ) thedevelopment of strategies to support research programs during unforeseen economic loss, )establishment of institutionalguidelines for clinical trials, )measures to ensure a healthy clinical research team, ) useof innovative technologies to maintain access to clinical trials, ) amendment of protocols to avoid costly trial closures, and ) the strategic reopening of suspended clinical trials. herein we discuss how each of these strategies has been used to sustainclinical trials and surgical research in the face of extraordinaryadversity. the covid- pandemic has caused significant financial ramifications to all stakeholders in clinical trials -pharmaceutical and device companies, patients, and healthcare systems. as such,institutional leadership may need to reprioritize clinical research efforts and shift resources away from high-risk pilot studies and phase i trials to support more advanced phase iii studies. in addition, increasing efforts in grant submissions, other public and private funding, and charitable donations are needed to boost funds to support clinical research. while the economic impact will be felt for years and is well beyond the control of the academician and clinical trialist, using the principles outlined below to reduce costly errors and to preserve ongoing trial efforts may be our best chance to mitigate the effects of the covid- pandemic. institutional guidelines should be established relative to initiating new clinical trials, managing accrual efforts to current studies, and discerning between essential and non-essential studies. patient safety must remain top priority and a reappraisal of ongoing clinical trialsis vital to ensure appropriate balance of risks and benefits to the patient and the health system.navigating these types of decisions can be difficult and errors in judgement can be copyright © wolters kluwer health, inc. unauthorized reproduction of this article is prohibited. costly.decisions should include all stakeholders and be informed by guidelinespublished by thefood and drug administration (fda). in addition to these guidelines and frequent updates from the irb, our institution has established a monitoring team that is readily available to review evolving research decisions. specifically, an irb "hotline" has been established to ensure accessibility to this team. experienced researchers and clinical trialists are available to help navigate unexpected challenges and ensuring patient safety and trial integrity. maintaining the wellbeing of the clinical research workforce is paramount to preserving clinical trials during a pandemic. however, the availability of staff can decrease by as much as - % due to illness, death, absenteeism related to fear and anxiety, as well as competing needs such as caring for children or ill family members. one of the most important strategies to help preserve the workforce is to ensure adherence to national and international guidelines regarding the use of ppe. in addition to physical safety, measures to ensure mental fitness are just as important to ensuring the research team's health and wellbeing.while many centers have already experienced their surge of covid- infections and the clinical burden is waning, the psychological impact of the pandemic may linger. , in addition, as clinical volume and research begin to ramp back up, a strained workforce may result in extended hours for clinical staff. as such,it is imperative that employees are provided with resources to safeguard mental health( figure) . these strategies may be helpful for nonclinical research staff, as well as individuals who may be anxious about the timing of laboratory reopenings and/or the ability to obtain research funding or grant extensions. at our institution, telemedicine is being used to interface with patients to discuss diagnoses, review imaging, and provide multidisciplinary consultation. in addition, telemedicine is being employed to contact trial participants, enroll and screen patients, as well as assess the status of patients already accrued in clinical trials. in march, over the course of one week, the volume of telemedicine visits jumped from less than encounters per day to well over , per day. in addition to maintaining access to clinical trials, the improved communication afforded by telemedicine has aided in reassuring participants that their involvement remains copyright © wolters kluwer health, inc. unauthorized reproduction of this article is prohibited. important during this time,which has helped to prevent costly dropouts or nonadherence. recently, telemedicine has also been approved as a means to have discussions regarding informed consent as long as a document that can be mailed to and signed by the patients is completed.these benefits have resulted in an exponential increase in telemedicine utilization that have been invaluable to preserving access to clinical trials and research and to capturing important data points with regards to efficacy and adverse events that otherwise would have been lost. while technology and telemedicine platforms may improve access to care and clinical trials, the availability of specific devices or implants may threaten continuity of certain trials. for essential clinical trials, investigators should work closely with the trial sponsor and device manufacturer. in addition, for multicenter studies excess inventory at one center can be shared with other participating centers to ensure that equitable trial accessibility is maintained. efforts to modify protocols in order to comply with the emergency public health response and the guidelines established by the fda and irbareencouraged.investigators need to prioritize collection of data, focusing on the primary endpoint and important secondary endpoints to remain in compliance.clinical protocols should be reexamined thoroughly and amendments should be made to reduce superfluous clinical visits.while many protocol changes typically required an amendment and lengthy review process, to avoid costly delays, reviews of amendments at our institution are being waived or expedited if the adjustment pertains to patient safety in the setting of covid- . for other modifications, a more formalized process is generally necessary, but should be expedited when possible. amendments should be used primarily to protect patientsand research staff, but can also be used strategicallytoadapt protocols andavoid trial suspension. in this way, investigators may be able tomaintain accrual of new patients or continue to collect important data from currently enrolled patients. for example, an amendment to monitor remotely an implanted device may facilitate continued treatment and collection of important data when nonessential interactions must be suspended. alternatively, for patients receiving infusion therapy, an amendment can be written that allows the medication to be administered by a home health nurse to avoid crowded outpatient infusion centers.protocols should be updated with ongoing data collection. for copyright © wolters kluwer health, inc. unauthorized reproduction of this article is prohibited. example, collecting patient-reported endpoints for device or implant studies may include mailed or telephone surveys to replace scheduled in-person visits. in summary, adapting protocols todeliver an intervention or collect valuable outcomes can help to avoid costly suspensions,and updating trial procedures to ensure patient safety are important in order to mitigate the challenges posed by the covid- pandemic. given the current environment, irbs should be prepared for a significant increase in protocol deviations. at our institution we are actively increasing the capacity of the irb to review and address deviations expeditiously since a timely review is important to preserving trial integrity.to this end, deviationsto clinical protocols should be tracked carefully, as well as documented, reported, and dealt with according to institutional guidelines.guidelines distributed by the department of health and human resourceshelp to define protocol deviations.for any type of deviation, thorough documentation in the medical record is vital to maintaining trial integrity. it is important to note that major protocol violations may not result in a corrective action plan or even a protocol amendment if related to challenges posed by the covid- pandemic. a thorough reappraisal of all trials should be conductedregularly to reassess the development ofnew risks to participants, the availability of clinical staff andof the investigational drug or device,and disruptions related to quarantines, site closures, or travel limitations (table) . a reduction in clinical research staff may force clinical trials and other studies to be suspended. in addition, multiple, major deviations in study protocolor errors in data collection related to the pandemicmay lead to afailed clinical trial, or worse, erroneous conclusions. as such, one may need to consider suspending aclinical trial until conditions improve. thoughtful, timely appraisals and a multidisciplinary approach to these decisions can helpavoid costly failures and ensure the safety of patients and providers, as well as the integrity of the trial, remain intact. now that many centers are beyond the anticipated surge, guidelines should be established for reopening clinical trials that were suspended during the peak response to covid- . disease copyright © wolters kluwer health, inc. unauthorized reproduction of this article is prohibited. site leaders should work with treating physicians, research teams, clinical trial offices, as well as irb leadership to create a succinct list of feasible studies that could be reactivated. the should be limited to essential trials that provide needed treatment for patients with limited alternatives. trials of lower complexity and research-only procedures should be prioritized. careful consideration should be given to whether more complex studies that involve extensive visits to the medical center or those trials that pose a challenge to social distancing should be included. in addition, only studiesin whichthe protocol calendar can be followed without excessive deviationshould be considered. when prioritizing trials, investigator-initiated studies may be more desirable since these studies can be amended more readily to permit changes in procedures and thus avoid protocol deviations. the ramp-up period must provide ample time for coordinators and the research team to begin returning to campus as needed and for acquiring the necessary approval from sponsors to begin enrolling again.to ensure safety and avoid costly errors,the reopening of clinical trials should occur in phases, based on priority, and within the guidelines established for social distancing. in summary, the covid- pandemic has created many unforeseen challenges that threaten important and costly clinical trials and research efforts. while the impact of this pandemicis broad, the research team mustremaindedicatedtocreatinginnovative approaches to advanceclinical research for our patients. copyright © wolters kluwer health, inc. unauthorized reproduction of this article is prohibited. copyright © wolters kluwer health, inc. unauthorized reproduction of this article is prohibited. fda guidance on conduct of clinical trials of medical products during covid- pandemic. . fda.gov/media/ /download accessed disease control priorities: improving health and reducing poverty strategy and technology to prevent hospitalacquired infections: lessons from sars, ebola, and mers in asia and west africa mental health in the covid- pandemic preserving clinical trial integrity during the coronavirus pandemic estimated costs of pivotal trials for novel therapeutic agents approved by the us food and drug administration wolters kluwer health, inc. unauthorized reproduction of this article is prohibited key: cord- -g o forj authors: rai, ansaar t; leslie-mazwi, thabele m; fargen, kyle m; pandey, aditya s; dabus, guilherme; hassan, ameer e; fraser, justin f; hirsch, joshua a; gupta, rishi; hanel, ricardo; yoo, albert j; bozorgchami, hormozd; fiorella, david; mocco, j; arthur, adam s; zaidat, osama; siddiqui, adnan h title: neuroendovascular clinical trials disruptions due to covid- potential future challenges and opportunities date: - - journal: j neurointerv surg doi: . /neurintsurg- - sha: doc_id: cord_uid: g o forj to assess the impact of covid- on neurovascular research and deal with the challenges imposed by the pandemic. methods: a survey-based study focused on randomized controlled trials (rcts) and single-arm studies for acute ischemic stroke and cerebral aneurysms was developed by a group of senior neurointerventionalists and sent to sites identified through the clinical trials website (https://clinicaltrials.gov/), study sponsors, and physician investigators. results: the survey was sent to institutions, with responding ( %). stroke rcts were being conducted at ( %) sites, aneurysm rcts at ( %) sites, stroke single-arm studies at ( %) sites, and aneurysm single-arm studies at ( %) sites. following covid- , enrollment was suspended at ( %) sites—completely at ( %) and partially at ( %) sites. missed trial-related clinics and imaging follow-ups and protocol deviations were reported by ( %), ( %), and ( %) sites, respectively. negative reimbursements were reported at ( %) sites. the majority of sites, ( %), had put new trials on hold. of the coordinators, ( %) worked from home and ( %) reported a personal financial impact. remote consent was possible for some studies at ( %) sites and for all studies at ( %) sites. at sites with suspended trials (n= ), endovascular treatment without enrollment occurred at ( %) sites for stroke and ( %) sites for aneurysms. a total of patients with acute ischemic stroke and with cerebral aneurysms were treated without consideration for trial enrollment. conclusion: widespread disruption of neuroendovascular trials occurred because of covid- . as sites resume clinical research, steps to mitigate similar challenges in the future should be considered. the neurovascular community has had to date a relatively homogeneous response to the covid- pandemic with regards to clinical care. measures including elective case suspension and modifications for emergency treatment and safety procedures to accommodate patients with covid- have been widely adopted. societal guidelines containing overlapping recommendations, based on evidence in the literature and expert consensus, were published to codify this. [ ] [ ] [ ] by contrast, the impact of covid- on clinical research efforts has been less clear. the food and drug administration (fda) published its guidance on the 'conduct of clinical trials of medical products during covid- pandemic' for the industry, investigators, and institutional review boards in march and updated these on april , (https://www. fda. gov/ media/ / download). this guidance is expected to remain in effect until the public health emergency related to the pandemic is lifted by the department of health and human services. the fda document is comprehensive and recognizes the challenges due to quarantines, suspension of trials, and interruption of supply chains. it also recognizes that protocol deviations are likely and that trial modifications may be required. similar guidelines were issued by the european medicines agency on the management of clinical trials during the covid- pandemic on april , (https:// ec. europa. eu/ health/ sites/ health/ files/ files/ eudralex/ vol- / guid ance clin ical trials_ covid _ en. pdf). disruption of clinical research infrastructure has implications for patient enrollment, study timing, sponsorship, and advances in clinical care and disease understanding. at the time of writing, fears of resurgent or re-emergent disease remain prominent. the impetus of this study was clinical trial suspension and challenges in patient monitoring experienced by several physician investigators and enrolling sites. this led to formulation of a hypothesis and study design which aimed to evaluate the impact of covid- on neurovascular research studies and had a goal of better understanding the implications for existing and future trials. institutional review board approval was not obtained as the survey did not involve study of human subjects. a survey writing group committee, comprising neurointerventional physicians (four neuroradiologists, five neurosurgeons, and three standards neurologists) each with or more years of experience in clinical practice, was assembled. all were involved in neurovascular clinical trials as site principal investigators and six were leading clinical trials as national principal investigators. the writing group was tasked with composing a concise survey, limited to fewer than questions, to be completed by institutional research coordinators to explore the effect of covid- on neurointerventional research and clinical trials (full survey can be found in online supplementary materials a). the survey focused on endovascular trials for acute ischemic stroke and cerebral aneurysms, including both randomized clinical trials (rcts) and single-arm studies. the specific trial names or study sponsors were not recorded to protect confidentiality. the site name was only logged to assure one response from each site. no compensation was offered to participants. no requests to complete the survey were placed on public social media platforms. the questions were intended to explore key aspects of managing clinical trials in a restricted environment as imposed by the pandemic. these were identified by the writing group and fell into four categories: general disruption caused by trial suspensions and missed opportunities of enrollment, compromised trial quality due to inability of timely clinical and imaging follow-up, inability to enroll neurologically debilitated patients because legally authorized representatives were not at hand for face to face consent and dated remote consent procedures did not apply and, finally, personal effect of compensation or working conditions on study staff. the question about remote consent related to existing operating procedures for ongoing trials. the suspension of clinical trials was divided into partial or complete suspension based on whether some or all neuroendovascular trials were suspended. in cases of partial suspension, the question did not give details of the criteria for suspending some trials and not others. the survey also did not seek information on the size of the program, case volume, or association with a research consortium. this was partly to ensure that the brevity of the survey was maintained, and partly to maintain broad applicability of the survey regardless of program size, academic affiliation, or geographic location. the final questionnaire was also reviewed by three industry sponsors of clinical trials, two of which agreed to disclose their names (stryker neurovascular; fremont, california, usa and cerenovus; miami, florida, usa). the survey sample was identified by creating a list of participants from the federal repository website ( clinicaltrials. gov). the website allows combinations of search terms to identify clinical trials. the search terms used included 'stroke', 'thrombectomy', and 'brain aneurysms' with the geographic localizer of clinical trials in the 'united states'. the results were filtered for recruiting trials. the website provided contact details for each study. in a separate process, various study sponsors and principal investigators of ongoing clinical trials were approached by members of the writing group and asked to provide lists of participating sites. the accumulated sites were cross referenced with the list generated from clinicaltrials. gov to eliminate duplicates and when site information was missing, physician investigators localized by state were identified and invited individually through the society of neurointerventional surgery senior physician membership (https://www. snisonline. org/). the final sample resulted in a total of institutions identified for potential participation. the finalized survey was prepared on qualtrics (https://www. qualtrics. com) using an institutional license. the survey was distributed to the identified sites by email and was open for a -week period from april , to may , . to boost the response rate, a digital link to the survey and a printable version (pdf, word document) were emailed to the physician investigators and study coordinators midway through the -week period as a prompt. surveys were completed voluntarily by a research coordinator at each institution with the help of the physician investigator as required. each center was permitted only one response. the survey is presented as descriptive statistics. the survey was sent to study coordinators at institutions with responding ( %). all responding sites were actively involved in neurovascular trials before the covid- pandemic. of the returned forms, the only missing responses related to entry of the date of clinical trial suspension (if applicable). an evaluation of endovascular studies showed that a majority of sites were conducting stroke-related rcts or single-arm studies. most sites were also conducting single-arm aneurysm trials while a minority were participating in aneurysm-related rcts ( figure a) . during the pandemic, the majority of the sites either partially or completely suspended enrollment in clinical trials (figure b), with the suspension occurring between the middle to the end of march . there was no correlation between site trial suspension and geographic location or academic affiliation. suspension of sites by state showed a heterogeneity of response, with different sites in same state having different responses (figure ). no correlation was found between trial suspension and the intensity of the pandemic. endovascular stroke care continued in sites where enrollment had been suspended (n= ), and most of these sites treated patients with acute ischemic stroke without enrollment. almost half of the sites treated patients with an aneurysm (figure a). a total of patients with endovascular stroke and with endovascular aneurysm were treated without consideration for enrollment following suspension of clinical trials. effects were also observed on patients already enrolled in trials. forty-two percent of sites missed clinical follow-up visits, and % missed imaging follow-up. under half of the sites ( %) reported protocol deviations because of the pandemic and a quarter reported loss of reimbursements possibly due to delayed or missed milestones or inability to enroll patients ( figure b ). almost all ( / , %) sites had developed alternative mechanisms such as telemedicine for clinical visits. most of the sites had received institutional review board guidance for the conduct of clinical trials (n= , %) and most had received guidance from sponsors for either all (n= , %) or some (n= , %) of the clinical trials. over half of the sites allowed electronic or phone consent for enrollment ( figure ). there were widespread challenges for study staff, with the majority of the study coordinators working from home (n= , %) or a combination of home and office (n= , %) and almost a third (n= , %) reporting being personally affected by the pandemic either due to loss of pay, paid time off, or loss of benefits. the majority of the sites (n= , %) had placed new trials on hold, with the number of postponed trials per site ranging from one to seven. two sites reported a diagnosis of covid- among the enrolled patients. this study identified widespread disruptions to neurovascular clinical trial networks, most notably due to a suspension or postponement of trial enrollment, interruptions in scheduled patient follow-up, and challenges with study staff routines. cessation of ongoing trials was nearly uniform, which may have downstream effects on trial validity and solvency. understanding the effect of covid- on clinical trials during the early months of the pandemic is especially important given the uncertainty about the future and potential need for subsequent lockdowns should widespread infection recur after resumption of elective procedures and relaxation of social distancing measures. one previous letter has been published, documenting decreased enrollment related to the pandemic in cancer-related clinical trials, and another systematic review on cancer research during covid- . the federation of italian cooperative oncology group issued a brief commentary and practice indications for drug studies. other publications have included clinical trials for alzheimer's research, and a review for guidance in head and neck research. at the time of writing, no published studies measuring the comprehensive impact of covid- on research are available, and while this study focuses specifically on neurovascular research and trials, disruptions captured may well be generalizable to united states clinical trials in other medical specialties. a prominent source of disruption identified by this survey was the complete or partial suspension of trial enrollment. many sites with suspended enrollment treated potentially eligible patients with stroke and aneurysm. a total of patients with stroke and with an aneurysm reportedly underwent endovascular therapy at these sites during the period in which the survey was conducted and thus could not be considered for enrollment owing to the suspension of clinical research. in view of the number of sites that did not respond to the survey, this number may be even higher. unlike the 'clinical deficit', which is expected to be gradually overcome by returning case volume and case rescheduling, no similar option exists for research enrollment deficits. the most likely means of compensating for missed enrollments is to extend trial enrollment timelines. such extension has implications for funding, compounded by potential contraction of funding following the wider economic impact of the global pandemic. beyond enrollment alone, trial quality was compromised. many patients missed clinical and imaging follow-up milestones, and many sites reported protocol deviations due to the pandemic. these may increase as pandemic effects continue and could affect the veracity of overall trial data. site trial infrastructure will have to adjust to new realities of reduced funding for many centers, with just over a quarter reporting a negative impact on reimbursements. human trial expertise has also been affected by the covid- pandemic. sites could experience reassignment, furloughing, or termination of study coordinators. one-third of responding coordinators reported a direct personal effect through loss of pay, time off, or benefits. as operations gradually return, this human expertise may need to be rebuilt. trialists and staff will also need ongoing protection from potential infectious exposures. failure to do so could affect a site's ability to perform existing and future trials unless alternative mechanisms are developed. as clinical research activities resume with lifting of mandated lockdowns, this survey highlights several opportunities to update the conduct of clinical trials in accordance with the virtual care revolution brought about by the covid- pandemic. in this survey, % of the responding sites did not have any protocols in place for remote consent. challenges to the traditional consent process could persist into the foreseeable future as hospital visitor restrictions are likely to remain in place for some time. study sponsors, local institutional review boards, and federal authorities regulating these trials will have to demonstrate creativity in modernizing these procedures. harnessing emerging virtual technologies that allow for face-to-face interactions with legally authorized representatives, electronic exchange of documents, and digital signatures, represents a clear opportunity. it is possible to comply with the ethical principles governing informed consent while updating these procedures to meet the current and future needs of operating with restricted visits. in the recovery phase there are calls in both public and medical circles for our healthcare infrastructure to focus on broadly improving our approaches, not simply returning to the prior state of normalcy. for acute trials in particular this could have broad positive implications for enrollment. further, there are opportunities to enhance virtual capabilities to conduct trials, with reduced reliance on in-person monitoring and reporting forms. based on existing data about telehealth, this may expand the trial footprint, decrease trial costs, and potentially increase trial efficiency. trial reorganization into adaptive designs like platform trials may allow us to respond more rapidly to design changes and randomize more effectively as new questions arise. early efforts are underway to develop a neurovascular platform trial infrastructure, and this should remain a focus for the neurointerventional community. another important consideration is how to handle potential trial candidates regarding covid- exposure. in this survey, a small number (two sites) reported a positive covid- diagnosis in enrolled patients. there is current uncertainty about how to process patients with covid- in neurovascular trials or whether to screen based on symptoms or exposures. the direct effect of sars-cov- on cerebrovascular disease remains largely unknown, but early evidence suggests endothelial involvement and a range of syndromes - that could confound the natural history of the disease or treatment response. this may be particularly relevant for studies of cerebrovascular conditions where patient outcomes may be directly influenced apart from the interventions being studied. how patients with covid- should be handled by trials that are not focused on that disease specifically, is unknown. testing all potential enrollees for covid- and excluding all positive patients a priori seems reasonable, but this could result in unnecessary delays in enrollment or exclusion of patients who are covid false positive or who have recovered from covid. this larger question will probably be answered as we learn more about the virus. as more data are collected on the impact of the pandemic on clinical trials it may be possible to crystallize strategies for operating in a restrictive environment due to lingering concerns of outbreaks. this study has several important limitations. as a survey study, this study is subject to the inherent limitations of survey methodology, including recall and selection bias. the potential for selection bias is strong as those most affected by the pandemic may be more likely to complete the surveys. additionally, centers with furloughed or terminated coordinators were unable to provide responses. a number of potentially important research or practice-specific questions, such as region or the prevalence of covid- in the population, were not included in the survey by the writing group to improve participation rates. questions were intentionally generated to assess a broad overview of research as opposed to specifically focus on individual trial statistics. the survey questions were purposely kept under , and therefore details that might have been helpful in further exploring some responses were not obtained. for example, the criteria for choosing to keep some trials open and others suspended are not known. likewise, details of the causes of loss of trial-related reimbursements were not obtained. additionally, implications for neurovascular basic or laboratorybased research, or for research in other domains, were similarly not assessed. the questions used were not previously validated and were designed specifically for the ni research coordinators involved in clinical trials, which may limit their generalizability to other specialties and make comparison with data from other surveys challenging. lastly, since this type of pandemic has never occurred in modern history, there is no previously developed, validated survey instrument to study the effects of a global pandemic on research. this is the first study comprehensively evaluating the effect of covid- on clinical trials. respondents indicated widespread disruption of neurovascular clinical research and research infrastructure during the covid- pandemic. it has also revealed opportunities for increased infrastructural strength and pliability, including the increased use of virtual access for consent and follow-up, and the benefits of platform trial design at times when flexibility and responsiveness are key advantages. many questions remain to be answered, including how we should incorporate covid- screening into trial criteria. it should be expected that disruptions due to covid- will continue, based on expert predictions of the disease course. trial conduct in this restrictive environment will need to evolve. decisions about how to maintain enrollment during the current pandemic and in the event of future similar disruptions must be prioritized. society of neurointerventional surgery recommendations for the care of emergent neurointerventional patients in the setting of covid- letter: considerations for performing emergent neurointerventional procedures in a covid- environment preserving access: a review of stroke thrombectomy during the covid- pandemic how will country-based mitigation measures influence the course of the covid- epidemic? association of the coronavirus disease (covid- ) outbreak with enrollment in cancer clinical trials cancer patients and research during covid- pandemic: a systematic review of current evidence indications regarding the management of interventional clinical trials with drugs during the current covid- emergency in italy clinical trials and tribulations in the covid- era clinical trials during covid- risk factors of healthcare workers with corona virus disease : a retrospective cohort study in a designated hospital of wuhan in china a digital embrace to blunt the curve of covid pandemic virtually perfect? telemedicine for covid- a rationale and framework for seeking remote electronic or phone consent approval in endovascular stroke trials -special relevance in the covid- environment and beyond what the coronavirus crisis reveals about american medicine. the new yorker patient and clinician experiences with telehealth for patient follow-up care this is a platform alteration: a trial management perspective on the operational aspects of adaptive and platform and umbrella protocols endothelial cell infection and endotheliitis in covid- characteristics of ischaemic stroke associated with covid- covid- -related stroke large-vessel stroke as a presenting feature of covid- in the young innovation for pandemics key: cord- - zu fn authors: riley, william t; glasgow, russell e; etheredge, lynn; abernethy, amy p title: rapid, responsive, relevant (r ) research: a call for a rapid learning health research enterprise date: - - journal: clin transl med doi: . / - - - sha: doc_id: cord_uid: zu fn our current health research enterprise is painstakingly slow and cumbersome, and its results seldom translate into practice. the slow pace of health research contributes to findings that are less relevant and potentially even obsolete. to produce more rapid, responsive, and relevant research, we propose approaches that increase relevance via greater stakeholder involvement, speed research via innovative designs, streamline review processes, and create and/or better leverage research infrastructure. broad stakeholder input integrated throughout the research process can both increase relevance and facilitate study procedures. more flexible and rapid research designs should be considered before defaulting to the traditional two-arm randomized controlled trial (rct), but even traditional rcts can be designed for more rapid findings. review processes for grant applications, irb protocols, and manuscript submissions can be better streamlined to minimize delays. research infrastructures such as rapid learning systems and other health information technologies can be leveraged to rapidly evaluate new and existing treatments, and alleviate the extensive recruitment delays common in traditional research. these and other approaches are feasible but require a culture shift among the research community to value not only methodological rigor, but also the pace and relevance of research. despite increasing demands to produce timely and relevant research findings, our traditional research process remains painstakingly slow. randomized efficacy trials take approximately . years from the initiation of enrollment to publication [ ] , and years or longer after adding the time from grant application submission to enrollment initiation. extensive follow-up periods for relevant outcomes such as morbidity/mortality as well as delays in participant recruitment and publication can extend this time period to a decade or longer. during this period, scientific and technological advances will occur that may make the eventual findings less relevant or even obsolete. for illustration, figure shows a few of the salient consumer technologies introduced during a typical seven year clinical trial. these recent advances in consumer technologies are most impactful for mobile and wireless health research, [ ] but many less mainstream scientific and medical technological advances also occur while clinical trials are being conducted. for example, one explanation for the recently reported negative results of the sammpris trial was that stent technologies and surgery procedures had advanced substantially since study initiation [ , ] . this protracted period from concept to publication is further exacerbated by the slow and limited uptake of research findings into practice. balas and boren have estimated that it takes approximately years from concept to evidence implementation for the % of evidence that progresses to implementation [ ] . a recent report from the president's council of advisors on science and technology (pcast) estimated that , treatments are in development and concluded that major upgrades are needed in the research system to evaluate these treatments [ ] . an institute of medicine report on clinical trials states that "recognition is growing that the clinical trials enterprise in the united states faces substantial challenges impeding the efficient and effective conduct of clinical research to support the development of new medicines and evaluate existing therapies [ ] ." clearly, our current research enterprise is too slow, inefficient, and cumbersome to meet the rapidly evolving demand. what are needed are "rapid-learning research systems" that integrate researchers, funders, health systems, practitioners, and community partners asking clinically relevant questions, using efficient and innovative research designs, and leveraging rich, longitudinal data sets from millions of patients. to begin progress toward such a system, we considered and have described approaches to make research more rapid, responsive and relevant (r ), organized into four sections for the purpose of this paper: ) stakeholder engagement, ) design, ) review, and ) infrastructure (see table ). these suggested approaches are viewed as a starting point for a dialogue among the health research community to challenge our current cumbersome research enterprise and to consider these or other approaches to maintain scientific rigor while speeding the process by which more responsive and relevant research findings are produced. broad stakeholder engagement involving patients, providers, health plans, policy makers and other relevant stakeholders may seem counterintuitive as a strategy to speed research, but this time investment has the potential to improve the recruitment and retention of study participants, thus increasing the pace of conducting the study. more importantly, stakeholder engagement increases the likelihood that findings will be relevant to stakeholders and more readily adopted into practice, thereby making the overall research pipeline more efficient. these "evaluability assessments" [ ] or participatory approaches are considered key to facilitating the adoption of research findings by practitioners [ ] . the patient centered outcomes research institute (pcori), for example, is creating public advisory groups and soliciting patient input on specific comparative effectiveness questions that are relevant to practitioners and stakeholders [ ] . an ongoing relationship between researchers, healthcare providers, health plans, and patients is critical to a better, faster research system. clinical trial recruitment is a major problem with about % of us trials failing to meet enrollment goals [ ] . the nih health care system collaboratory (hcsc) offers an important resource for rapid research. like the hmo research network and the va queri program, the hcsc will make available opportunities to conduct large-scale studies within well-organized healthcare delivery systems [ ] . research embedded in organized delivery systems and networks enhances not only research relevance, but also facilitates recruitment, retention, study start-up, operations, data capture, and integration into practice. an accelerated research system can also use information technologies to speed the process of seeking and obtaining stakeholder feedback. consistent with a citizen-scientist model [ ] , a virtual network of various stakeholders can participate and provide feedback throughout the research process using online surveys, virtual meetings, and social media systems [ ] . via innovative technologies, stakeholder feedback can be obtained efficiently to increase research relevance and responsiveness, even for researchers who are not fully integrated into the practice setting or community where their research findings are likely to be translated into practice. traditional study designs and procedures are wellestablished, rigorous, and notoriously slow and costly. this belabored research process typically begins with pilot trials that we posit have limited benefit, are used inappropriately to estimate effect size [ ] , and often prematurely concretize a less than optimal intervention. instead, we recommend replacing the traditional pilot trial with a more flexible iterative intervention testing and optimization approach, analogous to the agile software development process that places a premium on failing early to succeed later [ ] . for example, n-of- trial designs provide intervention development flexibility. with the increasing availability of intensive longitudinal data from wireless sensors and mobile devices, n-of- trials can be rapidly implemented and provide results congruent with a more personalized medicine approach [ ] , and bayesian analyses from a series of such trials [ ] may provide sufficient evidence of generalizability to limit the need for a larger trial. intervention optimization designs such as fractional factorial and sequential multiple assignment research trials (smart) are particularly valuable when the intervention development questions involve combinations or sequences of intervention components [ ] . dynamic system models have also been used to optimize treatments [ ] . some optimization approaches may take more time than the traditional pilot trial, but the pace of the overall research enterprise will be improved by more quickly discarding or modifying interventions that are unlikely to be found effective in larger and more expensive trials. within the traditional rct, researchers have a number of design decisions that can increase efficiency. trials that utilize within-group designs in which participants serve as their own controls can speed the research process by reducing the number of study participants needed to detect outcomes, and can often simplify study procedures as well. the minimal intervention needed for change (minc) standard [ ] provides a standard pragmatic comparison anchor across studies for comparative effectiveness research. the va is adopting a "point of care" randomization that computer-randomizes patients to different treatments, and then uses adaptive algorithms to change allocation of new patients as evidence accumulates [ ] . recent technological advances make it possible to conduct "automated rcts" in which the enrollment, random assignment, intervention delivery, and outcome assessments are fully automated. to fully realize the potential of automated rcts and other rapid learning systems, the nature of and procedures for informed consent need to be resolved. follow-up periods also can be shortened or segmented. results can be analyzed at the point where the maximal benefit of the intervention is hypothesized to occur. longer-term outcomes can be modeled from these results, or one of the investigators can remain blind to conduct the follow-up portion of the study and publish the follow-up results separately. in addition to improving the efficiency of rcts, we also need to consider alternative designs that may be more appropriate to the research question and provide more rapid and relevant answers. a range of within-subject and quasi-experimental designs such as interrupted time series [ ] , stepped wedge [ ] , and regression discontinuity [ ] may have less internal validity than the rct, but offer a number of advantages [ ] . for example, these quasiexperimental approaches facilitated participation of the major minnesota health insurers in the diamond depression treatment program [ ] . these designs may be particularly appropriate for evaluating treatments already adopted in practice. it takes to months from nih grant submission to funding [ ] . if revised and resubmitted, and assuming a six month revision period, it can take two years from initial submission to the award of a revised (a ) nih grant application. during this time, science and technology continue to advance; research partnerships, especially with nonresearch stakeholders, must be maintained; and the research questions may become less relevant or timely. grant review and funding processes could be streamlined in a number of ways. for the recovery act challenge grants [ ] , a flexible, two-stage review process was implemented to reduce to five months the time from receipt to funding. rapid review processes are already used by the nih for time sensitive natural experiments [ , ] . in response to the sars (severe acute respiratory syndrome) outbreak, the canadian institutes of health research developed and issued a funding announcement that resulted in submissions within weeks, and these submissions were reviewed and four approved for funding within days of submission [ ] . these rapid review examples clearly indicate that it is possible to review and fund research applications quickly when necessary, and that such rapid review systems should be considered for a broader range of research, including timely and pressing clinical and public health questions. the grant application review process could facilitate more rapid research not only by reviewing more efficiently, but also by placing a greater premium on more rapid and innovative research designs. despite the addition of innovation as an nih review criteria, a recent study of grant applications revealed that novelty is associated with a . percentile point drop, and that feasibility concerns did not contribute substantially to this "novelty penalty" [ ] . innovative designs, especially those that speed the pace of research relative to traditional designs, should be rewarded, not penalized. institutional review boards (irbs) also should consider streamlining review procedures. slowness of research should be considered a risk, both to study participants who may continue in their assigned treatment even as newer treatments become available, and to the broader public who are delayed getting answers to relevant research questions. revisions to the common rule are anticipated to allow for a more flexible and rapid review process [ ] . online and open access publication practices have greatly reduced the time from acceptance to publication [ ] but could be further facilitated by a better or more incentivized process for acquiring reviewers and obtaining reviews. as green noted, new technologies for publication, systematic reviews, and dissemination of evidence-based guidelines reduce the time from research findings to practitioner adoption, but the publication and dissemination process should continue to be reviewed to further reduce the time lapses between the various stages of the dissemination and implementation process [ ] . improving our research infrastructure has the potential not only to speed the pace of research, but also increase its rigor and relevance. the health system has lagged decades behind other sectors in it implementation [ ] . as a result, health research has been severely constrained by a datapoor environment in which acquiring needed research data is expensive, difficult, and time-consuming. since the rapidlearning health system and learning healthcare system concepts were advanced in [ ] , major investments have been made in databases and learning networks to take advantage of the research potential of electronic health records. it is now possible to conduct some studies in weeks or months instead of years. the fda mini-sentinel system accesses million patient records to generate several studies per week on drug safety questions [ ] . large biobanks are now coming on-line at kaiser-permanente, [ ] the veterans health administration [ ] , the en-code network, [ ] , and the uk biobank [ ] . using these and other patient databases, researchers have been able to assess the unintended effects of treatments [ ] and produce outcome findings comparable to rcts [ ] . large future investments are now being considered that could offer extraordinary opportunities for researchers and a faster, more efficient infrastructure for rapid learning research. the nih director has proposed a new national patient-oriented research system with electronic health records databases, including genomics, for - million patients [ ] , and pcori recently released a funding announcement to support development of the national patient centered clinical research network [ ] . the big data to knowledge (bd k) initiative [ ] also provides the opportunity to leverage these large data sets for rapid research. researchers can accelerate the collective pace of learning with greater attention to reporting comparable data. there have been a number of efforts to encourage the use of common data elements [ ] . standardized outcome data are particularly problematic for patient-reported outcomes. to address this problem, there have been consensus measurement efforts such as phenx [ ] as well as efforts to co-calibrate various patient-reported outcome measures on a single metric [ ] . the national research council report on precision medicine calls for a new science commons and national learning system that will revolutionize biomedical research, clinical care, and public health [ ] . one of the benefits of such a system is that researchers can more readily target drug approval studies to predicted high-response populations, and cut years from the drug research process. gleevec, an anti-cancer drug, was approved in a trial of only patients because nearly all showed benefit [ ] . with targeted therapeutics and research, it took only four years from target discovery to drug approval for the lung cancer drug xalkori [ ] . rapid learning systems of large patient populations appear to provide the infrastructure to rapidly evaluate treatments. we have outlined a number of actions to enhance relevance, streamline design, speed review, and use new research infrastructures to make research more rapid, relevant, and responsive to the st century demands on health research. this transformation to a rapid research learning system will require a concerted effort by research funders, academic institutions, healthcare systems, researchers, and a variety of practice, community, and policy stakeholders to produce a culture change among the health research community. will we continue to use limited funds to support the currently slow and cumbersome research enterprise that produces costly results that may not be relevant or easily translated into practice, or are we willing to pursue alternative approaches that in other research disciplines have produced rapid and relevant improvements? the rationale and opportunities for such a culture change have never been greater or rapid answers more needed. our recommendations to speed research are undoubtedly incomplete, and we invite the research community to contribute additional recommendations to increase the speed and relevance of the research enterprise. this call to streamline and speed the research process is also likely to be met with skepticism, especially among those who fear that methodological rigor might be compromised in a quest for greater efficiency. we believe that the recommendations outlined in this paper can be achieved without compromising scientific rigor, and any efforts to streamline research should be judged based on methodological soundness. we are convinced, however, that the currently dominant health research paradigm is too slow and inefficient to address today's challenges, and that we must produce a more rapid, responsive, and relevant research enterprise. effect of the statistical significance of results on time to completion and publication of randomized efficacy trials advancing the science of mhealth stenting versus aggressive medical therapy for intracranial arterial stenosis interpretations and implications of the prematurely terminated stenting and aggressive medical management for preventing recurrent stroke in the intracranial stenosis (sammpris) trial managing clinical knowledge for health care improvement: yearbook of medical informatics council of advisors on science and technology (pcast) report to the president on propelling innovation in drug discovery, development, and evaluation envisioning a transformed clinical trials enterprise in the united states evaluability assessment to improve public health policies, programs, and practices making research relevant: if it is an evidence-based practice, where's the practice-based evidence? a new dawn for citizen science web-based dynamic delphi: a new survey instrument the role and interpretation of pilot studies in clinical research agile methods for open source safety-critical software the n-of- clinical trial: the ultimate strategy for individualizing medicine individual (n-of- ) trials can be combined to give population comparative treatment effect estimates: methodologic considerations the multiphase optimization strategy (most) and the sequential multiple assignment randomized trial (smart): new methods for more potent e-health interventions using engineering control principles to inform the design of adaptive interventions: a conceptual introduction finding the minimal intervention needed for sustained mammography adherence a point-of-care clinical trial comparing insulin administered using a sliding scale versus a weight-based regimen applying a propensity score-based weighting model to interrupted time series data: improving causal inference in programme evaluation the stepped wedge trial design: a systematic review a randomized experiment comparing random and cut-off based assignment practical clinical trials for translating research to practice: design and measurement recommendations partnership research: a practical trial design for evaluation of a natural experiment to improve depression care grants review process recovery act limited competition: nih challenge grants in health and science research (rc ) time-sensitive obesity policy and program evaluation (r ) innovation and challenges in funding rapid research responses to emerging infectious diseases: lessons learned from the outbreak of severe acute respiratory syndrome the novelty paradox and bias for normal science: evidence from randomized medical grant proposal evaluations human subjects research protections: enhancing protections for research subjects and reducing burden, delay, and ambiguity for investigators what does the marriage of open access and online publication bring? what' s holding back online medical data etheredge lm: a rapid-learning health system a systematic review of validated methods for identifying pancreatitis using administrative data the encode project: encyclopedia of dna elements unintended effects of statins in men and women in england and wales: population-based cohort study using the qresearch database use of primary care electronic medical record database in drug efficacy research on cardiovascular outcomes: comparison of database and randomized controlled trial findings nih common data element research portal the phenx toolkit: get the most from your measures measuring fatigue in persons with multiple sclerosis: creating a crosswalk between the modified fatigue impact scale and the promis fatigue short form delivering value through personalized medicine: an industry perspective rapid, responsive, relevant (r ) research: a call for a rapid learning health research enterprise the authors declare that they have no competing interests. preparation of this manuscript was supported in part by a grant from the robert wood johnson foundation to george washington university to the third author. the funders had no role in this manuscript. the opinions expressed are those of the authors and do not necessarily reflect those of the robert wood johnson foundation or the national cancer institute.authors' contributions wtr led the writing and preparation of this paper. reg, le, and apa all contributed sections to this paper, and revised and edited drafts of the paper. all authors read and approved the final manuscript. convenient online submission rigorous peer review immediate publication on acceptance open access: articles freely available online high visibility within the fi eld retaining the copyright to your article submit your next manuscript at springeropen.com key: cord- - sku a authors: pahus, laurie; suehs, carey meredith; halimi, laurence; bourdin, arnaud; chanez, pascal; jaffuel, dany; marciano, julie; gamez, anne-sophie; vachier, isabelle; molinari, nicolas title: patient distrust in pharmaceutical companies: an explanation for women under-representation in respiratory clinical trials? date: - - journal: bmc med ethics doi: . /s - - -y sha: doc_id: cord_uid: sku a background: patient skepticism concerning medical innovations can have major consequences for current public health and may threaten future progress, which greatly relies on clinical research. the primary objective of this study is to determine the variables associated with patient acceptation or refusal to participate in clinical research. specifically, we sought to evaluate if distrust in pharmaceutical companies and associated psychosocial factors could represent a recruitment bias in clinical trials and thus threaten the applicability of their results. methods: this prospective, multicenter survey consisted in the administration of a self-questionnaire to patients during a pulmonology consultation. the questionnaires distributed collected demographics, socio-professional and basic health literacy characteristics. patients were asked to rank their level of trust for pharmaceutical companies and indicate their willingness to participate in different categories of research (pre or post marketing, sponsored by an academic institution or pharmaceutical company). logistic regression was used to determine factors contributing to “trust” versus “distrust” group membership and willingness to participate in each category of research. results: one thousand patients completed the survey, corresponding to a response rate of . %. data from patients were analyzed in this study. . % of respondents declared that they trusted pharmaceutical companies, while . % declared distrust. being female (p = . ), inactive in the employment market(p = . ), and not-knowing the name of one’s disease(p = . ) are factors related to declared distrust. distrust-group membership is associated with unwillingness to participate in certain categories of trials such as pre-marketing and industry-sponsored trials. conclusion: distrust in pharmaceutical companies is associated with a specific patient profile and with refusal to participate in certain subcategories of trials. this potential recruitment bias may explain the under-representation of certain categories of patients such as women in pre-marketing drug trials. public distrust in healthcare systems and directed towards physicians, regulatory authorities and the pharmaceutical industry in general has increased over the past decades [ , ] . a succession of health-related controversies and scandals related to safety issues involving pharmaceutical companies' or physicians' conflicts of interest has led to this erosion of trust [ ] . the diffusion of selected information via mass media, which tends to sensationalize negative phenomena, greatly shapes the public's perception of healthcare systems. the influence on medical decision-making among both physicians and patients is difficult to assess [ ] [ ] [ ] [ ] . pharmaceutical companies are suspected of putting profits above public interest, using marketing techniques to distort scientific evidence, and actively influencing both physicians and health policy makers [ ] [ ] [ ] . this weak level of trust translates into skepticism about using pharmaceutical products, thus leading to new patient behaviors ranging from poor adherence [ , ] to strong rejection of health policies, such as vaccine campaigns [ , ] . rejecting the implementation of medical and scientific findings not only has major consequences for current public health, but may also threaten future innovations and advances in medicine that principally rely on clinical research led by pharmaceutical companies, who have ever-growing needs for enrollment [ ] . for clinicians, it can be fascinating to observe whether or not patients who have waited several weeks in order to attend a highly-specialized clinic for improving their condition are equally willing (or not) to participate in a clinical trial. in some countries, financial issues may affect this decision, but in the french health care system, all patients freely access all available drugs for a condition. one of the main motivations for attending a highly-specialized clinic in france is therefore the possibility of early access to new drugs or devices via participation in clinical research. the recruitment and retention of patients in clinical trials is challenging, but necessary, because it is the cornerstone of medical evidence production. the additional challenge is to recruit a panel of patients as representative as possible of the future target population. the latter is meant to ensure the accuracy and generalizability of the efficacy and safety conclusions generated by clinical trials. nevertheless, the representativeness of study populations vis-à-vis real-life patients has been demonstrated as quite poor due to the narrow eligibility criteria required to enter a clinical trial [ ] . excessive eligibility criteria are a patient-extrinsic factor that can damage a trial's external validity. in contrast, patient-determined factors may also apply, leading to the non-inclusion of seemingly otherwise eligible patients. moreover, for geographical or investigator-determined cognitive bias some patients are never approached by centers conducting clinical trials which may weaken even more the external validity of trials [ ] . in this study, we hypothesized that distrust in pharmaceutical companies is correlated with a specific psychosocial profile among patients. we sought to evaluate patient willingness to participate in different categories of clinical trials and associated predictive factors in order to determine whether or not global distrust in the healthcare system could represent a recruitment bias in clinical trials. this prospectively designed study was approved by an independent ethics committee (le comité de protection des personnes sud-méditerranée i; reference number: - ). as per french legislation and ethics committee approval, no written inform consent was required. patients' consent was implied upon the completion of the questionnaire. on march , study has been retrospectively registered on open science framework and www.clinicaltrials.gov with reference number nct . anonymous self-questionnaires were filled out before consultation by consecutive patients consulting for respiratory diseases (excluding oncology and tobaccology) in the respiratory disease departments of respiratory centers in southern france between december and december . the participating establishments included two public (teaching/research) hospitals (the university hospitals of marseille and montpellier) and two private centers (the polyclinique saint-privat, boujan-sur-libron, and the clinique de marignane, marignane). the protoaccept questionnaire included fields corresponding to age, sex, employment, education level (university schooling or not), whether or not the participant knew the name of his/her disease (yes/no), whether or not he/she had already participated in a research protocol (yes/no/i don't know) and whether or not this was his/her first consultation in the department (yes/no). the general diagnosis (interstitial lung disease, chronic airway disease, sleep disorders, other), indicated by the investigator, was also recorded for each patient, as well as the public or private nature of the including center. patients were also asked to rank their distrust for pharmaceutical company medical research studies according to a -point likert scale ("i would refuse participation in a medical research study because i distrust pharmaceutical companies": strongly disagree -disagree -neither agree nor disagree -agree -strongly agree). they were further asked whether or not they would agree to test a new drug before (yes/no) or after (yes/no) commercialization, and whether or not they would agree to participate in research at different kinds of institutions (pharmaceutical company and three types of public facilities: university hospital, general hospital, or a public research institute)(yes/no for each option). this questionnaire was built by a multidisciplinary team (private and public physicians, psychologist, nurses, and a pharmacist) and validated by a psychologist in a steps process. the first step included a qualitative pre-survey with thirty patients and consisting in open questions designed to capture events to report in the questionnaire. in the second step, a first version of the questionnaire was administered face to face by the psychologist to ten patients. ambiguous words were removed and medical vocabulary simplified. the final version was tested on five patients in typical survey settings, in the presence of the psychologist but without her intervention. patients who indicated that they agreed or strongly agreed with the statement "i would refuse participation in a medical research study because i distrust pharmaceutical companies" defined the "distrust" group. patients who checked one of the other three categories (neutral or disagreement) defined the "trust" group. patients who did not answer the question were not classified. the sample size was arbitrarily set at answered questionnaires. variations in count data are due to missing data, which was consistently under % for all questionnaire fields and therefore not otherwise addressed. all statistical analyses were carried out using the r (version . . ) programming environment [ ] . results are presented as numbers and percentages for qualitative variables. after evaluating the distribution of the age variable via a shapiro-wilks test, the latter was presented via medians with interquartile ranges. group comparisons were performed using unadjusted χ tests for qualitative variables and mann-whitney tests for age. multiple (logistic) regressions were performed with covariates (variables were selected if their associated p value was less than . in univariate analyses and a backward procedure was used to select the final model) and then presented as adjusted odds ratios with % confidence intervals. the first model performed used age, gender, education, employment, doctor-reported diagnosis group, patient experience (first consult in respiratory disease ward, previous participation in research), the public/private nature of the including center and whether or not the patient knew the name of his/ her pathology to predict patient membership in "trust" versus "distrust" groups. to additionally explain patient willingness to participate in new-drug studies or research associated with pharmaceutical companies or with public institutions, further models used the same predictive variables as for the first model, plus distrustgroup-membership as an additional explanatory variable. one thousand questionnaires were answered (the response rate to this questionnaire was . %.), ( · %) patients were in the "distrust" group, ( · %) in the "trust" group, and ( · %) did not answer the question (unclassified). thus, our analysis was performed on patients characterized by their membership in the "trust" or "distrust" groups. as compared with the "trust" group, individuals in the "distrust" group were slightly but significantly older and significantly more often professionally inactive ( table ). the two groups were statistically similar in terms of patient experience, be it as concerns previous consults in the respiratory disease ward, whether or not the latter was public or private, or as concerns previous participation in clinical research. despite similar rates of universitylevel education, the distrust group had significantly fewer patients who knew the name of their pathology, and fewer patients with a doctor-reported diagnosis relating to a sleep disorder (table ) . in terms of selecting potential factors for predicting "distrust", the following variables provided significant (or near-significant) results at the univariate level: age, gender, inactive employment status, doctor-reported sleep disorder, and whether or not the patient knew his/her pathology. of the latter, only three variables remained significant at the multivariate level: in summary, being female, inactive in the employment market, and not knowing the name of one's pathology are factors related to the patient's declared distrust in pharmaceutical companies (adjusted odds ratios (aor) of · [ · - · ; p = · ], · [ · - · ; p = · ] and · [ · - · ; p = · ], respectively) ( table ) . patient willingness to participate in different clinical trial subcategories % ( / ) of surveyed patients declared their willingness to participate in at least one subcategory of trial. % ( / ) indicated that they were willing to test a drug ( % ( / ) if the drug was already on the market and % ( / ) prior to market authorization). a (table ). our study aimed at evaluating variables associated with patient willingness to participate in different categories of clinical trials and at identifying a potential recruitment bias in clinical trials related to patient distrust in the pharmaceutical industry and healthcare systems. for this purpose, we conducted a two-part analysis in respiratory disease patients surveyed about their opinions concerning clinical research and potential willingness to participate. the majority of patients surveyed suffered from chronic airway diseases (asthma, chronic obstructive pulmonary disease (copd), i.e. the most prevalent chronic respiratory diseases in france [ ] . first, we compared patient characteristics on the basis of their level of trust in the pharmaceutical industry. in a second step, we assessed willingness-to-participate in clinical trials for our whole population to identify factors associated with acceptance or refusal. several studies have previously evaluated such rates and highlighted that altruism, hope for personal benefit, contribution to advances in science as well as financial benefit are the main reasons for agreeing to participate, whereas fear of adverse events, impossibility to cope with the logistic constraints accompanying participation, poor knowledge about or negative perception of clinical trials and distrust in pharmaceutical industry are potential barriers [ ] [ ] [ ] [ ] . thus, distrust in pharmaceutical companies was suspected to decrease patient willingness to participate in clinical trials but, to our knowledge, this has never been confirmed by juxtaposing patient-reported willingnessto-participate and their level of trust in pharmaceutical companies. we found that . % of respiratory disease patients declared they did not trust pharmaceutical companies. one major finding is that the profile of the latter, "distrusting" patients significantly differs from trusting patients: distrust is associated with female gender, inactive professional status and lack of knowledge on one's own disease. in the second step of our analysis, we calculated rates of willingness to participate in clinical trials. such rates [ ] [ ] [ ] [ ] [ ] [ ] . this variation seems to depend on certain study characteristics: for example, willingness increases with disease severity [ ] . clinical research is a wide and heterogeneous area with multiple methodologies, interventions and objectives. referring to "clinical trials" as a single concept is simplistic. similarly, considering willingness-to-participate in a specific subcategory of clinical trials in order to assess global perceptions of clinical research may be misleading. because we distinguished different categories of clinical trials in our survey, we were able to identify trialcategory-related barriers and associated differences in patient profiles. thus, the overall rate of willingness-toparticipate in at least one category of trial was very high ( %) among the surveyed patients we analyzed. globally, patients are not reluctant to join clinical research but, as expected, we found rates that vary significantly depending on the type of trial (intervention tested, market status of the drug and type of trial sponsor) ranging from to %. refusal of drug trials was associated with sex, women being more reluctant to join drug trials than men. among drug trials, an important difference exists in patient willingness-to-participate between pre-marketing and post-marketing studies. only % of patients we surveyed would have accepted enrolment in a pre-marketing drug trial, while % would have participated in a post-marketing trial. we assume this highlights patient trust in the regulatory health authorities responsible for marketing approval. poor knowledge about one's own disease and distrust in the pharmaceutical industry were associated with the refusal of pre-marketing drug trials, but played no role in refusing post-marketing trials. when comparing willingness-to-participate among different types of clinical trial sponsors, we found consistent results with only % of patients considering participation in industry-sponsored trials, while % would have agreed if the sponsor were academic. again, distrust in pharmaceutical companies plays a role in this difference. this is a major finding which, to our knowledge, has never been identified previously. it may also present a lever for external validity improvement in industry-sponsored trials. thus, targeted educational programs for improving knowledge on both diseases and clinical trials, as well as unbiased media communications about the benefits and risks of collaboration with pharmaceutical companies, could restore patient trust in pharmaceutical companies and increase willingness-toparticipate in clinical trials. a joint effort for education and communication involving mass media, the pharmaceutical company, regulatory authorities and physicians is crucial for both current and future public health. additionally, post-marketing academic trials should be encouraged as well as pooled analyses of industrial and academic results. the main limitation of the present survey is that it was conducted on a specific patient population, i.e. chronic respiratory disease patients. the results should not be extrapolated beyond chronic diseases which are not life-threatening in the short term. the second specificity is that the patients surveyed, living in france, have access to a robust health system which covers the costs of medical care and treatment. in some countries, participation in a clinical trial may be the only way to access care and treatment. our study does not analyze if the barriers to participation that we have identified are likely to influence decision-making in this context. one other limitation of our survey was that we assessed the hypothetical intention to participate, which has been shown to be higher than real participation rates [ ] . some patients who declare they would accept participation may refuse once the offer to participate becomes real. we hypothesize that this situation may occur in any subgroup of patient and thus assume that the ranking of rates among trial subcategories, as well as refusal risk factors, are accurate. the questionnaire was developed specifically in french for this study and has not been previously published in any peer-reviewed journal. a bilingual version, provided without cultural validation of the translation, is available as additional file. in short, our results suggest that patient distrust in the pharmaceutical industry could help explain recruitment bias in industry─sponsored clinical trials conducted in similar settings. attention should be paid to this phenomenon because the majority of therapeutic innovations are provided by pharmaceutical companies able to invest significant resources in research and development [ ] . moreover, we found that patient distrust is associated with a distinct patient profile. thus, we have no guarantee of the applicability of trial results for these patients. the under-representation of women, minorities and patients with poor health literacy in industrysponsored clinical trials has recently gained awareness [ ] [ ] [ ] [ ] and may threaten the generalizability of results [ ] . however, there is still very little available data focusing on women enrolment by country or disease. in the french asthma population for example, percentages of women enrolled are higher in women in france in a national academic interventional cohort than in a national industry led early access program to mepolizumab ( . and %, respectively) [ , ] . some data suggest that this under-representation of women exists worldwide even in countries where patients might be willing to participate in trials only to have their treatment free of charge [ , ] . in this survey, we were not authorized to collect ethnicity, so we cannot conclude on this specific characteristic. however, we found that women and patients with poor knowledge concerning their disease were more likely to distrust pharmaceutical companies and are less willing to participate in industry-sponsored trials. this could at least partly explain underrepresentation of these sub-populations in industry-sponsored clinical trials and could be addressed by implementing educational strategies [ ] [ ] [ ] as mentioned previously. the publishing process of the present manuscript takes place during the covid- pandemic period. it is difficult to anticipate how the current situation will shape and modify patients' perception of pharmaceutical companies. trusting opinions associated with the hope that pharmaceutical companies will find a treatment and distrusting opinions that accuse such companies of taking advantage of the situation to make money coexist. this should be the endpoint of a specific future study. distrust in pharmaceutical companies is associated with a specific patient profile and with refusal to participate in pre-marketing industry-sponsored drug trials. our results suggest that patient distrust in the pharmaceutical industry could help to explain recruitment bias in indus-try─sponsored clinical trials. further studies are needed to extrapolate results to different healthcare systems and beyond chronic diseases. a potential means for improving the external validity of industry-sponsored clinical trials is to implement educational strategies to increase unbiased knowledge of these trials. there are many ways to achieve this call to action. any initiative is beneficial and all communication media, small and large scale, are complementary. a joint effort dedicated to fighting against fake news and biased media supported by the payers and policy makers appears mandatory and eventually protective. supplementary information accompanies this paper at https://doi.org/ . /s - - -y. additional file . questionnaire protoaccept -french/english bilingual version. public trust in physicians--u.s. medicine in international perspective trust relations in health care--the new agenda scandals have eroded us public's confidence in drug industry medical decisions regarding hormone therapy for menopausal women are significantly influenced by the media the impact of media attention, family history, politics and maturation on women's decisions regarding hormone replacement therapy mass media and medicine: challenges and opportunities adverse publicity of serious side effects to healthy volunteers has limited effect on willingness-to-participate in clinical trials characteristics of general practitioners who frequently see drug industry representatives: national cross sectional study pharmaceutical industry sponsorship and research outcome and quality: systematic review evidence b(i)ased medicine--selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications patient trust-in-physician and race are predictors of adherence to medical management in inflammatory bowel disease medication adherence: truth and consequences effect of mass media on influenza vaccine coverage in the season / : a regional survey in lazio, italy association between vaccine refusal and vaccine-preventable diseases in the united states: a review of measles and pertussis financial anatomy of biomedical research external validity of randomized controlled trials in severe asthma incentives and disincentives to participation by clinicians in randomised controlled trials r: a language and environment for statistical computing. vienna: r foundation for statistical computing so different, yet so similar: meta-analysis and policy modeling of willingness to participate in clinical trials among brazilians and indians unique perception of clinical trials by korean cancer patients involving south asian patients in clinical trials hypertensive patients' willingness to participate in placebo-controlled trials: implications for recruitment efficiency patients' willingness to participate in clinical trials and their views on aspects of cancer research: results of a prospective patient survey participants and study decliners' perspectives about the risks of participating in a clinical trial of whole genome sequencing the new nih and fda medical research policies: targeting gender, promoting justice underrepresentation of women, elderly patients, and racial minorities in the randomized trials used for cardiovascular guidelines barriers to cancer clinical trial participation among saudi nationals: a cross-sectional study sex effect on average bioequivalence clinical and biological characteristics of the french cobra cohort of adult subjects with asthma real-life experience with mepolizumab in the french early access program for severe eosinophilic asthma the more things change, the more they stay the same: a study to evaluate compliance with inclusion and assessment of women and minorities in randomized controlled trials participation of women in clinical trials supporting fda approval of cardiovascular drugs what the public knows and wants to know about medicines research and development: a survey of the general public in six european countries patient perceptions and willingness to participate in clinical trials factors associated with willingness to participate in clinical trials: a nationwide survey study publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations none. the datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request after approval from all the authors.ethics approval and consent to participate this prospectively designed study (www.clinicaltrials.gov: nct ) was approved by an independent ethics committee (le comité de protection des personnes sud-méditerranée i; reference number: - ). as per french legislation and ethics committee approval, no written inform consent was required. patients' consent was implied upon the completion of the questionnaire.consent for publication n.a. none of the authors reports interests related to the present work. dr. laurie pahus reports consultancies for astra zeneca and chiesi pharmaceuticals. dr. carey suehs has no conflict of interest to declare. dr. laurence halimi reports industry-sponsored grants and lecturer activities for glaxosmithkline, actelion and novartis. dr. arnaud bourdin reports industry-sponsored grants from astrazeneca-medimmune, boehringer-ingelheim, cephalon/teva, glaxosmithkline, novartis, sanofi-regeneron; consultancies for astrazeneca-medimmune, boehringer-ingelheim, glaxosmithkline, novartis, regeneron-sanofi, med-in-cell, actelion, merck, roche, chiesi; investigator/co-investigator activities for trials promoted by astrazeneca-medimmune, boehringer-ingelheim, glaxos-mithkline, novartis, regeneron-sanofi, chiesi, actelion, merck, roche, vertex, galapagos; no personal financial support from a non-commercial source; no personal relationships with tobacco industry entities; no off-label disclosures. dr. pascal chanez, as an advisory board member, consultant or lecturer, has previously received honoraria or grants from boehringer ingeheim, almirall key: cord- -co nk pt authors: eichler, hans‐georg; cavaleri, marco; enzmann, harald; scotti, francesca; sepodes, bruno; sweeney, fergus; vamvakas, spiros; rasi, guido title: clinical trials for covid‐ : can we better use the short window of opportunity? date: - - journal: clin pharmacol ther doi: . /cpt. sha: doc_id: cord_uid: co nk pt the scientific community has risen to the covid‐ challenge, coming up with an impressive list of candidate drugs and vaccines targeting an array of pharmacological and immunological mechanisms. yet, generating clinical evidence of efficacy and safety of these candidate treatments may be frustrated by the absence of comprehensive trial coordination mechanisms. many small stand‐alone trials and observational studies of single‐agent interventions are currently running or in planning; many of these will likely not deliver robust results that could support regulatory and patient‐level treatment decisions. we here discuss actions that all stakeholders in the clinical trial ecosystem need to take to ensure that the window of opportunity during this pandemic will not shut, both for patients in need of treatment and for researchers to conduct decision‐relevant clinical trials. absence of comprehensive trial coordination mechanisms. many small stand-alone trials and observational studies of single-agent interventions are currently running or in planning; many of these will likely not deliver robust results that could support regulatory and patient-level treatment decisions. we here discuss actions that all stakeholders in the clinical trial ecosystem need to take to ensure that the window of opportunity during this pandemic will not shut, both for patients in need of treatment and for researchers to conduct decision-relevant clinical trials. this article is protected by copyright. all rights reserved the scientific community has risen to the covid- challenge. research funding bodies, academic and clinical centres, life science companies, and regulatory agencies are bending over backwards to enable the rapid development and authorisation of anti-covid- treatments. the result is an impressive and growing list of candidate drugs and vaccines targeting an array of pharmacological and immunological mechanisms . experience teaches that most candidate treatments will fail but with so many shots at goal there is reason for cautious optimism that at least a handful may succeed. so far, so good. yet, one bottleneck may frustrate many useful efforts. in the absence of comprehensive trial coordination mechanisms, stand-alone trials and observational studies of single-agent interventions are mushrooming. we and others have drawn attention to this development in the global covid- clinical trials landscape which we consider unfortunate, for the reasons outlined below. a recent search of the european eudract database (date of search / / ) shows no less than interventional trials being planned or running in the eu alone; at least of these have already started. of note, clinical trial applications were submitted by non-commercial sponsors, versus by commercial ones. most trials are mono-country trials, trials are conducted in different eu-member states while one trial involves eu-member states. forty-six trials involve less than subjects, trials include between and subjects, and trials plan to enrol more than a thousand subjects. even early on during the pandemic, the question has rightly been asked "do we need trials? is that a good use of resources?" . as of / / , well over , interventional and non-interventional trials have been registered world-wide . aside from the important resource issue and the ethical issue of having to enrol so many patients in small, individual control groups, we raise another obvious concern: can these trials possibly deliver? we recall that the worst outcome of a clinical trial is not a negative result. while clear-cut negative results may dash hopes of a cure (and professional aspirations) they are useful in showing us what treatments not to expose patients to and what lines of research to abandon early on. the real worry is trials that leave us as much in the dark as we were before the trial was conducted. consider a series of compassionate use applications of remdesivir for patients with severe covid- . the conclusion from observing patients treated was "measurement of efficacy will require ongoing randomized, placebocontrolled trials of remdesivir therapy." . could we have gleaned more useful information from the fate of those same patients by including them in a well-coordinated multi-arm trial? it is easy to detect large treatment effects but tricky to demonstrate the relatively small effect sizes we will likely see with single-agent treatment approaches for covid- (with the possible exception of this article is protected by copyright. all rights reserved vaccines) . for example, a -patient randomized controlled trial of lopinavir plus ritonavir showed no benefit beyond standard care but beneficial effects in subgroups cannot be excluded. the authors concluded that "future trials in patients with severe illness may help to confirm or exclude the possibility of a treatment benefit" . for every week that trials don't deliver, more and more patients are exposed to the wrong treatments, which well-designed and rapidly run clinical trials could have taken off the table, making space to pursue, other, and ultimately more meaningful, therapeutic options. we acknowledge the difficulties of running larger, coordinated multicentre trials in an extremely challenging environment, especially in the early days of the pandemic. however, as drug regulators, we ask: how many of the ongoing trials will be able to support robust regulatory (and individual treatment) decisions? even under ideal clinical trial settings, a substantial fraction of trials in epidemiologically stable diseases like diabetes or depression fail; one of the most frequent reasons for failure is lack of patient enrolment . covid- is characterized by a highly dynamic epidemiology in all global regions. shown that despite compressed development plans no study for therapeutics and vaccines could be completed as initially planned due to the waning epidemic, and only one prematurely terminated clinical study gave sufficient results to be used for vaccine authorization. similarly, with zika virus, vaccines were ready for large clinical testing when the virus was disappearing making it impossible to generate clinical evidence. while covid- is different from ebola, in that it affects essentially all global regions, we are still concerned that a high fraction of trials starting in a confined geography at the peak of this pandemic will not be able to enrol patients up to their pre-established sample size. to make matters even more complex, it is improbable that a single drug will be enough to control and improve the most severe forms of covid- , given its systemic pathophysiology. successful treatment regimens will likely need to address: treatment of the viral infection itself, to reduce and eliminate the viral load; prevention and treatment of the development of the severe acute respiratory distress syndrome and of hyperinflammation and cytokine storm; and additional pathophysiologic components of the severe disease such as blood coagulation activation/ thrombosis. we agree with gaborit et al that investigators should join their efforts in proposing studies of combined approaches through multifactorial this article is protected by copyright. all rights reserved designs. large platform trials offer the possibility to compare multiple treatments and approaches within a trial and across trials especially when they share standard disease classifications and outcome measures. such large studies afford an opportunity to agree on clear and consistent definitions and categorization of disease stages, viral infection and respiratory distress, clear consistent endpoints and standard approaches for the measurement of these. yet, even platform trials may not provide the sole answer because even very large platform trials will not be able to test every plausible permutation for mixed antiviral and anti-inflammatory interventions. in addition, platform trials come with their own weaknesses such as (frequently) a lack of blinding, and the fact that, in a pandemic situation, controls become historical as disease management improves, requiring more sophisticated analyses. ideally, platform trials must be preceded by well designed (placebo) controlled phase ii trials that will quickly read out and would be the ideal partner to platform trials to define the next intervention to add. however, this requires proper co-ordination between these phase ii and iii type trials. an additional dimension of complexity is the planning of study populations: while studies may be initiated and performed more rapidly when focused on well-defined homogenous populations, the overall development strategy must aim for representativeness and include the elderly, pregnant women and younger participants in their numbers. fortunately, there are attempts underway to address the issue of uncoordinated clinical research. platform trials have been established by a range of public and private institutions; they aim to bundle the clinical assessment of treatments across clinical scenarios, ranging from community-level prophylaxis to critical disease treatment in icu settings. for example, who has now taken steps to provide greater coordination through its solidarity trial though the need to establish sponsorship in each jurisdiction caused some hurdles. the "eravscorona" action plan, launched by the european commission, aims to "[extend and support] large eu wide clinical trials for clinical management of coronavirus patients" by providing rapid, dedicated funding and infrastructure . such initiatives are welcome but will only deliver if the clinical research community comes onboard. why has this not happened to a larger extent and what are the possible reasons behind the fragmented trial landscape, globally and in the eu? there remain substantial organisational and bureaucratic obstacles to rapid research coordination in all regions . in the eu specifically, and with few exceptions (e.g. imi funded combacte clinical trial network), there is a limited culture of large networks that go beyond national borders. as the crisis unfolded, the focus has been on using national resources for setting up this article is protected by copyright. all rights reserved protocols at national level, with limited ambition of expanding to a larger context despite the obvious benefits. moreover, our eudract database search shows a predominant role for non-commercial, largely academic clinical trial sponsors. this is not surprising; in the context of epidemics, academic groups, public health authorities, and funding bodies generally step up in their contribution to the crisis including design and conduction of clinical trials. historically, a high proportion of clinical trials run by academia or public health bodies are run in a single member state, often based around single institutions, resulting in many smaller trials and/or trials constrained by national boundaries from achieving larger, more rapid recruitment. this is seen also in the case of covid- clinical trials of antivirals. by contrast, vaccine development is largely driven by commercial sponsors. vaccines are still in early stages of development, but we expect them to be tested mostly in multi-member state trials. first, we reiterate our call on academic researchers and companies alike, to first consider if their planned trial or development plan could become part of a broader platform trial, as discussed above. second, we call on ethics committees to consider in their assessment of clinical trial protocols whether a given stand-alone trial for covid- can be assumed to meet the ethical requirement that "medical research involving human subjects may only be conducted if the importance of the objective outweighs the risks and burdens to the research subjects." . is this condition met by a small, (underpowered) standalone trial, perhaps with a high probability of failure due to lack of enrolment? or could the objective be better met by redirecting the energy to one of the larger ongoing trials? third, we highlight the importance for developers of covid- treatments, academic or industry alike, to seek interactions with drug regulators early on in their research. the goal of successful development of any repurposed or de novo treatment is to obtain regulatory authorization for its use. this is to provide reassurance to prescribers and patients of independent assessment of the data and of a favourable benefit-risk balance. our query of the eudract database (see above) shows that most covid- trials, at least in the eu, are proposed and run by academic group, ngos or public health authorities that are not always used to interact with regulators and may perceive regulatory authorization as less relevant compared to patient care. to strengthen the dialogue between such organizations and regulators, the european medicines agency (ema) has established the covid- ema pandemic task force (covid-etf) in charge of a dedicated this article is protected by copyright. all rights reserved pathway and procedures to enable covid- drug and vaccine developers to obtain regulatory input on their development plans and clinical trial protocols in a rapid, unbureaucratic fashion, no fees payable. we invite researchers in the field to avail themselves of this opportunity. fourth, we need to support and bring together the well-established public or private consortia such as the coalition for epidemic preparedness innovations (cepi), or the european clinical research infrastructure network (ecrin) , to ramp up their activities and take on a wider role in the management of trials. ema is currently liaising with such groups to rapidly explore how their activities could facilitate regulatory acceptability of trial results. in the course of formal or informal scientific advice interactions, ema can also guide drug developers towards existing trial collaborations. a key goal is to establish entities that can be the single lead sponsor for large platform trials. this might be one per trial or one overall for multiple trials. it might be a single entity, or an entity composed via joint sponsorship between several (national) research bodies. there is a framework for such joint sponsorship in ec guidance on clinical trials . fifth, infrastructure to support clinical trial conduct needs to be established. at a first level it could make a strong contribution by managing clinical trial applications to competent authorities and ethics committees, dealing with administrative challenges such as insurance/indemnity requirements, linking investigators and networks to clinical trials, and organisation of logistics of supplies to sites. it could go on to broader trial management/sponsor activities such as balancing trial participation across sites and protocols to improve recruitment, minimise interprotocol competition, matching protocol demands with site capabilities, running randomisation schemes, monitoring, data management and analysis and reporting of trials. organisations such as ecrin are well placed to carry out such activities if given the collective buy-in and resource needed. sixth, we ask umbrella patient organizations, like the european patients forum, and learned societies to bring to bear their considerable influence to support trial coordination efforts. last, regulatory flexibility needs to be exercised in the face of these challenges, in order to keep the process moving despite the practical and infrastructural difficulties that lockdown imposes to enable the rapid pace of development required. eu regulators, like those in north america and elsewhere, have issued guidance on adapting clinical trial and good clinical practice processes to the challenges of the pandemic environment with its social distancing and high demands on front line health care staff and facilities. this guidance enables clinical trials, and especially those for covid- treatments, but also other this article is protected by copyright. all rights reserved important therapies, to be carried out and to ensure their ethical conduct and scientific validity . the aim is to avoid the perfect becoming the enemy of the good and avoid attention being given to minor aspects that would cause unnecessary delays. covid- trials are an international concern and the best knowledge is that shared quickly and effectively at global level. to ensure international cooperation, workshops of regulators, hosted by the international coalition of medicines regulatory authorities (icmra), have been convening experts from dozens of medicines regulatory authorities worldwide and the world health organization (who). these workshops helped share expertise, streamline and standardise requirements, and focus on what is essential to the process of development and authorisation of vaccines and therapeutics . the needs and interest of lowand middle-income countries and their patients need to be incorporated and clinical trial designs to include some which can also be run in resource constrained environments. ideally, in a pandemic situation, we would hope to see a group taking an overall look at the entire clinical trial endeavour, to assess all trials and determine if a certain type of trial is needed or if redundant trials are being conducted. in the eu, there is no single group conducting such review, though the ema's covid-etf, the eu clinical trial facilitation group plan have a number of actions aiming to achieve this kind of high-level assessment. given the amount of human suffering, there is an ethical imperative for the scientific community to make full use of the learning opportunity provided by each successive pandemic or epidemic. with this pandemic, we were slow to apply the lessons from ebola (and other waves) to covid- trials. now is the time to ensure that the window of opportunity will not shut, both for patients in need of treatment and for researchers to conduct clinical trials that deliver. a call to pool eu research resources into large-scale, multi-centre, multi-arm clinical trials against covid- flooded by the torrent: the covid- drug pipeline plea for multitargeted interventions for severe covid- international clinical trials registry platform (ictrp) compassionate use of remdesivir for patients with severe covid- a trial of lopinavir-ritonavir in adults hospitalized with severe covid- factors associated with clinical trials that fail and opportunities for improving the likelihood of success: a review european commission (ec) website: first eravscorona action plan wma) website: wma declaration of helsinki -ethical principles for medical research involving human subjects eudralex vol -guidance documents applying to clinical trials questions & answers -paragraphs and guidance to sponsors on how to manage clinical trials during the covid- pandemic points to consider on implications of coronavirus disease (covid- ) on methodological aspects of ongoing clinical trials ema website: international regulators pledge collective support to combat covid- pandemic clinical trials facilitation and coordination group (ctfg) key: cord- -qdpkidn authors: ghazawi, feras m.; lim, megan; dutz, jan p.; kirchhof, mark g. title: infection risk of dermatologic therapeutics during the covid‐ pandemic: an evidence‐based recalibration date: - - journal: int j dermatol doi: . /ijd. sha: doc_id: cord_uid: qdpkidn recommendations were made recently to limit or stop the use of oral and systemic immunotherapies for skin diseases due to potential risks to the patients during the current severe acute respiratory syndrome coronavirus (sars‐cov‐ ) covid‐ pandemic. herein, we attempt to identify potentially safe immunotherapies that may be used in the treatment of cutaneous diseases during the current covid‐ pandemic. we performed a literature review to approximate the risk of sars‐cov‐ infection, including available data on the roles of relevant cytokines, cell subsets, and their mediators in eliciting an optimal immune response against respiratory viruses in murine gene deletion models and humans with congenital deficiencies were reviewed for viral infections risk and if possible coronaviruses specifically. furthermore, reported risk of infections of biologic and non‐biologic therapeutics for skin diseases from clinical trials and drug data registries were evaluated. many of the immunotherapies used in dermatology have data to support their safe use during the covid‐ pandemic including the biologics that target ige, il‐ / , tnf‐α, il‐ , il‐ , and il‐ . furthermore, we provide evidence to show that oral immunosuppressive medications such as methotrexate and cyclosporine do not significantly increase the risk to patients. most biologic and conventional immunotherapies, based on doses and indications in dermatology, do not appear to increase risk of viral susceptibility and are most likely safe for use during the covid‐ pandemic. the limitation of this study is availability of data on covid‐ . the severe acute respiratory syndrome coronavirus (sars-cov- ), also named novel coronavirus disease covid- , is the causative agent of the ongoing pandemic. and mip- a. , this is consistent with the reported elevation of proinflammatory cytokines in sars and mers infections. the massive inflammatory cell infiltration and elevated proinflammatory cytokine/chemokine responses result in acute lung injury and ards. , , part : infectious risks associated with biologics: evaluating cytokine knockout data and reviewing data from randomized controlled trials (rcts) and biologic treatment registries infecting tnf-a À/À , tnf receptor (r ) À/À , and tnfr À/À mice with mouse hepatitis virus- (mhv- , belongs to the coronavirus family) revealed that a deficiency of either tnf-a or tnfr decreased morbidity and mortality (table ) . tnf receptors / knock-out mice infected with sars-cov were protected from infection-related morbidity. collectively, tnf-a promotes the deleterious effects of coronavirus infection presumably through excessive inflammation. from clinical trials ( the b-lymphocyte antigen cd is highly expressed on b cells starting at the pre-b-cell stage and on mature b cells, and it is downregulated during terminal differentiation into plasma cells. while the precise function of cd is not fully elucidated, igm expression in immature and mature b cells from cd -deficient mice was markedly reduced compared to wildtype. furthermore, reduced humoral immunity to adeno-associated viral antigens was demonstrated in cd -deficient mice. a patient who lacked cd expression due to homozygous mutations reported intermittent respiratory infections, associated with persistent hypogammaglobulinemia and strong reductions in circulating memory b cells. no significant differences in urti, nasopharyngitis, bronchitis, cough, and sinusitis between rituximab (anti-cd ) and placebo were demonstrated in a double-blind rct for rheumatoid arthritis (ra). however, in a prospective, open-label rct, it was noted that lung infections/ pneumonia were higher in the rituximab treatment arm by more than twofold ( % vs. % in control, no confidence intervals were presented). the role of cd + cells in presenting antigen to t cells and in generation of antibodies to protect from new infections remains unclear. the il- /il- common pathway plays a key role in the induction of inflammation in adaptive immune responses, where il- induces a th immune response with a downstream induction of cytokines such as tnf, interferon (ifn)-c, and il- promotes a th immune response through the induction of inflammatory cytokines such as il- and il- . mice defective in both il- / (p À/À ) and il- alone (p À/À ) were infected with a murine coronavirus (mhv). il- and il- / knockout mice had similar survival to wild-type animals. therefore, il- does not seem to contribute to antiviral function or survival. mice deficient in il- alone (p À/À ) were infected with murine coronavirus, and viral control was similar to wild-type mice, demonstrating that il- does not significantly confer protection from infection. this was also demonstrated thorough neutralization of mice using anti-il- p specific and anti-il- / p antibodies, followed by infection of mice with mhv. in the absence of il- / signaling, specific antiviral t-cell response was intact. clinical trials using il- / or il- inhibitors demonstrated no significant increase in respiratory adverse events (table ) . furthermore, the psolar study reported that ustekinumab had no increased risk of serious infections. of note, a recent case study reported covid- in a patient during il- inhibitor (guselkumab) treatment for psoriasis, and the patient had a good outcome. il- is a proinflammatory cytokine with important roles in tcell activation and neutrophil mobilization and activation. il- expression is induced during influenza infection as part of the th immune response that contributes to viral clearance. however, a growing body of evidence suggests that il- is also associated with promotion of viral infections and tissue pathology. this is thought to occur through direct suppression including tnf-a, il- b, and il- . in humans, chronic mucocutaneous candidiasis has been attributed to the disruption of th and th pathways. this was illustrated in patients with identified mutations in il- ra and stat genes. these patients have no increased risk of viral infections. clinical trials using il- inhibitors demonstrated no significant increase in respiratory adverse events (table ) . a recent case report reported a patient receiving therapy with an il- inhibitor (ixekizumab) who was completely asymptomatic but tested positive for covid- . ity. the use of anakinra in clinical trials was associated with a slightly higher frequency of serious infectious episodes, primarily pneumonia ( . % vs. . %, comparative risk . ), than the placebo group. it appears that normal il- expression/ function is required to mount an optimal antiviral immune response. il- is a key regulator in humoral and th adaptive immunity. mouse models demonstrated that the constitutive overexpression of il- prior to rsv infection delayed viral clearance, increased the density of the lymphocytic infiltrate in the lungs, and diminished induction of primary cytotoxic t lymphocyte responses. conversely, il- À/À mice cleared rsv readily after primary infection, with minimal pathology. cyclosporine is a calcineurin inhibitor that blocks il- signaling and t-cell proliferation. , the most common infectious side effects from cyclosporine were flu-like symptoms seen in % of patients enrolled in an rct for chronic idiopathic urticaria. psoriasis registries examining cyclosporine reported infection trials that combined mmf with corticosteroids had significantly higher rates of infections, up to %. mmf is reported to increase patients' susceptibility to viral infections, and an increase in nasopharyngitis and urtis was noted comparing prednisone plus mmf to prednisone monotherapy in pemphigus vulgaris. of note, mmf was used to treat eight patients with mers with a % survival rate; however, when analyzing the severity of illness and treatment, mmf was given to less severely ill patients. azathioprine azathioprine inhibits purine synthesis and downregulates b-cell and t-cell function. , documented types of infection with use of azathioprine include lower respiratory tract infections (lrti) and urti, which had rates of % and - %, respectively. , thirty-six percent of patients in one study had infections of moderate intensity. there were no registries evaluating the prevalence of infections during azathioprine therapy for dermatologic uses. one systematic review evaluating the off-label use of azathioprine found mild infections reported in . % of patients and severe infections in only . % of patients (table ). the use of methotrexate (mtx), a folic acid antagonist that inhibits nucleotide synthesis, infection were similar to the placebo group. a review of infectious risks in rheumatoid arthritis (ra) patients indicated that although mtx has previously been implicated not only with increased risk of infection but also increased severity, the evidence was not clear. the review concluded that mtx appears to be associated with minimal, if any, increased infection risk in the ra population. hydroxychloroquine is an antimalarial medication that inhibits lysosomal functions and interferes with a myriad of immune pathways. its exact mechanism in many dermatologic processes has never been fully elucidated. hydroxychloroquine has been shown to have a favorable side effect profile in terms of infection risk in many clinical trials. , it is currently under investigation in numerous phase clinical trials as treatment for covid- as it may inhibit viral fusion to the host cell and inhibit viral assembly and release. apremilast is a phosphodiesterase (pde ) inhibitor, with side effects including nasopharyngitis and urti. the incidence of urti in the apremilast-treated groups is comparable to placebo ranging from . to . % and . to . %, with higher rates being accounted for from one study examining apremilast in palmoplantar psoriasis (table ). overall, rates of infection were not increased in patients treated with apremilast. [ ] [ ] [ ] [ ] [ ] [ ] a recent case was reported of a patient with erythrodermic psoriasis, with contraindication to most treatments due to a recurrent brain oligodendroglioma who had psoriasis thalidomide thalidomide, an immunomodulatory drug with a range of activity that is not fully characterized, is effective for various refractory dermatoses, but its side effect profile is unfavorable, and risks of teratogenicity and neuropathy often preclude its use. table highlights four rcts where there was no increased risk of infection in thalidomide compared to placebo. prolonged use of oral corticosteroids is generally avoided due to side effects. none of the following studies reported infection as an adverse reaction. immunosuppression is not a comorbidity that is commonly reported in covid- patients despite it commonly being referred to as a risk factor. the limited data do not suggest increased risk of severe complications compared to the general population. lei et al. reported two heart transplant patients in china who survived covid- infections. two reported renal transplant patients who contracted covid- and succumbed to the illness had similar clinical courses compared to non-transplant patients. transplant recipients may practice more stringent physical distancing practices compared to the general population, resulting in falsely low numbers. the literature surrounding sars and transplant recipients is sparse. risk factors for severe sars included hypertension, diabetes, coronary heart disease, hepatitis, and pregnancy with a mortality rate with ≥ risk factor compared to none of . % vs. . %; p < . . there is no evidence that suggests transplant recipients had poorer outcome in the sars epidemic. a retrospective cohort study of a mers outbreak in korea revealed that the number of affected immunosuppressed patients was low and did not identify any transplant patients. immunosuppression was not identified as a poor prognostic factor in mers infection. immunomodulatory regimens have revolutionized the treatment of dermatological diseases. with the current covid- pandemic, it is imperative to examine the evidence and conduct a risk-benefit analysis for each patient. there may be patients who require more or less treatment, for instance some patients with existing comorbidities may require a more conservative figure a pictorial representation of covid- risk assessment of dermatologic treatments where green represents "safe" and red represents "higher risk" approach. the greatest risk of infections in biologics appear to occur with cd inhibition (fig. ) . for non-biologic immunotherapies, the greatest risk of infection appears to occur with the use of high doses of oral corticosteroids. a slight increased infection risk is seen with cyclosporine, although cyclosporine has been shown to inhibit coronavirus replication and did not increase susceptibility in transplant patients. severe acute respiratory syndrome coronavirus (sars-cov- ) and coronavirus disease- (covid- ): the epidemic and the challenges covid- and psoriasis: is it time to limit treatment with immunosuppressants? a call for action should biologics for psoriasis be interrupted in the era of covid- ? epidemiological and clinical characteristics of cases of novel coronavirus pneumonia in wuhan, china: a descriptive study dysregulation of immune response in patients with covid- in wuhan, china clinical features of patients infected with novel coronavirus in wuhan, china plasma inflammatory cytokines and chemokines in severe acute respiratory syndrome mers-cov infection in humans is associated with a pro-inflammatory th and th cytokine profile clinical characteristics of novel coronavirus cases in tertiary hospitals in hubei province covid- infection: the perspectives on immune responses tumor necrosis factor alpha (tnfalpha) receptor-i is required for tnf-alpha-mediated fulminant virus hepatitis caused by murine hepatitis virus strain- infection the effect of inhibition of pp and tnfalpha signaling on pathogenesis of sars coronavirus risk of serious infection with biologic and systemic treatment of psoriasis: results from the psoriasis longitudinal assessment and registry (psolar) inhibition of tumor necrosis factor reduces the severity of virus-specific lung immunopathology a randomized, open-label, controlled trial for the efficacy and safety of adalimumab injection in the treatment of patients with severe novel coronavirus pneumonia (covid- ) mouse cd expression and function reduced t-dependent humoral immunity in cd -deficient mice cd deficiency in humans results in impaired t cell-independent antibody responses rituximab: uses in dermatology rituximab for rheumatoid arthritis refractory to anti-tumor necrosis factor therapy: results of a multicenter, randomized, double-blind, placebo-controlled, phase iii trial evaluating primary efficacy and safety at twenty-four weeks first-line rituximab combined with short-term prednisone versus prednisone alone for the treatment of pemphigus (ritux ): a prospective, multicentre, parallel-group, open-label randomised trial il- and il- cytokines: from discovery to targeted therapies for immunemediated inflammatory diseases mice lacking il- develop polarized th cells during viral infection interleukin- (il- ), but not il- , deficiency ameliorates viral encephalitis without affecting viral control generation of a protective t-cell response following coronavirus infection of the central nervous system is not dependent on il- / signaling sars-cov- infection in a psoriatic patient treated with il- inhibitor immunity to infection in il- -deficient mice and humans the protective and pathogenic roles of il- in viral infections: friend or foe? il- -induced pulmonary pathogenesis during respiratory viral infection and exacerbation of allergic disease th cells contribute to viral replication in coxsackievirus b -induced acute viral myocarditis deficiency of autophagy protein map -lc b mediates il- -dependent lung international journal of dermatology ª the international society of dermatology pathology during respiratory viral infection via er stressassociated il- critical role of il- ra in immunopathology of influenza infection chronic mucocutaneous candidiasis in humans with inborn errors of interleukin- immunity stat mutations in autosomal dominant chronic mucocutaneous candidiasis sars-cov- infection in a psoriatic patient treated with il- inhibitor myd is required for protection from lethal infection with a mouseadapted sars-cov the nlrp inflammasome and il- beta accelerate immunologically mediated pathology in experimental viral fulminant hepatitis an autoinflammatory disease with deficiency of the interleukin- -receptor antagonist anakinra, a recombinant human interleukin- receptor antagonist (r-methuil- ra), in patients with rheumatoid arthritis: a large, international, multicenter, placebo-controlled trial overexpression of interleukin- delays virus clearance in mice infected with respiratory syncytial virus efficacy and safety of dupilumab monotherapy in adults with moderate-to-severe atopic dermatitis: a pooled analysis of two phase randomized trials (liberty ad solo and liberty ad solo ) safety of dupilumab in severe atopic dermatitis and infection of covid- : two case reports no evidence of increased risk for covid- infection in patients treated with dupilumab for atopic dermatitis in a high-epidemic area belloni fortina a. sars-cov- asymptomatic infection in a patient under treatment with dupilumab covid- infection in a patient with severe chronic rhinosinusitis with nasal polyps during therapy with dupilumab omalizumab for treating chronic spontaneous urticaria: an expert review on efficacy and safety omalizumab for the treatment of chronic idiopathic or spontaneous urticaria similar efficacy with omalizumab in chronic idiopathic/spontaneous urticaria despite different background therapy subcutaneous immunoglobulin for maintenance treatment in chronic inflammatory demyelinating polyneuropathy (path): a randomised, double-blind, placebo-controlled, phase trial a randomized doubleblind trial of intravenous immunoglobulin for bullous pemphigoid a randomized doubleblind trial of intravenous immunoglobulin for pemphigus inhibition of neutrophil responses by cyclosporin a. an insight into molecular mechanisms product monograph randomized double-blind study of cyclosporin in chronic 'idiopathic' urticaria long term safety of nine systemic medications for psoriasis: a cohort study using the biobadaderm registry drug safety of systemic treatments for psoriasis: results from the german psoriasis registry psobest infections in moderate to severe psoriasis patients treated with biological drugs compared to classic systemic drugs: findings from the biobadaderm registry suppression of coronavirus replication by cyclophilin inhibitors cyclosporin a inhibits the replication of diverse coronaviruses preferential suppression of lymphocyte proliferation by mycophenolic acid and predicted long-term effects of mycophenolate mofetil in transplantation mycophenolate mofetil treating pemphigus vulgaris with prednisone and mycophenolate mofetil: a multicenter, randomized, placebo-controlled trial risk of herpes zoster infection in patients with pemphigus on mycophenolate mofetil treatment outcomes for patients with middle eastern respiratory syndrome coronavirus (mers cov) infection at a coronavirus referral center in the kingdom of saudi arabia cd -dependent rac activation is the molecular target of azathioprine in primary human cd + t lymphocytes product monograph azathioprine in severe adult atopic dermatitis: a double-blind, placebo-controlled, crossover trial azathioprine dosed by thiopurine methyltransferase activity for moderate-to-severe atopic eczema: a double-blind, randomised controlled trial a randomized trial of methotrexate versus azathioprine for severe atopic eczema off-label use of azathioprine in dermatology: a systematic review product monograph molecular basis for pharmacokinetics and pharmacodynamics of methotrexate in rheumatoid arthritis therapy an intensified dosing schedule of subcutaneous methotrexate in patients with moderate to severe plaque-type psoriasis (metop): a week, multicentre, randomised, double-blind, placebocontrolled, phase trial a randomized controlled trial of methotrexate for patients with generalized myasthenia gravis a randomized placebo-controlled trial of methotrexate in psoriatic arthritis low-dose methotrexate for the prevention of atherosclerotic events rheumatoid arthritis and infection risk: what is the evidence? hydroxychloroquine in systemic lupus erythematosus (sle) effects of hydroxychloroquine in patients with cutaneous lupus erythematosus: a multicenter, double-blind, randomized, parallel-group trial hydroxychloroquine in systemic lupus erythematosus: results of a french multicentre controlled trial (plus study) in vitro antiviral activity and projection of optimized dosing design of hydroxychloroquine for the treatment of severe acute respiratory syndrome coronavirus (sars-cov- ) apremilast in psoriasis and beyond: big hopes on a small molecule otezla â (apremilast) efficacy and safety of apremilast in systemic-and biologic-naive patients with moderate plaque psoriasis: -week results of unveil safety and efficacy of apremilast through weeks in patients with moderate to severe psoriasis who continued on apremilast or switched from etanercept treatment: findings from the liberate study apremilast for the treatment of moderate-to-severe palmoplantar psoriasis: results from a double-blind, placebo-controlled, randomized study apremilast, an oral phosphodiesterase (pde ) inhibitor, in patients with moderate to severe plaque psoriasis: results of a phase iii, randomized, controlled trial (efficacy and safety trial evaluating the effects of apremilast in psoriasis [esteem] ) efficacy and safety of apremilast, an oral phosphodiesterase inhibitor, in patients with moderate-to-severe plaque psoriasis over weeks: a phase iii, randomized controlled trial (esteem ) apremilast for moderate hidradenitis suppurativa: results of a randomized controlled trial covid- pulmonary infection in erythrodermic psoriatic patient with oligodendroglioma: safety and compatibility of apremilast with critical intensive care management thalidomid â (thalidomide) binding of thalidomide to alpha -acid glycoprotein may be involved in its inhibition of tumor necrosis factor alpha production product monograph the role of systemic steroids and phototherapy in the treatment of stable vitiligo: a randomized controlled trial placebo-controlled oral pulse prednisolone therapy in alopecia areata a comparison of the efficacy, relapse rate and side effects among three modalities of systemic corticosteroid therapy for alopecia areata a randomized comparative study of oral corticosteroid minipulse and lowdose oral methotrexate in the treatment of unstable vitiligo adverse events of low-to medium-dose oral glucocorticoids in inflammatory diseases: a meta-analysis the international society of dermatology risk of infectious complications in patients taking glucocorticosteroids but not tnf antagonists, are associated with adverse covid- outcomes in patients with inflammatory bowel diseases: results from an international registry recovery trial: the uk covid- study resetting expectations for clinical trials the natural history of influenza infection in the severely immunocompromised vs nonimmunocompromised hosts coronaviruses and immunosuppressed patients. the facts during the third epidemic prevalence of comorbidities in the novel wuhan coronavirus (covid- ) infection: a systematic review and meta-analysis first cases of covid- from china covid- in kidney transplant recipients prognostic factors for severe acute respiratory syndrome: a clinical analysis of cases host susceptibility to mers-cov infection, a retrospective cohort study of the korean mers outbreak middle east respiratory syndrome coronavirus: risk factors and determinants of primary, household, and nosocomial transmission available at interleukin- is responsible for acute lung immunopathology but increases survival of respiratory influenza virus infection adalimumab therapy for moderate to severe psoriasis: a randomized, controlled phase iii trial certolizumab pegol for the treatment of chronic plaque psoriasis: results through weeks from phase , multicenter, randomized, doubleblinded secukinumab in plaque psoriasis-results of two phase trials a global phase iii randomized controlled trial of etanercept in psoriasis: safety, efficacy, and effect of dose reduction infliximab induction and maintenance therapy for moderate-to-severe psoriasis: a phase iii, multicentre, double-blind trial efficacy and safety of ustekinumab, a human interleukin- / monoclonal antibody, in patients with psoriasis: -week results from a randomised, double-blind, placebo-controlled trial (phoenix ) phase studies comparing brodalumab with ustekinumab in psoriasis comparison of ixekizumab with etanercept or placebo in moderate-to-severe psoriasis (uncover- and uncover- ): results from two phase randomised trials safety of guselkumab in patients with moderate-to-severe psoriasis treated through weeks: a pooled analysis from the randomized voyage and voyage studies efficacy and safety of guselkumab, an anti-interleukin- monoclonal antibody, compared with adalimumab for the treatment of patients with moderate to severe psoriasis with randomized withdrawal and retreatment: results from the phase iii, double-blind, placeboand active comparator-controlled voyage trial efficacy and safety of risankizumab in moderate-to-severe plaque psoriasis (ultimma- and ultimma- ): results from two double-blind, randomised, placebo-controlled and ustekinumab-controlled phase trials safety of tildrakizumab for moderate-to-severe plaque psoriasis: pooled analysis of three randomized controlled trials a multicentre, double blind, randomised, placebo controlled trial of anakinra (kineret), a recombinant interleukin receptor antagonist, in patients with rheumatoid arthritis treated with background methotrexate cyclosporine in chronic idiopathic urticaria: a double-blind, randomized, placebocontrolled trial adalimumab or cyclosporine as monotherapy and in combination in severe psoriatic arthritis: results from a prospective -month nonrandomized unblinded clinical trial cyclosporine for moderate-tosevere alopecia areata: a double-blind, randomized, placebocontrolled clinical trial of efficacy and safety a comparison of oral methylprednisolone plus azathioprine or mycophenolate mofetil for the treatment of bullous pemphigoid efficacy and safety of mycophenolate mofetil vs. methotrexate for the treatment of chronic plaque psoriasis evaluation of mycophenolate mofetil as a steroid-sparing agent in pemphigus: a randomized, prospective study mycophenolate mofetil (cellcept) for psoriasis: a two-center, prospective, open-label clinical trial a randomized, investigator-masked, double-blind, placebo-controlled trial on thalidomide in severe cutaneous sarcoidosis comparative efficacy of thalidomide and prednisolone in the treatment of moderate to severe erythema nodosum leprosum: a randomized study effect of thalidomide on clinical remission in children and adolescents with refractory crohn disease: a randomized clinical trial thalidomide in the treatment of the mucocutaneous lesions of the behcet syndrome. a randomized, double-blind, placebo-controlled trial cyclosporin in the treatment of patients with atopic eczema -a systematic review and meta-analysis long-term efficacy and safety of azathioprine in ulcerative colitis effectiveness of methotrexate in moderate to severe psoriasis patients: realworld registry data from the swiss dermatology network for targeted therapies (sdntt) real-world data on the efficacy and safety of apremilast in patients with moderate-to-severe plaque psoriasis risk of serious infections, cutaneous bacterial infections, and granulomatous infections in patients with psoriasis treated with anti-tumor necrosis factor agents versus classic therapies: prospective meta-analysis of psonet registries the international society of dermatology the authors declare no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. jpd is a senior scientist at bc children's hospital research institute. key: cord- -rc h drd authors: li, xuanyi; sigworth, elizabeth a.; wu, adrianne h.; behrens, jess; etemad, shervin a.; nagpal, seema; go, ronald s.; wuichet, kristin; chen, eddy j.; rubinstein, samuel m.; venepalli, neeta k.; tillman, benjamin f.; cowan, andrew j.; schoen, martin w.; malty, andrew; greer, john p.; fernandes, hermina d.; seifter, ari; chen, qingxia; chowdhery, rozina a.; mohan, sanjay r.; dewdney, summer b.; osterman, travis; ambinder, edward p.; buchbinder, elizabeth i.; schwartz, candice; abraham, ivy; rioth, matthew j.; singh, naina; sharma, sanjai; gibson, michael k.; yang, peter c.; warner, jeremy l. title: seven decades of chemotherapy clinical trials: a pan-cancer social network analysis date: - - journal: sci rep doi: . /s - - - sha: doc_id: cord_uid: rc h drd clinical trials establish the standard of cancer care, yet the evolution and characteristics of the social dynamics between the people conducting this work remain understudied. we performed a social network analysis of authors publishing chemotherapy-based prospective trials from to to understand how social influences, including the role of gender, have influenced the growth and development of this network, which has expanded exponentially from fewer than authors in to , in . while . % of authors were directly or indirectly connected by , our results indicate a tendency to predominantly connect with others in the same or similar fields, as well as an increasing disparity in author impact and number of connections. scale-free effects were evident, with small numbers of individuals having disproportionate impact. women were under-represented and likelier to have lower impact, shorter productive periods (p < . for both comparisons), less centrality, and a greater proportion of co-authors in their same subspecialty. the past years were characterized by a trend towards increased authorship by women, with new author parity anticipated in . the network of cancer clinical trialists is best characterized as strategic or mixed-motive, with cooperative and competitive elements influencing its appearance. network effects such as low centrality, which may limit access to high-profile individuals, likely contribute to the observed disparities. the modern era of chemotherapy began in , with publications describing therapeutic uses of nitrogen mustard , . over the next years, the repertoire of available cancer treatments has expanded at an ever-increasing pace. chemotherapeutics have a notably low therapeutic index, i.e., the difference between a harmful and beneficial dose or combination is often quite small . consequently, a complex international clinical trial apparatus emerged in the s to study chemotherapeutics in controlled settings, and prospective clinical trials remain the gold standard by which standard of care treatments are established , . discoveries made by successive generations have led to overall improvement in the prognosis of most cancers . while social network analysis has been used to study patterns of co-authorship in scientific settings , , the social component of clinical trial research is not well characterized. little is known about how social factors have shaped the progress of the field, as cancer care has become increasingly subspecialized, and how social network baseline characteristics. n = of reviewed publications with an aggregate of n = , authors met the inclusion criteria (consort figure s ). cumulatively, most authors in the network (n = , , %) published at least one randomized trial, with n = , ( . %) participating in the publication of a "positive" trial (table s ). most of the included authors (n = , , . %) participated in the primary publication of a clinical trial, while a smaller subgroup (n = , , . %) participated in the publication of updates. the most common venues for publication were high-impact clinical journals: the journal of clinical oncology (n = , . %), the lancet family (n = , . %), the new england journal of medicine (n = , . %), and the blood family (n = , . %). co-authorship has changed in a non-linear fashion over time: the median number of authors per publication increased from n = in to n = (iqr [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] in ( figure s ). across subspecialties, the median number of co-authors per publication varied somewhat, from a low of n = (iqr - ) in gynecologic oncology to a high of n = (iqr [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] in dermatologic oncology. median longevity is < year at all times, although the number of authors with multiple years in the field grows substantially over time ( figure s ). a small number of individuals maintained the highest impact over time-nearly years each in the case of chemotherapy pioneers sidney farber and james f. holland ( figure s ). in any given year, most authors had a betweenness centrality of < % of the maximum; conversely, a very small number of authors had an exceptionally high score, with % of authors accounting for % of the total in recent years ( figure s ). accordingly, an increasingly smaller proportion of authors were both very highly connected and highly impactful; in , the % highest-impact authors (n = ) account for . % of links and . % of impact; in , the same proportion (n = ) account for . % of links and . % of impact. first/last authorship has also become concentrated; in publications, % of authors had at least one such role, whereas prior to it was on average > % ( figure s ). the structure of the network changes considerably over time, from relatively dense and connected to sparse and modular (fig. b) . the final network is very sparse ( . % of possible links are present); nevertheless, n = , ( . %) authors are in a single connected component; the next-largest component comprises authors. each of the cancer subspecialties developed at different rates, with clear influence of seminal events in several subspecialties, e.g., the introduction of adjuvant therapy and tamoxifen for breast cancer, completely new classes of drugs for plasma cell disorders, and immunotherapy for melanoma (fig. c) - . network visualization and cumulative metrics. the final cumulative network visualization is shown in figs. & s . the impact score of authors is unevenly distributed, median . (range - . ); however, the log-transformed impact scores approximate a normal distribution ( figure s ). authors with longevity ≥ year who changed primary subspecialty at least once (n = ) had nearly twice the median impact and longevity of those who remained in one subspecialty (n = , ), . (iqr . - . ) versus . (iqr . - . ) and years (iqr - ) versus years (iqr - ), respectively (p < . for both comparisons). cumulatively, subspecialized authors with calculable homophily (n = , ) have a median proportion of co-authors sharing the same subspecialty of % (iqr - %); , ( . %) of these authors' outlinks are within-subspecialty. this is reflected by a high assortativity by subspecialty since the mid- s (fig. b) modularity follows a sigmoid pattern with a period of linear increase between - followed by a plateau at high modularity; assortativity rapidly increases in early decades; median normalized pagerank decreases to a low plateau from the s onward; (c) subspecialties develop at different but broadly parallel rates, with seminal events apparently preceding accelerations of individual subspecialties, e.g.,: ( ) in the four years after , combination therapy (ac ), adjuvant therapy , and tamoxifen were introduced in breast cancer; ( ) thalidomide and bortezomib were reported to be efficacious for multiple myeloma; and ( ) immunotherapy (ipilimumab , ) was introduced in the treatment of melanoma. www.nature.com/scientificreports/ sensitivity analysis. normalized score distributions did not change significantly, although modulation of the trial design coefficient led to a bimodal peak ( figure s ). correlation of assortativity and modularity was high, ranging from . - . for the former and . - . for the latter (table s ; figure s ). the remarkable gains in the fields of hematology and oncology can be ascribed to the tireless work of numerous trialists and the generosity of countless patient participants. as a result, systemic antineoplastics now stand beside surgery and radiotherapy as a pillar of cancer care. our analysis of clinical trialists as a social network, particularly with respect to the density distribution of pagerank, reveals a mixed-motive network that differs only authors assigned to a subspecialty are visualized; these account for % of all authors in the database. this figure highlights various clustering trends by subspecialty, such as the apparent sub-clusters of sarcoma research (yellow) and the two dominant clusters of breast cancer research (pink). it is clear as well that certain subspecialties are more cohesive than others, such as the tightly clustered dermatology (black) compared to the spread-out head and neck cancer authors (red). www.nature.com/scientificreports/ substantially from "collegial" and "friend-based" online social networks. while clinical trials are conducted towards a collaborative goal-improved outcomes for all cancer patients-there are significant competitive pressures. examples of these pressures include resource limitations (e.g., funding and patients available for accrual), the tension between prioritization of cooperative group versus industry-funded trials, personal motivations such as academic promotion or leadership opportunities, and institutional reputation. the emergence of formal and informal leaders in scientific networks has been shown to facilitate research as well as create clusters . as fig. shows, there is a strong tendency for clustering based on subspecialty in the complete network, although some subspecialties (e.g., lymphoid and myeloid malignancies) have many more interconnections than others (e.g., sarcoma and neuro-oncology). many of these clusters appear to be organized around an individual or group of individuals who have high impact and centrality. as an organizational principle, these individuals appear to rarely be in direct competition, but their presence is a clear indicator of scale-free phenomena within the network. the facts that betweenness centrality follows a power law cumulative distribution bolsters this theory. scale-free phenomena, which are defined by a power law distribution of connectedness, are very common in strategic networks, especially when they become increasingly sparse, as this network does . the two related theories for this network behavior are preferential attachment and fitness. the former observes that those with impact tend to attract more impact; the latter postulates that such gains for the "fittest" come at the expense of the "less fit" . seminal events (fig. c) are likely a driver of preferential attachment , and may the network is overwhelmingly dominated by men until , when a trend towards increasing authorship by women begins to be seen; however, representation by women in first/last authorship remains low; gray shaded lines are % confidence intervals of the loess curves; (b) men tend on average to have a longer productive period and to achieve a higher author impact score than women (p < . for both comparisons); (c) men tend on average to be more central and have more collaborations outside of their subspecialty. note that the homophily calculation requires a subspecialty assignment, which explains the slightly lower numbers in (c) as compared to (b). www.nature.com/scientificreports/ partially explain why some authors change their primary subspecialty at least once over time (e.g., through a "bandwagon" effect driven by the diffusion of ideas ). given that these authors were observed to have nearly twice the impact and longevity of their single subspecialty peers, this dynamic will be a focus of future study, including calculation of the q factor, a metric developed to quantify the ability of a scientist to "take advantage of the available knowledge in a way that enhances (q > ) or diminishes (q < ) the potential impact p of a paper" . in the analysis of network dynamics (fig. b) , the field as a whole appears to emerge in the s, which is also when medical oncology and hematology were formally recognized through board certification. measurements of field maturity are by their nature subjective, but the pessimism of the late s was captured by sidney farber: "…the anticancer chemicals, hormones, and the antibiotics…marked the beginning of the era of chemotherapy of cancer, which may be described after years as disappointing because progress has not been more rapid…" . these concerns prompted the us national cancer act of , which was followed by the leveling of modularity at a very high level from onwards, suggesting that the subspecialties generated in the s have remained stable. the assortativity by subspecialty has increased as well, with recent levels approximately twice those seen in a co-authorship network of physicists . while median pagerank has decreased markedly, indicating decreasing influence for the average author, the distribution in is broadly right-skewed ( figure s ). these findings reveal a high level of increasing exclusivity, suggesting that it is becoming progressively more difficult to join the top echelon of the network. this has major implications for junior investigators' mobility, and potentially for the continued health of the network as a whole. while there is much to be applauded in the continued success of translating research findings into the clinic, we observed clear gender disparities within the cancer clinical trialist network: women have a statistically significantly lower final impact score, shorter productive period, less centrality, and less collaboration with those outside of their primary subspecialty. these findings are consistent with and build upon previous literature on www.nature.com/scientificreports/ the challenges facing women in pursuing and remaining in academic careers , , , . they are also consistent with more recent gender disparity findings, such as those observed in research published on covid- . other studies investigating the basis for such a gender gap have identified several layers of barriers to the advancement of women in academic medicine. these include sexism in the academic environment, lack of mentorship, and inequity with regards to resource allocation, salary, space, and positions of influence , . our study suggests that additional network factors such as relatively low centrality, which indicates a lack of access to other individuals of influence, and high homophily, which indicates a lack of access to new ideas and perspectives, also perpetuate the gender gap-corroborating recent findings from graduate school social networks . it is somewhat encouraging that there has been a steady increase in the proportion of authorship by women since (fig. a) . this increase is observed approximately a decade after the passage of title ix of the us civil rights act in . given that the majority of authors in this network are clinicians, a partial explanation could be that us-based women began to attend previously all-male medical schools in the early s, completed their training, and began to contribute to the network as authors approximately years later. if the nearly linear trend continues, we predict that gender parity for new authors entering the network will be reached by the year , years after us medical school enrollment approached parity . however, the proportion of first/last authors who are women is growing much more slowly, and parity may not be reached for + years, if at all. given that senior authorship is a traditional metric of scholarly productivity, it may be particularly difficult for clinical trialists who are women to obtain promotion under the current paradigm. one possible solution is to increase the role of joint senior authorship, which remains vanishingly rare in the clinical trials domain (furman et al. is one of very few examples that we are aware of)-although this is predicated on the acceptance of these roles by advancement and promotion committees. the field itself may also suffer from slow entry of new talent and a lack of broad perspectives. while the gender mapping algorithm and manual lookups are imperfect, our approach is consistent with prior work in this area , . unisex names posed a particular challenge . it should be noted that we could not account for all situations where an author changed their name (e.g., a person assumed their spouse's surname); this could have led to overestimation of representation by women and underestimation of impact, since this practice is more common with women. it is also possible that an individual's gender identity does not match the gender assignment of their given name. future work will include further analysis of gender disparities, factoring in institutional affiliation and highest degree(s) obtained, which are both likely to have significant influence on publication and senior authorship , . there are several additional limitations to this work, starting with the fact that co-authorship is but one way to measure social network interactions and this study reports results from published trials, which induces publication bias. although hemonc.org aims to be the most comprehensive resource of its kind, non-randomized trials and randomized phase ii trials are intentionally underrepresented, given that findings at this stage of investigation infrequently translate to practice-changing results (e.g., approximately % of oncology drugs fail during phase ii) [ ] [ ] [ ] . the effect of any biases introduced by this underrepresentation is unclear, given the confounding influence of publication bias, which may itself be subject to gender disparity . some older literature which no longer has practice-changing implications may have been overlooked. during name disambiguation, some names could not be resolved, primarily because neither medline nor the primary journal site contained full names. this effect is non-random, since certain journals do not publish full names. the choice of coefficients and their relative weights was based on clinical intuition and consensus; given that the "worth" of metrics such as first/last authorship is fundamentally qualitative, there must be some degree of subjectivity when formulating a quantitative algorithm. while the sensitivity analysis demonstrated that neither normalized author impact score distribution, assortativity, nor modularity are majorly changed by variation in the trial design and author role coefficients, it remains possible that other combinations of coefficients and relative weightings could lead to different results. furthermore, our impact algorithm weighs heavily on first and last authorship, but the definition of senior authorship has changed over time. for example, in the article by goodman et al. , the authors were listed in decreasing order of seniority (personal communication). in general, the impact score used in this paper, although similar to others proposed in the academic literature, is not validated and should be interpreted with caution. finally, the majority of authors in this database publish extensively, and their impact as measured here should not be misconstrued to reflect their contributions to the cancer field more broadly. in conclusion, we have described the first and most comprehensive social network analysis of the clinical trialists involved in chemotherapy trials. we found emergent properties of a strategic network and clear indications of gender disparities, albeit with improvement in representation in recent decades. the network has been highly modular and assortative for the past years, with little collaboration across most subspecialties. as the field pivots from an anatomy-based to a precision oncology paradigm, it remains to be seen how the network will re-organize so that the incredible progress seen to date can continue. - and referenced on hemonc.org were considered for inclusion. hemonc.org is the largest collaborative wiki of chemotherapy drugs and regimens and has a formal curation process . in order for a reference to be included on hemonc.org, it generally must include at least one regimen meeting the criteria outlined here: https ://hemon c.org/wiki/eligi bilit y_crite ria. as such, the majority of references on hemonc.org are randomized controlled trials (rcts) or non-randomized trials with at least participants and/or practice-changing implications. one of the main goals of hemonc.org is creating a database of all standard of care systemic antineoplastic therapy regimens. this is difficult as there is no universally accepted definition of standard of care except in a www.nature.com/scientificreports/ legal capacity. for example, the state of washington, in its legislation on medical negligence, inversely defines the standard of care as "exercis[ing] that degree of skill, care, and learning possessed at that time by other persons in the same profession". we currently employ four separate definitions that meet the threshold of standard of care: . the control arm of a phase iii randomized controlled trial (rct). by implication, this means that all phase iii rcts with a control arm must eventually be included on the website. . the experimental arm(s) of a phase iii rct that provide(s) reasonable evidence (p-value less than . ) of superior efficacy for an intermediate surrogate endpoint (e.g., pfs) or a strong endpoint (e.g., os). . a non-randomized study that is either: . any study (including case series and retrospective studies) that is specifically recommended by a member of the hemonc.org editorial board. all section editors of the editorial board with direct oversight of diseasespecific pages are board-eligible or board-certified physicians. in order to identify new regimens and study references for inclusion on hemonc.org, we undertake several parallel screening methods: as part of the process of building hemonc.org, we have also systematically reviewed all lancet, jama, and new england journal of medicine tables of contents from to december , . in addition, the citations of any included manuscript are hand-searched for additional citations. for any treatment regimen that has been subject to randomized comparison, we additionally seek to identify the first instance in which such a regimen was evaluated as an experimental arm; if no such determination can be made, we seek the earliest non-randomized description of the regimen for inclusion on the website. in order or prioritization, phase iii rcts are added first, then smaller rcts such as randomized phase ii, followed by non-randomized trials, followed by retrospective studies or case series identified by our editorial board as relevant to the practice of hematology/oncology. when a reference is added to hemonc.org, bibliographic information including authorship is recorded. the usually coincides with medline record details, although some older references in medline are capped at ten authors and are manually completed based upon the publication of record. for trials that do not list individual authors (e.g., the elderly lung cancer vinorelbine italian study group ), the original manuscript and appendices are examined for a writing committee. if a writing committee is identified, the members of this committee are listed as authors in the order that they appeared in the manuscript. if no writing committee is identified, the chairperson(s) of the study group are listed as the first & last authors. if no chairpersons are listed, the corresponding author is listed as the sole author. www.nature.com/scientificreports/ publications solely consisting of the evaluation of drugs not yet approved by the fda or other international approval bodies were not included. trials that appeared in abstract form only, reviews, retrospective studies, meta-analyses, and case reports were excluded, as were trials reporting only on local interventions such as surgery, radiation therapy, and intralesional therapy. non-antineoplastic trials (table s ) and trials of supportive interventions (e.g., antiemesis; growth factor support) were also excluded. disambiguation of author names. for each included publication, author names were extracted and disambiguated. author names on hemonc.org are stored in the medline lastname_firstinitial (middleinitial) format, which can lead to two forms of ambiguity: ( ) the short form, e.g., smith_j, can refer to two or more individuals, e.g., julian and jane smith; ( ) two short forms can refer to the same individual, e.g., kantarjian_h and kantarjian_hm. additionally, names can be misspelled and individuals can change their name over time (e.g., a person assumes their spouse's surname). we undertook several steps to disambiguate names: ( ) full first and middle names, when available, were programmatically accessed through the ncbi pubmed eutils application programming interface; ( ) when not available through medline, full first names were searched for on journal websites or through web search engines; ( ) automatic rules were developed to merge likely duplicates; and ( ) some names were manually merged (e.g., misspellings: benboubker_lofti and benboubker_lotfi; alternate forms: rigal-huguet_francoise and huguet-rigal_francoise; and subsumptions: baldotto_clarissa and serodio da rocha baldotto_clarissa). transformation algorithms are available upon request, and the full mapping table is provided in supplemental file . gender mapping. once the name disambiguation step was complete, we mapped authors with full name available to gender. we first mapped names to genders using us census data, which includes the relative frequencies of given names by gender in the population of us birth from to . we calculated the gender ratio for names that appeared as both genders. for names with gender ratio > . for one gender (e.g., john, rebecca), we assigned the name to that gender. to expand gender mapping to include names that are more frequently seen internationally (e.g., jean, andreas), we used a program that searches from a dictionary containing gender information about names from most european countries as well as some asian and middle eastern countries . for unmatched first names (e.g., dana, michele), we manually reviewed for potential gender assignment. for some names that are masculine in certain countries and feminine in others (e.g., andrea, daniele, and pascale are masculine in italy and feminine elsewhere), we mapped based on surnames. finally, we performed manual internet searches to look for photographs and pronouns used in online content such as faculty profiles, book biographies, and professional social media accounts for the remaining unmapped full names associated with a longevity of greater than one year. a total of , ( %) authors were assigned to the categories of woman (n = ; . %) or man (n = , ; . %). the gender of most of the people with unassigned names could not be determined because they only appeared with initials (n = ; . %) in the primary publication and medline. the remaining n = ( . %) were ambiguously gendered names that could not be resolved through manual searching, and were excluded in the gender-specific analyses. the full mapping table is provided in supplemental file . author impact score. we considered existing metrics for measuring author impact - , but ultimately proceeded with our own formulation given some of the unique considerations of prospective clinical trials and their impact. every author was assigned an impact score, using an algorithm calculated per manuscript using four coefficients: ( ) author role; ( ) trial type; ( ) citation score; ( ) primary versus updated analysis. the coefficients are multiplied to arrive at the score, and the total author impact score is summed across all of their published manuscripts. author role: first and last author roles are assigned a coefficient of three; middle authors are assigned a coefficient of one. when joint authorship is denoted in a medline record, there is an additional attribute "equalcontrib" that is set to "y" (yes). we look for this during the parsing process and treat these authors as first or last authors when the attribute is detected. trial type: any prospective trial with randomization is denoted as randomized and the authors of any manuscript reporting on such a trial are assigned a coefficient of two. non-randomized trials are assigned a coefficient of one. for manuscripts that reported on more than one trial with mixed designs (i.e., one or more randomized and one or more non-randomized trials), the randomized coefficient was used. citation score: we programmatically obtained a snapshot of citation counts from google scholar from september and used unadjusted total citations as the citation score coefficient for the years - . as more recent publications are still accruing citations, raw citation count is not an appropriate measure of their impact. therefore, we have calculated a blended citation score for articles published between - , adding the phased in median citation count for the journal tier in which the article was published for the years - (see tables s & s and figure s ). the citations scores are normalized to the manuscript with the maximum number of citations (stupp et al. , with , citations), such that the maximum citation score is one. primary publications vs. updates: the baseline coefficient is one. for updates, this score is multiplied by a half-life decay coefficient; i.e., scores for the first update are multiplied by %; scores for the second update by %; and so forth. this rule is applied equally to updates and subgroup analyses. for manuscripts that reported on pooled updates of more than one trial, the score was multiplied by the half-life coefficient corresponding to the update that resulted in the maximum score. see examples in supplemental methods. www.nature.com/scientificreports/ subspecialty designation of each publication. each publication was assigned to one of diseasespecific cancer subspecialties based on the cancer(s) studied (table s ). the majority of publications report on a clinical trial carried out in one disease or several diseases mapping to the same subspecialty. for publications studying diseases that map to more than one subspecialty, each author's impact score for that publication was divided evenly across the subspecialties. several clinical trials employ a site-agnostic approach, e.g., to a "cancer of unknown primary" or to biomarker-defined subsets of cancers (e.g., a basket trial ); for these, impact across subspecialties was split manually (table s ) . subspecialty designation based on authorship. authors were eligible for assignment to a primary subspecialty based on whether they were a first or last author at least once in the subspecialty, or whether they had a cumulative impact of at least one standard deviation below the mean of the author impact score of all authors in the subspecialty. authors who met either of these criteria were assigned to a primary subspecialty based on where the majority of their impact lay; if an author had equal impact in two or more subspecialties they were assigned equally to the subspecialties. this assignment was recalculated on an annual basis if the author had new publications, and primary subspecialty was re-assigned if a new subspecialty met either of the criteria and the impact in that subspecialty was higher than in the previous primary subspecialty. authors not meeting either of these criteria were assigned a primary subspecialty of "none" and were not included in the homophily analysis or the network visualization. social network construction and metrics. a dynamic social network was created with nodes representing authors and links representing co-authorship. the dynamic social network was discretized by year and the authors, scores, and links were cumulative (e.g., the th network was cumulative from - ). therefore, once an author is added to the network, they remain in the network, with their impact score cumulatively increasing as they publish and remaining constant if publication activity ceases. the following temporal metrics were calculated: ( ) network density (the number of actual connections/links present divided by the total number of potential connections); ( ) modularity by subspecialty (a measure of how strongly a network is divided into distinct communities, in this case subspecialties, defined as the number of edges that fall within a set of specified communities minus the number expected in a network with the same number of vertices and edges whose edges were placed randomly); ( ) assortativity by subspecialty (a measure of the preference of nodes in a network to attach to others that are similar in a defined way, in this case the same subspecialty; assortativity is positive if similar vertices tend to connect to each other, and negative if they tend to not connect to each other); ( ) betweenness centrality (a measure reflecting how important an author is in connecting other authors, calculated as the proportion of times that an author is a member of the bridge that forms the shortest path between any two other authors); ( ) pagerank (another measure of centrality, this time considering the connection patterns among each author's immediate neighbors; its value for each author is the probability that a person starting at any random author and randomly selecting links to other authors will arrive at the author); and ( ) proportion of co-authors sharing either the same primary subspecialty designation or the same gender (hereafter referred to as homophily). network density, modularity, and assortativity are calculated at the network level, while betweenness centrality, pagerank, and homophily are calculated at the author (node) level. further definitions of these metrics are provided in the supplemental glossary. all metrics incorporated the weighted co-authorship score, which takes into account each co-author's impact modified by the number of authors of an individual publication. for each pairwise collaboration, as defined by co-authorship on the same manuscript, a co-authorship score was calculated and used as the edge weight; duplicated edges were allowed to reflect the fact that weights could be distributed in a non-even fashion (e.g., two co-authors could be middle authors on a lower-impact publication as well as senior authors on a separate high-impact publication). this score was first calculated by multiplying the individual authors' manuscriptspecific impact scores together. in order to acknowledge the role of middle authors in large multi-institutional studies, this preliminary score was divided by the total number of authors on the manuscript. this has the effect of decreasing the weight of any individual co-authorship relationship in a paper with many authors, while allowing the overall weight of the neighborhood consisting of all co-authorship connections to increase linearly with the number of authors (see examples in supplemental methods). in order to visualize the final cumulative network, layout was determined using the distributed recursive graph algorithm . nodes were sized by author impact score rank and colored by primary subspecialty designation. edge width was determined by the weighted co-authorship score. statistical analysis. non-independent network metrics including growth, density, assortativity, modularity, and pagerank are reported descriptively with medians and interquartile ranges (iqr). gender proportion over time was fit with locally estimated scatterplot smoothing (loess) regression using default settings of degree = with smoothing parameter/span α = . . for the final cumulative network, the independent variables author impact score and longevity were compared ( ) between genders and ( ) by whether the author changed subspecialties over time; only those authors with longevity ≥ year were included in the second comparison. these comparisons were made with the two-sided wilcoxon rank sum test; p value < . was considered statistically significant. www.nature.com/scientificreports/ sensitivity analysis. to determine whether the scoring algorithm was robust to modifications, we conducted a sensitivity analysis where the author role and trial design coefficients were varied by ± % and ± %, respectively. normalized density distributions for the final cumulative network under each permutation were calculated, and temporal assortativity and modularity were compared to baseline with pearson's correlation coefficient. a version of this manuscript is posted on the medrxiv preprint server, accessible here: https ://www.medrx iv.org/conte nt/ . / v . a very early version of the work was presented in poster format at the visual analytics in healthcare workshop (november ). there are no other prior presentations. the datasets generated and analyzed in this study are available at harvard dataverse . received: january ; accepted: september scientific reports | ( ) : | https://doi.org/ . /s - - - www.nature.com/scientificreports/ the biological actions and therapeutic applications of the b-chloroethyl amines and sulfides nitrogen mustard therapy; use of methyl-bis (beta-chloroethyl) amine hydrochloride and tris (beta-chloroethyl) amine hydrochloride for hodgkin's disease, lymphosarcoma, leukemia and certain allied and miscellaneous disorders general principles of cancer chemotherapy historical and methodological developments in clinical trials at the national cancer institute a history of cancer chemotherapy cancer statistics associating co-authorship patterns with publications in high-impact journals breast cancer publication network: profile of co-authorship and co-organization nepotism and sexism in peer-review inequality quantified: mind the gender gap expectations of brilliance underlie gender distributions across academic disciplines gender contributes to personal research funding success in the netherlands comparison of national institutes of health grant amounts to first-time male and female principal investigators women and academic medicine: a review of the evidence on female representation the 'gender gap' in authorship of academic medical literature-a -year perspective bibliometrics: global gender disparities in science gender disparities in high-quality research revealed by nature index journals the gender gap in highest quality medical research-a scientometric analysis of the representation of female authors in highest impact medical journals historical comparison of gender inequality in scientific careers across countries and disciplines the structure of scientific collaboration networks strategic networks access to expertise as a form of social capital: an examination of race-and class-based disparities in network ties to experts broadening the science of broadening participation in stem through critical mixed methodologies and intersectionality frameworks the perils of intersectionality: racial and sexual harassment in medicine combination chemotherapy with adriamycin and cyclophosphamide for advanced breast cancer -phenylalanine mustard (l-pam) in the management of primary breast cancer. a report of early findings tamoxifen (antiestrogen) therapy in advanced breast cancer antitumor activity of thalidomide in refractory multiple myeloma phase i trial of the proteasome inhibitor ps- in patients with refractory hematologic malignancies phase i/ii study of ipilimumab for patients with metastatic melanoma improved survival with ipilimumab in patients with metastatic melanoma leadership in complex networks: the importance of network position and strategic action in a translational cancer research network a unified framework for the pareto law and matthew effect using scale-free networks experience versus talent shapes the structure of the web topology of evolving networks: local events and universality threshold models of collective behavior quantifying the evolution of individual scientific impact cancer chemotherapy-present status and prospects chemotherapy in the treatment of leukemia and wilms' tumor women in academic medicine leadership: has anything changed in years? covid- amplifies gender disparities in research why aren't there more women leaders in academic medicine? the views of clinical department chairs the "gender gap" in authorship of academic medical literature-a -year perspective a network's gender composition and communication pattern predict women's leadership success distribution of medical school graduates by gender idelalisib and rituximab in relapsed chronic lymphocytic leukemia gender bias in scholarly peer review name-centric gender inference using data analytics research productivity in academia: a comparative study of the sciences, social sciences and humanities the gender gap in peer-reviewed publications by physical therapy faculty members: a productivity puzzle comparison of evidence of treatment effects in randomized and nonrandomized studies can the pharmaceutical industry reduce attrition rates? contradicted and initially stronger effects in highly cited clinical research double-blind peer review and gender publication bias org: a collaborative online knowledge platform for oncology professionals effects of vinorelbine on quality of life and survival of elderly patients with advanced non-small-cell lung cancer trying an authorship index measuring co-authorship and networking-adjusted scientific impact how has healthcare research performance been assessed? a systematic review a new index to use in conjunction with the h-index to account for an author's relative contribution to publications with high impact radiotherapy plus concomitant and adjuvant temozolomide for glioblastoma new clinical trial designs in the era of precision medicine: an overview of definitions, strengths, weaknesses, and current use in oncology modularity and community structure in networks assortative mixing in networks a set of measures of centrality based on betweenness the anatomy of a large-scale hypertextual web search engine drl: distributed recursive (graph) layout locally weighted regression: an approach to regression analysis by local fitting replication data for: seven decades of chemotherapy clinical trials: a pan-cancer social network analysis vanderbilt university) conducted and are responsible for the data analysis. we declare the following interests gibson are members of the editorial board of hemonc.org. rozina a. chowdhery, ronald s. go and eddy j. chen were members of the editorial board of hemonc.org. all positions at hemonc.org are voluntary and uncompensated, and the stock of hemonc.org llc has no monetary value none of the funders had any direct role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. supplementary information is available for this paper at https ://doi.org/ . /s - - - .correspondence and requests for materials should be addressed to j.l.w.reprints and permissions information is available at www.nature.com/reprints.publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.open access this article is licensed under a creative commons attribution . international license, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the creative commons licence, and indicate if changes were made. the images or other third party material in this article are included in the article's creative commons licence, unless indicated otherwise in a credit line to the material. if material is not included in the article's creative commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. to view a copy of this licence, visit http://creat iveco mmons .org/licen ses/by/ . /. key: cord- - py k e authors: buyse, marc; trotta, laura; saad, everardo d.; sakamoto, junichi title: central statistical monitoring of investigator-led clinical trials in oncology date: - - journal: int j clin oncol doi: . /s - - - sha: doc_id: cord_uid: py k e investigator-led clinical trials are pragmatic trials that aim to investigate the benefits and harms of treatments in routine clinical practice. these much-needed trials represent the majority of all trials currently conducted. they are however threatened by the rising costs of clinical research, which are in part due to extensive trial monitoring processes that focus on unimportant details. risk-based quality management focuses, instead, on “things that really matter”. we discuss the role of central statistical monitoring as part of risk-based quality management. we describe the principles of central statistical monitoring, provide examples of its use, and argue that it could help drive down the cost of randomized clinical trials, especially investigator-led trials, whilst improving their quality. medical practice largely relies on the evidence generated by clinical trials, particularly randomized controlled trials (rcts). these are considered the gold-standard approach for evaluating therapeutic interventions due to their capacity to allow for inferences about causal links between treatment and outcomes [ ] . a general property of experimental research is that internal validity (i.e., the reliability of results) and external validity (i.e., their generalizability) tend to move in opposite directions in response to attempts to control trial features such as the population, the intervention, and the assessment of outcomes. this gives rise to different attitudes towards clinical trials in general, and rcts in particular: one that prioritizes internal validity (the explanatory attitude), and one that places more emphasis on the generalizability of results (the pragmatic attitude) [ ] . industrysponsored trials, here defined as trials that aim to investigate experimental drugs with largely unknown effects, are typically characterized by an explanatory approach, which is suitable for the development of these novel agents or combinations. in contrast, investigator-led clinical trials, here defined as trials that aim to investigate the benefits and harms of treatments in routine clinical practice, are typically characterized by a pragmatic attitude. table characterizes some of the contrasts between an explanatory and a pragmatic approach to clinical trials. these contrasts have direct implications on the conduct of investigator-led trials, notably with regards to ways of ensuring their quality, which is the focus of this paper. investigator-led clinical trials belong to a research area known as comparative-effectiveness research. we note that "real-world evidence" is a broader concept, given that it is often applied to observational research, something that falls outside the scope of our paper [ , ] . industry-sponsored clinical trials are essential for the development of new treatments. these clinical trials need to fulfil commercial interests and market expectations, which may not always address all patients' needs [ ] . moreover, clinical trials that lead to the approval of novel drugs or devices often have shortcomings that have been recognized for decades. such shortcomings include the strictness of the eligibility criteria, the choice of comparators, the effect size of interest, the choice of outcomes, and insufficient data on long-term toxicity [ ] . arguably, some of these shortcomings are a by-product of the general principles underlying marketing approval by regulatory agencies, such as the japanese pharmaceutical and medical devices agency (pmda), the european medicines agency (ema), and the us food and drug administration (fda). these agencies must determine whether a new drug is sufficiently safe and effective to be made available for clinical use, which requires a careful assessment of the quality of the pivotal trial design, conduct, data and analysis whilst allowing safe and effective new drugs to enter the market quickly [ ] . however, the need remains to generate additional, post-approval evidence on novel drugs or devices [ , ] . such evidence is required for clinical practice, as it provides a far better understanding of the effectiveness and safety of competing interventions in "real life". moreover, it allows the assessment of patients and settings not necessarily covered by the initial approval, thus leading to potential extensions of indications and refinement of the drug usage in patient subgroups. even for newly approved drugs, many questions of clinical interest typically remain unanswered at the time of approval, including the duration of therapy, dose or schedule modifications that may lead to a better benefit/risk ratio, combinations of the new drug with existing regimens, and so on. likewise, repurposing of existing drugs, whose safety and efficacy profile is well documented in other indications, is more likely to be attractive in the setting of investigator-led trials than to pharmaceutical companies for whom a given product ceases to be financially attractive towards the end of its life-cycle [ ] . finally, large, simple trials that address questions of major public health importance have been advocated for decades as one of the pillars of evidence-based medicine [ ] . all in all, more and larger investigator-led trials are needed, and it is crucially important to identify ways of conducting them as cost-effectively as possible [ , ] . in particular, excessive regulation of investigator-led trials, using industrysponsored trials as a model, is both unnecessary and counterproductive [ ] . taruno (table ) in [ ] ). publicly available clinical-trial registries are useful to assess the importance of investigator-led clinical trials in worldwide clinical research. the longest established and largest registry is clinicaltrials.gov, with , trial protocols as of march , . clinicaltrials.gov contains trial protocols from both the us and other countries, and distinguishes between four major types of funders: ( ) industry (e.g., pharmaceutical and device companies), ( ) the us national institutes of health, ( ) other federal agencies (e.g., fda, centers for disease control and prevention, or department of veterans affairs), and ( ) all others (including individuals, universities, and community-based organizations). for the purposes of this paper, we focus on clinical trials conducted by sponsors other than the pharmaceutical and device industry, i.e., funder types ( )-( ), as opposed to funder types ( ). we call these trials "investigator-led" clinical trials for simplicity. figures and show the number of registered interventional clinical trials in oncology, by funder type and year the trial started, in the us (fig. ) and all other countries (fig. ). in the us, about such trials were reported to have started in , about being industry trials and about investigator-led trials (roughly half of which sponsored by nih and other federal agencies, and half by other sponsors). in other countries, about such trials were reported in , about being industry trials versus about investigator-led trials (with there may be substantial under-reporting of clinical trials to clinicaltrials.gov, especially for non-us trials and for investigator-led trials, so it is conservative to assume that investigator-led trials outnumber industry-sponsored trials worldwide. as such, investigator-led trials have the potential to generate much of the evidence upon which the treatment of cancer patients is decided. yet, as stated above, investigator-led trials may be under threat because of excessive regulation and bureaucracy, and the accompanying direct and indirect costs. the rising costs of clinical trials have been a matter of major concern for some time [ ] . the contribution of clinical trials to the overall costs of drug development is not known with precision, but recent estimates suggest that pivotal clinical trials leading to fda approval have a median cost of us$ million; such costs are even higher in oncology and cardiovascular medicine, as well as in trials with a long-term clinical outcome, such as survival [ ] . interestingly, the cost of clinical trials was found to have huge variability, with more than -fold differences at the extremes of the cost distribution among the trials surveyed [ ] . the extent to which the skyrocketing costs of clinical research depend on individual components of clinical-trial conduct can vary substantially across trials, and likely when industry-sponsored studies are compared with investigator-led trials. in industry-sponsored trials, a great deal of resources are spent in making sure that the data collected in clinical trials are free from error. this is usually done through on-site monitoring (site visits) including source-data verification and other types of quality assurance procedures, alongside with centralized monitoring including data management and the statistical monitoring that is the focus of the present paper. while some on-site activities make intuitive sense, their cost has become exorbitant in the large multicenter trials that are typically required for the approval of new therapies [ ] . it has been estimated that for large, global clinical trials, leaving aside site payments, the cost of on-site monitoring represents about % of the total trial [ ] . the high costs of monitoring could be justified if monitoring activities were likely to have an impact on patient safety or on the trial results [ ] . yet, there is no evidence showing that extensive data monitoring has any major impact on the quality of clinical-trial data, and none of the randomized studies assessing more intensive versus less intensive monitoring has shown any difference in terms of clinically relevant treatment outcomes [ ] [ ] [ ] [ ] [ ] . besides, there may also be a lack of effectiveness of sending large numbers of data queries to the centers as part of the data management process. in one limited study, only six queries were found ( . % of queries) that might have influenced the results of three phase cancer clinical trials, had the discrepancy not been revealed [ ] . but without question, the most time-consuming and least efficient activity is source-data verification, which can take up to % of the time spent for on-site visits, hence it is especially important to make sure that such time is well spent. a large retrospective study of industrysponsored clinical trials has shown that only . % of all data were changed as a result of source-data verification [ ] . moreover, it has been shown via simulations that random errors, which comprise most of the errors detected during source-data verification, have a negligible impact on the trial results [ ] . in contrast, systematic errors (those that create a bias in the comparison between the treatment groups of a randomized trial) can have a huge impact on the trial results, but these types of errors can either be prevented or detected and corrected centrally [ , ] . all in all, the monitoring of clinical trials needs to be re-engineered, not just for investigator-led trials, but also for industry-sponsored trials. to instigate and support this much-needed transition, regulatory agencies worldwide have advocated the use of risk-based quality management, including risk-based monitoring and central statistical monitoring (csm) [ , ] . the central principle of risk-based quality management is to "focus on things that matter". what matters for a randomized clinical trial is to provide a reliable estimate of the difference in efficacy and tolerance between the treatments being compared. it is important to stress that the criteria to assess efficacy and tolerance may differ between industrysponsored trials and investigator-led trials. for instance, in terms of efficacy, industry-sponsored trials often use the centrally reviewed progression-free survival (pfs), which may provide the most sensitive indicator of the antitumor effect of a treatment, while investigator-led trials use the locally assessed pfs, which may provide the most relevant indicator of disease progression for clinical decision-making (for instance to change therapy). neither of these two assessments of pfs is better than the other; they serve different purposes and have their own advantages and limitations. centrally reviewed pfs is arguably a "cleaner" endpoint, but it is quite expensive to measure and does not reflect clinical routine; as such it is neither feasible nor desirable in investigator-led trials. in terms of safety, investigator-led trials can collect much simpler data than industry-sponsored trials of drugs for which safety has not yet been demonstrated. typically, in investigator-led trials, the occurrence of common terminology criteria for adverse events grade or toxicities will suffice, plus any unexpected toxicity not known to be associated with the drug being investigated. finally, medical history and concomitant medications, which may be important to document drug interactions with an experimental treatment, serve no useful purpose in investigator-led trials. all in all, investigator-led trials should collect radically simpler data than industry-sponsored trials. similarly, data quality needs to be evaluated in a "fit for purpose" manner: while it may be required to attempt to reach % accuracy in all the data collected for a pivotal trial of an experimental treatment, such a high bar is by no means required for investigator-led trials, as long as no systematic bias is at play to create data differences between the randomized treatment groups (for instance, a higher proportion of missing data in one group than in the other) [ ] . both types of trials may benefit from central statistical monitoring of the data; industry-sponsored trials to target centers that are detected as having potential data quality issues, which may require an on-site audit, and investigatorled trials as the primary method for checking data quality. central statistical monitoring (csm) is part of risk-based quality management [ ] . as shown in fig. , the process starts with a risk assessment and categorization tool (ract) [ ] . csm helps quality management by providing statistical indicators of quality based on data collected in the trial from all sources. a "data quality assessment" of multicenter trials can be based on the simple statistical idea that data should be broadly comparable across all centers [ ] . note that this idea is premised on the fact that data consistency is an acceptable surrogate for data quality. note also that other tools of central monitoring can be used in addition, to uncover situations in which data issues occur in most (or sometimes all) centers; these other tools, which include "key risk indicators" and "quality tolerance limits", are beyond the scope of this article. taken together, all these tools produce statistical signals that may reveal issues in specific centers. actions must then be taken to address these issues, such as contacting the center for clarification, or in some cases performing an on-site audit to understand the cause of the data issue (fig. ) . although it is a simple idea to perform a central data quality assessment based on the consistency of data across all centers, the statistical models required to implement the idea are necessarily complex to properly account for the natural variability in the data [ , ] . essentially, a central data quality assessment is efficient if: . data have undergone basic data management checks, whether automated or manual, to eliminate obvious errors (such as out-of-range or impossible values) that can be detected and corrected without a statistical approach; . data quality issues are limited to a few centers, while the other centers have data of good quality; . all data are used, rather than a few key data items such as those for the primary endpoint or major safety variables; . many statistical tests are performed, rather than just a few obvious ones such as a shift in mean or a difference in variability. it is worth emphasizing the last two points, namely that it is statistically preferable to run many tests on all data collected than on a few data items carefully selected for their relevance or importance. hence, what matters for a reliable statistical assessment of data quality is volume rather than clinical relevance. the reason is that the power of statistical detection comes from an accumulation of evidence, which would not be available if only important items and standard tests were considered [ ] . in addition, investigators pay more attention to key data (such as the primary efficacy endpoint or important safety variables), which, therefore, do not constitute reliable indicators of overall data quality. this being said, careful checks of key data are also essential, but such checks, for the most part, are not statistical in nature. figure shows a made-up example of systolic blood pressure, measured during six successive visits, in nine centers (numbered c -c ) of a fictitious multicentre trial. each colored line represents one patient. it is easy, even visually, to spot centers that deviate from the norm: a lack of variability is apparent in center c , an upward shift in mean in center c , and data propagation in center c . while these inconsistencies are too extreme to be commonly seen in practice, others may escape visual scrutiny and yet be revealing of issues worth investigating further. for instance, the fig. the risk-based quality management process data of center c may well be inconsistent with the data of other centers, as it seems to have smaller variability, but it is impossible to tell from fig. if this inconsistency falls beyond the play of chance. figure depicts only one variable, but the power of the statistical approach is to perform many tests on all variables. this can lead to a large number of tests: in a trial of centers, if data are available on variables, and if five tests on average are performed on each variable, the system generates × × = , tests. there is obviously a need to summarize the statistical information produced by all these tests in an overall inconsistency index. essentially, if p ij represents the p value of the j th statistical test in center i, the data inconsistency score for center i is equal to where w j is a weight that accounts for the correlation between the tests. put simply, the dis is a weighted geometric mean of the p values of all tests performed to compare center i with all other centers. in fact, the calculation of the dis is more complex than this formula suggests, but the technical details are unimportant here [ ] . venet et al. discusses other ways of combining many statistical tests to identify data issues in multicenter trials [ ] . it is visually useful to display the dis as a function of center size, as shown in fig. [ ] . when the trial includes many centers, it may be useful to limit the number of centers found to have statistical inconsistencies by setting the false discovery rate to a low probability, such as % [ ] . timmermans et al. provide a detailed example of csm applied to a completed trial, the stomach cancer adjuvant multi-institutional trial (samit) group trial, involving patients across centers in japan, which was subsequently published [ , ] . this trial, like many trials in oncology, included many centers with only a couple of patients [ ] . table shows the main findings of csm in this trial, which led to further checks and data corrections prior to final analysis [ ] . this example shows the power of csm to identify data issues even in small centers, providing a large enough number of patient-related variables are included in the analysis [ ] . table also shows the actions taken, when required, to correct the few data issues that remained in this final dataset. it is noteworthy that some of the statistical findings led to no action if an explanation was found for them (e.g., visits on unusual days of the week), or if, upon further investigation, the findings seemed likely to be due to the play of chance. experience from actual trials [ , , , , ] as well as extensive simulation studies [ ] have shown that a statistical data quality assessment based on the principles outlined above is quite effective at detecting data errors. experience from actual trials suggests that data errors can be broadly classified as: . fraud, such as fabricating patient records or even fabricating entire patients [ , , ] . data tampering, such as filling in missing data, or propagating data from one visit to the next [ ] . sloppiness, such as not reporting some adverse events, making transcription errors, etc. [ ] . miscalibration or other problems with automated equipment [ ] fig. a made-up example of systolic blood pressure, measured during six successive visits, in centers (numbered c -c ) of a multicentre trial. each colored line represents the systolic blood pressure of one patient over time whilst some of these data errors are worse than others, in so far as they may have a more profound impact on the results of the trial, all of them can potentially be detected using csm, at a far lower cost and with much higher efficiency than through labor-intensive methods such as source data verification and other on-site data reviews. investigatorled trials generate more than half of all randomized evidence on new treatments, and it seems essential that this evidence be submitted to statistical quality checks before going to print and influencing clinical practice. the magic of randomization versus the myth of real-world evidence explanatory and pragmatic attitudes in therapeutical trials real-world evidence -what is it and what can it tell us? safeguarding the future of independent, academic clinical cancer research in europe for the benefit of patients design characteristics, risk of bias, and reporting of randomised controlled trials supporting approvals of cancer drugs by european medicines agency, - : cross sectional analysis postapproval studies of drugs initially approved by the fda on the basis of limited evidence: systematic review generating comparative evidence on new drugs and devices after approval drug repurposing in oncology-patient and health systems opportunities why do we need some large, simple randomized trials more trials -to do more trials better improving public health by improving clinical trial guidelines and their application analysis of the status of specified clinical trials using jrct (japan registry of clinical trials) researchers facing increasing costs for clinical research, with few solutions estimated costs of pivotal trials for novel therapeutic agents approved by the us food and drug administration developing systems for cost-effective auditing of clinical trials forum on drug discovery development and translation. transforming clinical research in the united states randomized clinical trials-removing unnecessary obstacles the value of source data verification in a cancer clinical trial risk-adapted monitoring is not inferior to extensive on-site monitoring: results of the adamon cluster-randomised study triggered or routine site monitoring visits for randomised controlled trials: results of temper, a prospective, matched-pair study a randomized evaluation of on-site monitoring nested in a multinational randomized trial validation of a riskassessment scale and a risk-adapted monitoring plan for academic clinical research studies-the pre-optimon study improving the quality of drug research or simply increasing its cost? an evidence-based study of the cost for data monitoring in clinical trials evaluating source data verification as a quality control measure in clinical trials the impact of data errors on the outcome of randomized clinical trials ensuring trial validity by data quality assurance and diversification of monitoring methods reflection paper on risk based quality management in clinical trials department of health and human services ( ) food and drug administration guidance for industry. oversight of clinical investigations. a risk-based approach to monitoring data-driven risk identification in phase iii clinical trials using central statistical monitoring available at https ://trans celer atebi ophar mainc .com/asset s/rbm-asset s/ (accessed a statistical approach to central monitoring of data quality in clinical trials linear mixed-effects models for central statistical monitoring of multicenter clinical trials use of the betabinomial model for central statistical monitoring of multicenter clinical trials a hercule poirot of clinical research detection of atypical data in multicenter clinical trials using unsupervised statistical monitoring statistical monitoring of data quality and consistency in the stomach cancer adjuvant multi-institutional the control of the false discovery rate in multiple testing under dependency sequential paclitaxel followed by tegafur and uracil (uft) or s- vs. uft or s- monotherapy as adjuvant chemotherapy for t a/b gastric cancer (samit): a phase factorial randomised controlled trial fraud in clinical trials the role of biostatistics in the prevention, detection and treatment of fraud in clinical trials guidance for industry, investigators, and institutional review boards. fda guidance on conduct of clinical trials of medical products during covid- pandemic