JOURNAL OF THE NATIONAL CANCER INSTITUTE CER Integrating Economic Analysis Into Cancer Clinical Trials: 1998 | TI | ITE the National Cancer Institute-American Society of Clinical Number 24 ae Oncology Economics Workbook Contents Introduction 1 Part I: Economic Analysis and Cancer Clinical Trials 1 Part II: Planning an Economic Analysis 5 Part III: Implementing an Economic Analysis 8 Part IV: Designing an Effective Data Collection Strategy 13 Part V: Statistical Issues 15 Part VI: Economic Evaluation in the Cooperative Group Setting 22 Conclusion 26 References 26 Notes 28 ''PUBLIC HEALTHY LIBRAR) /vencecer | LIBRARY UNIVERSITY Of ea Seeiai ''MONOGRAP JOURNAL OF THE NATIONAL CANCER INSTITUTE HS Number 24 1998 ISSN 0027-8874 ISBN 019-922378-5 EDITORIAL BOARD Barnett S. Kramer ASSOCIATE EDITORS STATISTICAL Editor-in-Chief J. Gordon McVie European Editor Eric J. Seifter Book Reviews Editor J. Paul Van Nevel News Editor Frederic J. Kaye Douglas L. Weed Reviews Editors Martin L. Brown Economics Editor Susan G. Arbuck Frank M. Balis William J. Blot Peter M. Blumberg John D. Boice, Jr. Louise A. Brinton Bruce A. Chabner Ross C. Donehower Susan S. Ellenberg Suzanne W. Fletcher Michael A. Friedman Patricia A. Ganz Edward L. Giovannucci John K. Gohagan Frank J. Gonzalez Michael M. Gottesman Peter Greenwald Donald E. Henson Susan M. Hubbard Colin R. Jefcoate Frederic J. Kaye Hynda K. Kleinman Theodore S. Lawrence Bernard Levin W. Marston Linehan Marc E. Lippman Scott M. Lippman Darrell T. Liu Dan L. Longo Reuben Lotan Douglas R. Lowy Susan G. Nayfield David L. Nelson Kenneth Olden David G. Poplack Ross L. Prentice Alan S. Rabson Edward A. Sausville Robert H. Shoemaker David Sidransky Richard M. Simon Michael B. Sporn Maryalice Stetler-Stevenson J. Paul Van Nevel Douglas L. Weed Noel S. Weiss EDITORS Janet W. Andersen Jacques Benichou Donald A. Berry Barry W. Brown Bernard F. Cole Susan S. Ellenberg Scott S. Emerson Eric Feuer Edmund A. Gehan Barry I. Graubard Sylvan B. Green Susan G. Groshen Richard A. Olshen Philip C. Prorok Timothy R. Rebbeck Philip S. Rosenberg Harland N. Sather Daniel J. Schaid Richard M. Simon Donald M. Stablein Robert E. Tarone Sholom Wacholder EDITORIAL ADVISORY BOARD Hans-Olov Adami Harry B. Burke Pelayo Correa Ethan Dmitrovsky Merrill J. Egorin Soldano Ferrone Isaiah J. Fidler Jean L. Grem Curtis C. Harris Emily L. Harris Richard D. Klausner Kathy J. Helzlsouer Maureen M. Henderson Gloria H. Heppner Allan Hildesheim Waun Ki Hong William J. Hoskins Peter J. Houghton David H. Johnson V. Craig Jordan Ilan R. Kirsch Director, National Cancer Institute Susan Molloy Hubbard Director, International Cancer Information Center Julianne Chappell Chief, Scientific Publications Branch; Executive Editor Kenneth H. Kraemer Alexandra M. Levine Martha S. Linet Henry T. Lynch Frank L. Meyskens Anthony B. Miller Malcolm S. Mitchell David L. Page Kenneth J. Pienta David Schottenfeld Richard K. Severson Jerry W. Shay Sandra M. Swain G. Marie Swanson Mario Sznol Raymond Taetle Ian Tannock Peter R. Twentyman ''EDITORIAL STAFF Scientific Editors: Martin Brown, Ph.D. Mary McCabe, R.N. Kevin A. Schulman, M.D. Monograph Coordinator: Elizabeth Horowitz Manuscript Editors: Elaine Price Beck Elizabeth Horowitz MARKETING STAFF Marketing Director: Jean Griffin Baum Press Contact: Dan Eckstein EDITORIAL POLICY: Manuscripts from key conferences dealing with cancer and closely related research fields, or a related group of papers on specific subjects of importance to cancer research, are considered for publication, with the understanding that they have not been published previously and are submitted exclusively to the Journal of the National Cancer Institute Monographs. All material submitted for consideration will be subject to review, when appropriate, by at least one outside reviewer and one member of the Editorial Board of the Journal of the National Cancer Institute. Opinions expressed by the authors are not necessarily those of the publisher or the editors. Proposals for monographs should be submitted to the Editor-in-Chief, Journal of the National Cancer Institute, Oxford University Press, P.O. Box 31110, Bethesda, MD 20824-1110. Journal of the National Cancer Institute Monographs are available to members of the Information Associates Program of the International Cancer Information Center (one copy of each monograph per member, on request prior to publication). Monographs are also available from Oxford University Press, Journals Subscription Department, 2001 Evans Road, Cary, NC 27513 USA. Telephone toll-free in the U.S. and Canada, 1-800-852-7323, or 919-677-0977. Fax: 919-677-1714. E-mail: jnlorders @ oup-usa.org. © Oxford University Press ''Integrating Economic Analysis Into Cancer Clinical Trials: the National Cancer Institute-American Society of Clinical Oncology Economics Workbook Supported by an unrestricted educational grant from Amgen, Inc. Writing Committee Martin Brown, Ph.D. Carol Moinpour, Ph.D. National Cancer Institute Southwest Oncology Group Bethesda, MD Seattle, WA Henry A. Glick, M.A. Kevin A. Schulman, M.D., Chairperson University of Pennsylvania Georgetown University Medical Center Philadelphia, PA Washington, DC Frank Harrell, Ph.D. University of Virginia Charlottesville, VA Tom Smith, M.D. Massey Cancer Center Richmond, VA James Herndon, Ph.D. Cancer and Leukemia Group B Statistical Jane Weeks, M.D. Center Dana Farber Cancer Institute Durham, NC Boston, MA Mary McCabe, R.N. Damon M. Seils, Contributing Editor National Cancer Institute Georgetown University Medical Center Bethesda, MD Washington, DC Scientific Editors Martin Brown, Ph.D.; Mary McCabe, R.N.; Kevin A. Schulman, M.D. The NCI-ASCO Economics Working Group Charles L. Bennett, M.D., Ph.D.; Martin Brown, Ph.D.; Haim Erder, Ph.D.; Richard D. Gelber, Ph.D.; Henry A. Glick, M.A.; Frank Harrell, Ph.D.; James Herndon, Ph.D.; Bruce E. Hillner, M.D.; John Hornberger, M.D., M.S.; Mary McCabe, R.N.; Carol Moinpour, Ph.D.; Bernie J. O’Brien, Ph.D.; Arnold Potosky, Ph.D.; Scott Ramsey, M.D., Ph.D.; Kevin A. Schulman, M.D., Chairperson; Jeffrey Silber, M.D., Ph.D.; Tom Smith, M.D.; Koen Torfs; Edward Trimble, M.D.; Richard Ungerleider, M.D.; Jane Weeks, M.D.; Richard Willke, Ph.D.; and Robert E. Wittes, M.D. ''''Integrating Economic Analysis Into Cancer Clinical Trials: the National Cancer Institute-American Society of Clinical Oncology Economics Workbook Contents 1998 Number 24 INTRODUCTION PART I: ECONOMIC ANALYSIS AND CANCER CLINICAL TRIALS Why Economic Analysis? The Rising Costs of Cancer Care The Need to Improve Decision-Making Perspectives on the Use of Resources When Are Economic Evaluations Justified? Significant Resources at Stake Resource Considerations Are Prominent Resource Allocation Decisions Are Imminent When Not to Use Economic Analysis Generalizability of Observed Costs of Cancer Care Direct Medical Costs Direct Nonmedical Costs Productivity Costs Intangible Costs The Need to Make Decisions Criteria for Evaluating the Soundness of an Economic Analysis Conclusion Ww fu se PB BP BP PB BBW www w PART II: PLANNING AN ECONOMIC ANALYSIS nN Selection of Appropriate Trials Timing of Data Collection Choice of Perspective Time Horizon Outside the Time Horizon Sequelae Relating Costs and Benefits Endpoints Baseline Characteristics External Validity Salvage Protocols Importance of Early Collaboration Among Investigators Pilot Testing Conclusion Nn Nn | o fo NN NN ND DTD DTD DD '' PART III: IMPLEMENTING AN ECONOMIC ANALYSIS 8 Study Outline 8 Resource-Use Categories 8 Direct Medical Costs: Inpatient and Outpatient Resource Utilization 9 Hospitalizations 9 Physician Services 9 Surgical Procedures 9 Diagnostic Tests 9 Pharmaceutical Services 9 Radiation Oncology 9 Ancillary Services 9 Subacute-Care Facilities 9 Direct Nonmedical Costs 10 Measuring Resource Utilization 10 Medical Records 10 Administrative Data Sources 10 Patient Self-Report 10 Assigning Unit Costs: Direct Medical Costs 10 Hospital Costs 11 Physician Costs 1] Nursing Services 12 Diagnostic Tests 12 Pharmaceutical Services 12 Subacute-Care Facilities 12 Capital Costs 12 Protocol Costs 12 Assigning Unit Costs: Direct Nonmedical Costs 12 Aggregating Price and Quantity 12 Generic Economic Evaluation Strategies 12 Inpatient Studies 12 Outpatient Studies 13 Subacute-Care Studies 13 Conclusion 13 PART IV: DESIGNING AN EFFECTIVE DATA COLLECTION STRATEGY 13 Designing Data Collection Forms 13 Hospital Resource Data 13 Outpatient Resource Data 13 Subacute-Care Institutional Resource Use 13 Caregiver Support 13 Covariates 14 Collecting Data 14 Beginning Data Collection 14 Why Collect Nonstudy-Site Data? 14 Obtaining Informed Consent 14 Maintaining Communication Between Research Teams 14 Conclusion 14 '' PART V: STATISTICAL ISSUES The Importance of a Statistical Analysis Plan Important Properties of Economic Data Anticipated Uses of Economic Data Tke Complexity of Economic Data Distributional Characteristics of Economic Data Statistical Methods for Analyzing Economic Data Descriptive Methods Univariable Methods Covariable Adjustment and Multivariable Modeling Methods Analysis in the Presence of Informative Censoring Survival Techniques Missing Data Item Nonresponse Unit Nonresponse Imputation Methods for Missing Data Variance of Price and Quantity Vectors Statistical Issues Related to Cost-Effectiveness Ratios What Should Be in the Denominator? Variance of the Cost-Effectiveness Ratio Sample-Size Analysis Price Adjustments Discounting the Costs and Benefits of Cancer Care Conclusion 19 19 19 20 20 PART VI: ECONOMIC EVALUATION IN THE COOPERATIVE GROUP SETTING Are the Data Required? Who Will Collect the Data at Cooperative Group Institutions? How Will Data Be Collected, Monitored, and Submitted? Training: An Important Quality-Control Procedure Small-Scale Pilot Studies Who Will Analyze and Share the Data? Sharing of Data Data Analysis Issues Intergroup Collaboration Issues Specific to Cost Studies Quality Control Intergroup Pilot Test Data Issues Information-Sharing How Will Economic Studies Be Funded in Cooperative Group Trials? '' CONCLUSION 26 REFERENCES 26 NOTES 28 ''Integrating Economic Analysis Into Cancer Clinical Trials: the National Cancer Institute-American Society of Clinical Oncology Economics Workbook Introduction Clinical economics is a new and evolving discipline that ad- dresses the economic implications of changes in medical prac- tice. As applied to cancer care, clinical economics assesses the costs and effectiveness of new cancer interventions and can be a valuable endpoint in selected clinical trials. Through the inte- gration of economics into clinical evaluations, information can be developed that contributes to the decisions of patients, clini- cians, health care managers, and policymakers as to the most effective allocation of cancer care resources. To begin a formal effort to promote the development of eco- nomic analyses in National Cancer Institute (NCI) clinical trials, NCI sponsored a conference in 1994 with cancer center and cooperative group representatives to initiate discussions on the importance, appropriateness, and complexity of such evalua- tions. In 1995, the American Society of Clinical Oncology (ASCO) established its Health Economics Working Group with a charge to develop specific guidelines for implementing eco- nomic evaluations in cancer clinical trials. As a follow-up to these initiatives, in 1996, NCI and ASCO convened a workshop to consider the practical implementation of economic evalua- tions in cancer clinical trials. The participants in this small work- shop included experts from cooperative groups, NCI staff, and other experts who are actively involved in health economics. This workbook is the product of the meeting and subsequent discussions by the participants. It is meant to identify and elu- cidate the important characteristics of economic studies in the context of clinical trials, to indicate the considerations that in- vestigators should address in their planning and implementation of such studies, and to suggest possible approaches. The work- book is neither a definitive text defining how all aspects of such studies should be handled nor an official NCI document pre- scribing how studies must be done. Rather, it is a developing guide to be used as a practical reference that will be revised as the state of the art progresses. The writing committee hopes that the workbook will serve as a useful tool for NCI cooperative groups as they incorporate economics as a research endpoint into the evaluation of new cancer treatment, prevention, and diagno- sis strategies. Part I: Economic Analysis and Cancer Clinical Trials This section briefly presents the framework of economic theory underlying cost-effectiveness analysis and its application to the field of oncology. It is not, however, an in-depth review of the theory of economic evaluation, because many comprehen- sive sources currently exist (/,2). Journal of the National Cancer Institute Monographs No. 24, 1998 Why Economic Analysis? The Rising Costs of Cancer Care The percentage of total deaths in the United States attributable to cancer has risen from 16.3% in 1965 to 23.3% in 1997 (3). From 1990 through 1996, the estimated costs of cancer treatment increased from $35 billion (4) to $50 billion due to higher in- flation, increasing numbers of procedures and cases, and the aging of the population (5). Even conservative estimates, mea- suring only the direct costs of treatment, show cost increases from $18.1 billion in 1985 to $27.5 billion in 1990 to $41.4 billion in 1994 (6). Cancer will become the foremost cause of death in the United States by the end of the decade. Statistics and predictions such as these underscore the likeli- hood that cancer will continue to absorb more of the increasingly limited resources of the U.S. health care system. Some research- ers have suggested that increasing costs and demands from a sophisticated patient audience will require both implicit and ex- plicit rationing (7,S). In any event, economic forces will con- tribute to a growing need to better evaluate treatment practices by all clinicians, thus insuring that we utilize the relatively scarce resources of the health care system in an appropriate manner. Cancer therapies are increasingly resource intensive, as evi- denced by stem cell transplantation for hematologic disorders and solid tumors, paclitaxel for palliative chemotherapy of breast and ovarian cancers as well as non-small-cell lung cancer (9), serotonin—antagonist antiemetics, and growth factors for supportive care during treatment. In an era of capitated physician and hospital payments, the resources available for cancer treat- ment will be increasingly constrained, and payers and purchasers will want to understand the value of cancer treatments, espe- cially for resource-intensive therapies, before allowing wide- spread access to them. Furthermore, when health outcomes are identical for alternative therapies, the costs of these treatments may be the most important factor in determining whether to recommend or reimburse one of the treatments. Encouraging examples of less intensive strategies have the potential to improve efficiency of cancer treatment, such as evi- dence-based, minimalist follow-up care for breast cancer pa- tients (/0), decreased use of tumor markers in breast and colo- rectal cancers (//), and conservative use of hematopoietic growth factors (/2), as well as a shift to less expensive outpatient treatments for stem cell transplantation (/3). However, these strategies remain a minority of cases of new treatments in on- cology. The Need to Improve Decision-Making Because few medical interventions that provide health ben- efits also result in reduced health care expenditures (Table 1), ''Table 1. Efficacy and cost-effectiveness of selected cancer and other treatments* Intervention Benefit Cost/life-year (U.S.$ 1992) IV IgG for CLL CEA in monitoring colon cancer Aggressive Dx and Rx of unknown primary cancer ABMT for limited metastatic breast compared with standard treatment Captopril for moderate hypertension Zidovudine for HIV Screening proctosigmoidoscopy for colon cancer Adjuvant CMF for breast cancer in 75-year-old woman Renal dialysis, in-center benefit, men Screening mammography, women >50 y TPA vs. streptokinase for acute myocardial infarction ABMT for Hodgkin’s disease, second relapse only Hydrochlorothiazide for moderate hypertension Palliative chemotherapy for GI cancers Coronary artery bypass, left main disease + angina Chemotherapy vs. AIloBMT for ANLL Chemotherapy for non-small-cell lung cancer Adjuvant CMF for breast cancer Adjuvant chemotherapy for Dukes’ C colon cancer Smoking cessation counseling, men Cost-saving interventions Total hip replacement Pneumovax Influenza vaccine Hepatitis B vaccine Screening for familial hemachromatosis Coordinated home care for terminally ill cancer patients 0.8 days 0.1% increase in OS 11%-11.5% increase in OS 6.0 mo May improve survival 1.8 mo Reduces mortality by 31% 1/100 patients alive at 2 y 10.3 y 5 mo Reduces deaths OS 21%-48% 7-15 weeks 9.2-10.7 mo 2ly +7 100 000 31.000 to 6 599 000 2 400 000 116 000 82 600 8200 to 88 500 43 000 to 47 000 44 000 42 000 to 80 300 20 000 to 50000 31 000 26 200 23 500 20 200 17 400 12 500 —8400 to +20 000 4900 2000 to 3000 1300 All<0. *Modified from Smith et al. (94). IV = intravenous; IgG = immunoglobulin; CLL = chronic lymphocytic leukemia; CEA = carcinoembryonic antigen; Dx = diagnosis; Rx = treatment; OS = overall survival; ABMT = autologous bone marrow transplantation; HIV = human immunodeficiency virus; CMF = cytoxan, methotrexate, 5-flourouracil; TPA = tissue plasminogen activator; GI = gastrointestinal; AlloBMT = allogeneic bone marrow transplantation; and ANLL = acute nonlymphocytic leukemia. reliable economic analyses must be developed to improve our decision-making about the allocation of resources for cancer treatments (9, /4,/5). The rapid advancement of science and bio- technology, coupled with increasing fiscal pressures, may se- N verely limit access to cancer treatments for which economic data are not available. It is worth remembering that medical ethicists have argued that the least ethical way to allocate resources is to continue to spend money on unexamined treatments (/6,/7). As shown in Table 1, most treatments provide additional clini- cal benefits at an additional cost. Thus, economic efficiency does not imply a reduction in cost of care but provides a measure of the value provided for the cost of the treatment (/8). Economic information can strengthen arguments for adoption of new technologies that are economically attractive, provide additional input for decision-making in indifferent or gray areas, and provide clear-cut answers when costs are a major consider- ation (/9). Most existing cancer treatments have not been evalu- ated with the stringency applied to newer treatments. For ex- ample, we have no ‘‘proof’’ that intensive treatment of metastatic breast cancer, compared with best supportive care, improves quality or quantity of life or saves money, because no randomized clinical trial has been performed in the past 20 years. This is not to say that intensive treatments are ineffective or exceed reasonable cost-effectiveness limits—only that they have not been studied with the same scrutiny as high-dose che- motherapy with stem cell transplantation (20). Measurement of incremental benefit (additional lives saved by a strategy) and incremental cost (additional costs of a strat- egy) forms the backbone of comparative economic analysis. The four possible approaches of such analysis are listed in Table 2 and represent the combinations of costs and outcomes measure- ment. Estimates of absolute incremental differences in cost be- tween trial arms are needed, just as they are for analyses of Table 2. Potential combinations of costs and outcomes Costs and outcomes Results Example Peripheral blood stem cells mobilized by hematopoietic growth factors allow earlier hospital discharge and offer cost savings of about 20% (95). Improved outcome and decreased costs A strategy with improved outcome and lower costs is dominant and would always be adopted. Paclitaxel instead of cyclophosphamide and cisplatin for advanced ovarian cancer results in improved survival at an additional cost of $20 000 to $30. 000 per life-year (96). Improved outcome and increased costs Clinical advantage at an increased cost. Generic prochlorperazine instead of ondansetron for emesis less effective, but drug costs are markedly less. Decrease in clinical advantage but at a reduced cost. Poorer outcome and decreased costs Poorer outcome and increased costs should not be adopted. Poorer outcome and increased costs Mitoxantrone instead of doxorubicin in treatment of metastatic breast cancer does not improve survival or quality of life and has higher costs. Journal of the National Cancer Institute Monographs No. 24, 1998 ''treatment benefits. For example, the primary outcome measure of a cost-effectiveness analysis is the additional cost incurred per year of life gained or quality-adjusted years of life gained. Such measurement requires explicit delineation of clinical benefit in terms of years of life saved or quality-adjusted years of life saved and of dollars spent on treatment, recurrence, survival, and death. The statistical implications of absolute and relative dif- ferences are discussed in Part V. Perspectives on the Use of Resources Cost data are meaningful only when considered from a spe- cific perspective, usually that of patient, caregiver, provider, payer, or society (1.e., the choice of perspective determines what events or expenditures are regarded as costs, as well as how those costs should be calculated). The societal perspective is often the most appropriate one for resource-allocation decisions (/), because its broad approach can minimize bias in compari- sons; however, other perspectives may be important in attempts to understand the economic effects of a treatment. As in clinical medicine, investigators must not make assump- tions about how patients value treatment outcomes. For ex- ample, a recent clinical trial demonstrated substantial intellectual deficits in glioblastoma patients undergoing radiotherapy; but, because the treatment offered a few weeks of improved survival, most patients reported that they would choose the radiotherapy (21,22). Multiple perspectives may be adopted within a single analysis to illustrate the views of patients, payers, policymakers, and health care providers. For example, the cost of providing a hos- pital service is usually different than the charge for the service and the amount paid by the insurance company. An episode of care may cost a hospital $7000, but the hospital charges $10 000 for the service. From the hospital’s perspective, the charge is an overstatement of the resources consumed. If the patient must pay the full charge, however, that charge accurately represents cost from the patient’s perspective. Alternatively, while a hospital can sometimes decrease its costs by discharging patients early, patients’ costs may increase due to increased outpatient ex- penses that are not covered by health insurance. When Are Economic Evaluations Justified? Inclusion of economic evaluations in clinical trials can be justified in a variety of circumstances: when significant re- sources are at stake; when resource considerations are promi- nent; and when resource allocation decisions are imminent (23,24). Examples of each circumstance are provided below. Significant Resources at Stake Antiemetics. Although individual doses of 5HT3-agonists, such as odansetron and granisetron, are not expensive, they carry a high potential volume of use and can account for 5% of a hospital’s oncology pharmacy budget (data on file, Medical Col- lege of Virginia Hospitals). Colony-stimulating factors (CSFs). The initial and addi- tional costs of CSFs (granulocyte, granulocyte-macrophage, and erythropoietin) can consume up to 5% of a hospital’s oncology pharmacy budget (data on file, Medical College of Virginia Hos- pitals). Transplant procedures. High-dose chemotherapy with stem Journal of the National Cancer Institute Monographs No. 24, 1998 cell transplantation and similar experimental therapies carry a high dollar cost, regardless of volume. The cost of treatment and follow-up for leukemia and lymphoma patients receiving autolo- gous bone marrow transplantations and granulocyte— macrophage CSF reached $79 892 (25). Genetic predictors. BRCA1 and BRCA2 tests, genetic pre- dictors for breast and ovarian cancer, can cost anywhere from $150 to $1500 per mutation, depending on whether a family’s specific mutation has been identified. Lerman et al. (26) note that the high cost of such testing may deter some individuals from undergoing the test, and individuals may hesitate to request coverage from insurers for fear of future discrimination. Resource Considerations Are Prominent Many therapies carry a high cost, regardless of their clinical outcome, such as high-dose chemotherapy with stem-cell trans- plantation for metastatic breast cancer and allogeneic bone mar- row or stem cell transplantation from an unrelated matched do- nor (estimates allow $100 000-$250 000 for each uncomplicated case). Other therapies vary widely in costs but produce similar clini- cal outcomes. For example, high-dose chemotherapy is com- monly used as treatment for first-remission lymphoma, although less expensive standard-dose chemotherapy is known to produce similar or identical outcomes (27). Resource Allocation Decisions Are Imminent Megakaryocyte growth factors have been proposed to elimi- nate the need for platelet transfusions and to allow for earlier hospital discharge after high-dose chemotherapy (25). Also, re- sults of a trial of high-dose chemotherapy as adjuvant therapy for high-risk breast cancer (>10 positive lymph nodes) quickly changed the standard therapeutic approach to this disease (28). When Not to Use Economic Analysis Economic analysis is unnecessary when a primary therapy works well in a small number of patients—for example, testicu- lar cancer treated with platinum (600 cases/year) (29). Further- more, some therapies vary little in costs and produce similar clinical outcomes in very common diseases. For example, a number of chemotherapeutic combinations are used as adjuvant therapy for node-positive breast cancer, but they cost about the same and produce similar effects (30). Also, economic analysis is unnecessary when its results are unlikely to alter clinical practice. For example, an economic analysis of breast self-examination might show that costs are higher in the screened population and fail to demonstrate clinical benefit. But the use of breast self-examination is so widespread that an economic analysis would not change the practice. Generalizability of Observed Costs of Cancer Care Cost reflects the resources needed to provide a service and it has two elements: ‘‘resource utilization’’ in some natural units (e.g., one hospital day or one dose of ceftazadime) and the ‘‘unit cost’’ or price (e.g., 1 hospital day in a semiprivate room for $460). Units and prices are multiplied to determine treatment costs. One might calculate physician costs by multiplying phy- sician time by hourly physician earnings. The same principle ''applies to productivity costs: the cost of patient time can be calculated by multiplying time by a patient’s daily wage rate. Some administrative datasets provide reliable information only on resource utilization, whereas others provide reliable in- formation only on total cost, and still others provide information on both. It is important to recognize that the resource utilization and price components of cost have different determinants. For example, resource utilization may be affected by treatment set- ting, organizational and technical innovation, and efficient use of fixed resources. Price may be affected by the degree of com- petition in health care markets, including geographic location, regulatory and legal restraints, and the bargaining power of health care payer organizations. The “‘learning curve’’ of better efficiency and lower cost that develops as procedures are per- formed more often may decrease some costs; alternatively, un- foreseen consequences may increase expected costs. It is impor- tant to keep these issues in mind, because economic data obtained from one clinical trial may not always be generalizable to another time, setting, or pricing scheme. Costs related to a disease entity have traditionally been clas- sified into a number of categories, depending on whether the costs are directly related to the expenditure of medical or non- medical resources related to treatment and care; whether the costs are a reflection of lost economic productivity due to dis- ease-related disability or premature mortality; or whether the costs are related to the pain and suffering caused by the disease (6). The four categories of costs are listed below: Direct Medical Costs ¢ Resource use related to services and procedures delivered by the health care system in the course of treatment and care (hospital and physician costs); patient payment for medical supplies. Direct Nonmedical Costs * Resources expended by the patient and her family related to treatment and care (travel to treatment facility, parking, etc.). Productivity Costs * Not a financial transaction, but the monetary value of lost economic production due to morbidity or mortality, often uti- lized in cost-effectiveness or cost-utility analyses; may be im- portant from a societal perspective or from the perspective of an employer—for example, in the analysis of treatments for substance abuse or treatments that effect the ability of cancer patients to return to work. Intangible Costs ¢ Also not a financial transaction, but the monetary expression of pain and suffering associated with disease or treatment; various methods for measuring intangible costs, such as ‘‘willingness to pay,’ indicate that such ‘“‘costs’’ can be quantified as a monetary value (i.e., the amount a patient would be willing to pay to avoid pain or suffering); for purposes of cost-benefit analysis, intangible costs are typically expressed in monetary terms, because the outcome of cost-benefit analysis is ex- pressed in purely monetary terms; for cost-effectiveness analy- sis, intangible costs often are ignored, and for purposes of cost-utility analysis, intangible costs are expressed as part of the denominator in units of quality-adjusted life-years (QALYs) While all of the categories of costs are included in cost-of- illness studies and cost-benefit analyses, only direct costs are usually included in cost-effectiveness (or cost-utility) analyses, which have become the most common approach in economic analysis of health care interventions (Table 3). The Need to Make Decisions The need for either implicit or explicit decision-making about resource allocation dictates that decision makers should know the total budget of a program, the efficiency or cost- effectiveness of the given modality, and the overall effect of each treatment. It is essential, therefore, that data be presented in such a way that decision makers can assess both the costs and potential clinical consequences of new cancer treatments (Table 3). Although the mechanics of such decision-making vary greatly from centralized planning to individual choice, only lim- ited economic information is currently available to make these important treatment decisions. Criteria for Evaluating the Soundness of an Economic Analysis Economic analyses must be evaluated using defined criteria with clear goals in mind, just as clinical trials themselves are evaluated against a variety of accepted criteria. A full economic analysis of any cancer intervention should go beyond a simple Table 3. Outcome and cost studies for presentation to decision makers Type of study Attributes examined Comments Clinical outcomes — Clinical consequences Ignores costs; easy to only. choose among clearly superior therapies such as cisplatin for testicular cancer; more difficult among all others that give lesser benefits at high costs. Cost analysis Costs only, e.g., costs of treating febrile neutropenia. Ignores clinical outcomes; does not help decision makers choosing among clinical strategies unless clinical outcomes are identical. Cost-effectiveness | Measures additional Most appropriate for benefit of one strategy over another in years; measures additional costs of that strategy in dollars; combines to give dollars per year of life gained ($/life-year). Adjusts benefit gained by a utility value, or value given to being in a certain health state. comparisons of alternative therapies to treat the same condition. For example, different chemotherapy regimens for small-cell lung cancer. Cannot be used to compare treatments across disease states. Most appropriate for comparisons of alternative therapies to treat the same condition, as well as for general comparisons of interventions across disease states. Journal of the National Cancer Institute Monographs No. 24, 1998 ''identification of costs (Table 4). As the competition for scarce health care dollars intensifies, more therapies will be marketed as ‘“‘cost-effective.’’ Failure to explain these criteria should make the investigator wary about the quality of the economic analysis (3/). Conclusion In this section, we review the rationale for economic evalua- tion of new cancer treatments, including a discussion of why economic analysis is increasingly being implemented in cancer clinical trials, a discussion of when economic evaluations are justified, and an overview of the economic framework for evalu- ation of cancer clinical therapies. Economic evaluation is an increasingly important component of clinical decision-making in the context of scarce or limited resources. Economic evaluation itself can contribute to decision- making in specific clinical settings as outlined in this section. Furthermore, economic evaluation goes beyond assessing the monetary cost of medical care to include such issues as produc- tivity costs (loss to the economy from morbidity and mortality) and intangible costs (costs of pain and suffering related to illness or injury). Part II: Planning an Economic Analysis The previous section introduced some of the concepts of eco- nomic evaluation of new cancer therapies by reviewing the con- ceptual framework underlying economic evaluation of cancer clinical trials. Parts HI through V will review these concepts in more detail and lay out a framework for the collection of eco- nomic data in clinical trials. The purpose of this section is to illuminate issues in the planning of economic analyses that may Table 4. Criteria for identifying a sound economic analysis* Criterion Comment Competing alternative strategies described Incremental costs and clinical consequences described Sensitivity analysis of differing assumptions performed Costs and consequences discounted All important costs and consequences identified for each strategy Well-defined question Costs and consequences valued credibly, in usual terms Costs and consequences measured accurately in understandable units Published in a peer-reviewed journal *Adapted from (2). “‘Cost-effective’’ compared to what specific alternative? Should be a cost/life-year gained, “‘apples-to-apples’’ comparison Costs and clinical outcomes may differ, so all assumptions should be justified Time and dollars in the present worth more than in the future Should include all relevant costs, e.g., both inpatient and outpatient Was the cost study part of the initial design or done afterwards? Should be stated in $/life-year; watch out for ratios made up just for a particular study Should be in $/life-year or similar format Given market pressures, more and more such articles will show up in industry-sponsored journal supplements without the rigor of peer-review Journal of the National Cancer Institute Monographs No. 24, 1998 be important to clinical investigators designing protocols with an economic component. Selection of Appropriate Trials Economic evaluation can be an important secondary endpoint in clinical trials of new drugs and technologies, especially new cancer therapies. Implementing an economic evaluation, how- ever, requires additional time and effort, including additional data collection, management, and analysis. In deciding whether to make this investment, investigators must consider whether economic analysis is likely to aid in decision-making regarding reimbursement for new therapies or technologies. In general, investigators have suggested that economic evalu- ation is most appropriate in pivotal trials of new technologies or strategies when the investigational therapy is resource intensive or is likely to be utilized by a significant number of patients. Economic evaluation is less important in trials of ‘‘me-too’’ therapies, which may be similar in efficacy and cost to other clinical therapies. Here, however, the cost of therapy may be the important distinction between two treatment arms. Clinical in- vestigators should develop economic hypotheses before ap- proaching economic investigators to discuss the inclusion of economic analysis as an endpoint in the clinical trial. Economic evaluations usually are proposed for inclusion in phase HI clinical trials but can be included in phase II trials to allow for collection of pilot data that will be used in plans of pivotal phase III assessments. Phase IV studies are also appro- priate for consideration for economic evaluation. Some estimate of the potential difference in cost between treatment arms, and the resulting cost-effectiveness ratio if the targeted difference in survival is achieved, should be included in the concept sheet and protocol for any study with an economic component. Ideally, this estimate will be based on data collected in a pilot study, but a ‘‘back-of-the-envelope’’ calculation should be performed when pilot data are not available. Such estimates play a critical role in justifying the investment in an economic component and are needed for the planning of an efficient data collection strategy. Timing of Data Collection Data collection for economic studies can be based on pro- spective data that have been collected concurrently with the clinical trial or on retrospective data that have been collected after the trial’s completion. The advantages and disadvantages of each approach are discussed in more detail in Part V. Prospective economic evaluation assures that economic data will be available at the time of a trial’s completion. It allows for collection of the four types of economic data (direct medical, direct nonmedical, productivity, and intangible), as well as qual- ity-of-life data from study patients. But prospective collection of economic data requires substantial planning and is, in this sense, no different than prospective collection of clinical data. Because economic evaluation is usually a secondary endpoint of clinical trials, and because there is a risk that the null hypothesis will be adopted at the end of the trial, investigators should carefully consider the potential use of resources for an economic evalua- tion in a trial of an unknown or untested therapy. Retrospective economic evaluation offers a more limited ap- proach to assessment of resource use within clinical protocols, ''typically focusing on direct medical costs, because other types of data often are not available in the medical record. In addition, retrospective approaches can not assess patients’ preferences or quality of life. However, in this approach, the expense of col- lecting economic data will only be required for therapies with proven efficacy. The choice between prospective and retrospective data col- lection usually rests on the potential clinical effect of the new therapy and on the amount of resources available to investiga- tors. If the trial is indeed an appropriate candidate for economic evaluation, and if resources are available for proper planning and design of an economic study, a prospective approach should be taken, because it will allow for a more thorough evaluation of the costs of a new therapy or technology. Choice of Perspective Economic evaluation of medical care is unique in that it as- sesses the costs and benefits of therapies from a variety of per- spectives, including those of patients, caregivers, employers, payers, and society. However, clinical and economic investiga- tors must determine which perspectives are most relevant to the clinical trial under consideration. For example, economic evalu- ation from a hospital’s perspective requires consideration of in- patient costs but not outpatient or out-of-pocket costs. Data col- lection from a patient’s perspective is more complex and may require primary data collection to yield more useful information on out-of-pocket costs and lost work days. Economic evaluation from a societal perspective is the most comprehensive of all and includes consideration of both patient and provider perspectives. Unless a therapy is particularly burdensome to patients, the most common perspective for clinical trials is a ‘‘modified”’ societal perspective, which considers only direct medical costs. In this type of analysis, direct medical costs are quantified as social costs, but direct nonmedical and productivity costs are not routinely collected. Time Horizon Time horizon is defined as the time from randomization to follow-up for patients in clinical trials. Time horizon can affect the economic evaluation of new therapies by limiting the amount of information available to the assessment of the long-term ef- fectiveness of treatment. Economic evaluation frequently re- quires a different time horizon than that proposed by clinical investigators. Ideally, the time horizon of an economic evaluation should be such that all costs related to the therapies being compared are captured for analysis. This concept differs from the idea of time horizon in clinical trials, which can be based on time until a measurable clinical endpoint is achieved or until the end of some predetermined time period (e.g., 5 years). Under the ideal eco- nomic scenario, subjects would be randomized to a trial and followed until death or cure (32). Unfortunately, most clinical trials are conducted for a limited length of time. In general, the time horizon for an economic evaluation of a clinical trial begins at the date of trial randomization. Alternative start dates for the time horizon include time of diagnosis or time of the onset of symptoms. Economic evaluations that begin be- fore the time of randomization usually are attempting to assess important costs incurred during the prerandomization time pe- riod (e.g., a trial of bone marrow transplantation for patients with breast cancer may need to provide induction chemotherapy to 10 patients to find one responder for the transplant portion of the study; an investigator may be interested in the costs of screening as well as the costs of the prerandomization hospitalization) (33). In determining the appropriate length of the time horizon for economic evaluation, it is helpful to distinguish between two types of clinical trials—trials of therapies for metastatic cancer (mortality studies) and trials of adjuvant therapies. Clinical trials in these two settings pose different issues with respect to clinical and economic endpoints. Knowledge of the survival curves for the type of cancer being studied, along with projections of the expected benefit of the therapy, can be used to determine how long patients should be followed for an economic study. In mor- tality studies, the follow-up time is likely to be relatively short, so resource utilization can be assessed for all patients until time of death for a high proportion of study patients. In clinical trials of adjuvant therapies, some proportion of patients may be cured and thus live a normal life expectancy. Generally, patients can be followed for recurrence. In adjuvant therapy trials, one can con- sider three potential time frames—short-term (e.g., 6 months, | year), intermediate, and lifetime. Outside the Time Horizon When the time horizon of a clinical study differs from that of the economic study, it may be necessary to track all resource use in both arms of the trial over a period that is longer than the relevant clinical endpoint, because important resource use may occur in the period beyond the time horizon of the clinical trial. Sequelae Substantial long-term sequelae that may occur in the rela- tively distant future, whether related to the cancer itself or to treatment of the cancer, should be included in the analysis. Relating Costs and Benefits The time horizon should be the same for the numerator and denominator of a cost-effectiveness ratio. For example, if an analysis includes 2 years of clinical follow-up data, it should also include 2 years of cost data. In a trial of adjuvant therapy, simply including the cost of treatment in the numerator and a longer follow-up in the denominator does not accurately reflect the relationship between the costs and benefits of the therapy being assessed. Endpoints Economic evaluation usually is based on a consideration of final outcomes, such as survival rather than response, disease- free survival, and time to progression. Thus, economic evalua- tion requires either consideration of a final outcome measure as an endpoint of the clinical trial or an understanding of how clinical trial endpoints will translate into final endpoints (for example, epidemiologic data may be available for estimations of cure rates for patients who demonstrate a complete response to treatment). Specification of outcomes measures must be deter- mined prospectively by both clinical and economic investigators before the trial proceeds. Journal of the National Cancer Institute Monographs No. 24, 1998 ''Baseline Characteristics Clinical studies, except for some “‘large and simple’’ trials and a small number of special cases, are generally underpowered for evaluating secondary cost or quality-of-life measurements. Analytic techniques that assess nontreatment-related variance can be used to address this issue. Thus, the ability to detect differences in costs due to treatment assignment will increase if information on baseline characteristics of study patients is in- cluded as a set of covariates in analyses of secondary study endpoints. Baseline characteristics include resource use in the period prior to the trial, clinical status, diagnostic profile, disease stage and severity, level of social support, and health status. Variation in covariates among patients stem from the type of patient rather than from treatment assignment—for example, whether a patient is an inpatient or outpatient at the time of randomization can predict resource utilization after randomiza- tion. Baseline covariates allow investigators to ‘‘disentangle’’ pa- tient characteristics prospectively. Covariates may be used as predictors in regression analysis at the end of a study, or they can be used to stratify the randomization of any parameter that may be a factor in predicting costs to a high degree. Many of these data elements can be collected as part of eligibility determina- tion and tagged as covariates in the economic evaluation. It is also important to consider any and all clinical information ob- tained prior to randomization as potential covariates. Ideally, covariates will be continuous variables. External Validity Where possible, the clinical protocol should be modified to reduce protocol-mandated tests or procedures to reduce proto- col-induced charges in medical care (protocol-induced costs or benefits) and to ensure that the protocol mirrors *‘usual care’’ as much as possible. Economic investigators should also review the clinical protocol to ensure that there are no economic biases in the structure of the trial (e.g., there are no fixed discharge criteria included in the study and/or there are no differences in pre- scribed treatment across study areas). In addition, whether or not the clinical protocol procedures are modified to reduce man- dated tests, efforts should be made to identify medical proce- dures that are mandated only for the research portion of the study. If these research resources are not related to treatment, their costs can be excluded from the analysis. Economic evaluation raises a unique set of issues about the generalizability of clinical trials themselves. These studies are designed to influence clinicians, patients, and decision makers in their perceptions of the importance of a new therapy. Thus, economic evaluation aims to ensure that patients being studied in the clinical trial are as similar as possible to relevant patient populations in the ‘‘real world.’’ For appropriate economic evaluation, therefore, clinical trials should be free of economic bias, should include patient entry criteria broad enough to pro- vide support for the external validity of the analysis, and should, to the greatest extent possible, follow strict and objective stan- dards in patient selection. Trial investigators can reduce economic bias in the design of clinical studies by avoiding prespecified indications for dis- charge, supportive care interventions, or second-line therapies. Journal of the National Cancer Institute Monographs No. 24, 1998 Investigators can further reduce bias by identifying and mini- mizing protocol-induced costs or benefits, thus rendering the phase HI clinical trial as realistic as possible in terms of actual practice in the community. Protocol-induced costs can include repeated diagnostic tests or other evaluations that do not directly contribute to clinical care or are not routine elements of clinical practice. Because these tests and evaluations can lead to diag- noses that otherwise might be missed in the usual care of pa- tients, they can also yield clinical outcomes that would not be replicated in usual practice. Finally, economic analysis is concerned with the generaliz- ability of data developed from clinical trials. The external va- lidity of studies is thus an important consideration in recruitment of subjects to the study. One such example of data that suggest clinical trial results can be replicated in the community are pre- sented in a study of breast cancer mortality in British Columbia by Olivotto et al., showing that after adoption of routine adju- vant chemotherapy, mortality declined with an effect size iden- tical to that of clinical trials (34). An alternative view is that patient selection will probably continue to be a strong factor in most trials. The task for re- searchers is to determine how well bias to external validity can be modeled by obtaining good data on the relevant characteris- tics of patients enrolled in trials compared to patients who are likely to receive the treatment shown to be efficacious in the trial. Salvage Protocols In oncology, the potential for salvage or late use of interven- tions is important to consider in advance for economic evalua- tion. As mentioned above, the economic period of a treatment may be in discordance with clinical trial measures. Late use of interventions may provide crucial information for economic evaluation of a study, because treatment for disease progression can be very costly. Economic evaluation can be implemented in this setting by including patients in the clinical trial for a fixed time period, allowing patients to remain in the economic protocol throughout this period even if they reach an endpoint of the clinical trial and withdraw from active treatment. Salvage pro- tocols with economic arms also provide an opportunity for fur- ther economic data collection from patients who have failed treatment. Finally, several unique statistical issues play a role in eco- nomic evaluation. Patients may incur fairly low costs until their disease has progressed, after which time they usually are with- drawn from the clinical trial. Yet, it is often at this point that patients become expensive and, thus, important to the economic evaluation. Economic studies often are designed to allow pa- tients the option of continuing with the economic protocol, even after withdrawing from active treatment on the clinical protocol. In this limited way, many economic evaluations have been able to successfully follow large portions of a trial population beyond the time horizon of the clinical trial (25). Importance of Early Collaboration Among Investigators To be successful, economic investigators must understand the clinical aspects of the trial being considered as well as the po- tential clinical implications of the therapy or technology being ''evaluated. Likewise, clinical investigators must be able to de- velop economic hypotheses and to understand all elements of the economic study to ensure appropriate implementation of the economic study’s design. Thus, clinical-economic evaluation is a truly collaborative undertaking between clinical and economic investigators. Economic investigators should become familiar with the eco- nomic profile of the intervention under study: What resources are required to implement the therapy? For how long will these resources be required? What is the potential benefit of therapy? What are the potential resource offsets or savings from the new therapy compared with existing therapies? In addition, economic investigators should ask these questions in the context of the current standard of care. They should consider the potential risks of the new therapy as well as how the clinical trial’s results might change the standard of care. Clinical investigators should understand that there is no “*standard’’ economic analysis to be simply appended to a clini- cal trial. Economic analysis is an important secondary endpoint of a study and, to be performed well, must be integrated through- out the clinical trial mechanism. Seamless integration of the economic component of the study with the quality-of-life data collection strategy is particularly important in any study that employs patient diaries as instruments of economic data collec- tion. Most important, clinical investigators must ‘‘buy in’’ to the economic evaluation so that they have a stake in seeing that economic data are treated with as much care as clinical data. Recipes for failure of an economic evaluation include lack of commitment by clinical investigators to the collection of eco- nomic data and a lack of understanding by clinical investigators of the purpose of the economic evaluation. Pilot Testing Economic evaluation can benefit from pilot testing in much the same way as clinical trials. Pilot testing offers investigators an opportunity to assess the use of case report forms (CRFs) and patient diaries in clinical trials so that they might become more familiar with the process of implementing a protocol and more knowledgeable about the economic profile of the disease under study. In addition, pilot tests can pinpoint a trial’s *‘cost drivers”’ and help economic investigators characterize the distributions of cost data and predict the differences in cost between treatment arms or the total cost of care for study patients. Identifying cost drivers in a pilot study can reduce the data collection burden in the final study. Furthermore, pilot data can be helpful in iden- tifying the toxicity profiles of a therapy and, in particular, the nature and frequency of high-cost events, such as hospitaliza- tions for febrile neutropenia or cardiac toxicity. Similarly, pilot testing can provide information for estimations of sample size and expected cost differences between study arms (35). Conclusion This section reviews issues important for clinical investigators to consider in contemplating whether to implement an economic evaluation within a specific clinical trial. Issues for clinical in- vestigators to consider include the selection of appropriate clini- cal trials for economic evaluation, timing of data collection, choice of perspective, time horizon, study endpoints, and exter- nal validity or generalizability of the study. Investigators must also consider that to design robust economic comparisons of clinical trials, economic evaluation requires input from eco- nomic investigators as early collaborators within the clinical team. To implement a successful protocol, clinical investigators must assist in the design of the economic evaluation and in the implementation of the economic component of the protocol. Part III: Implementing an Economic Analysis In the previous section, we reviewed some of the important concepts for a clinical investigator to consider when deciding whether to incorporate an economic evaluation into a clinical trial. In this chapter, we review in more detail many of the same issues from the perspective of someone who is designing an economic evaluation of a clinical trial. In the next chapter, we focus on the collection of data items themselves. Study Outline Economic evaluation measures the use of resources and often takes note of the same events of interest to clinical investigators, such as whether a patient receives a magnetic resonance imaging scan (MRI). Yet, while clinical investigators are most interested in the results of the MRI, economic investigators take greater interest in the number of MRIs consumed during the study pe- riod. In addition, economic investigators attempt to interpret changes in resource quantities. For example, a specific therapy may reduce length of stay for patients receiving autologous bone marrow transplant (ABMT) yet increase the need for outpatient follow-up visits. Costs are applied to these resource-utilization measures to demonstrate overall economic benefit or harm. For example, if inpatient days cost $700 each and physician visits cost $100 each, the ABMT scenario described above will save money if it reduces length of stay by | day and increases out- patient resource use by no more than the equivalent of six phy- sician visits. Thus, economic evaluation requires both resource utilization and unit cost information to provide an overall estimate of the resources consumed in the course of caring for an individual patient. The following sections focus specifically on the devel- opment of resource-use data and unit costs. Resource-Use Categories Oncology patients consume a variety of resources in the course of medical treatment, including hospital services, physi- cian and nursing services, diagnostic tests, pharmaceutical prod- ucts, radiation therapy, and ancillary services. Given the tremen- dous complexity of oncology care, economic investigators must make explicit decisions about the types and amounts of infor- mation they wish to incorporate into their protocols. In general, these data elements should include resources that carry signifi- cant costs (e.g., inpatient stays) and services that are likely to differ across treatment groups. The most important of these— known as “‘cost drivers’’—represent resources that are both costly and differ across treatment groups (36) and they should be validated through pilot data collection efforts. Cost drivers are not simply drivers of a study’s total cost but of the increments in costs between the study’s treatment arms. Investigators should, therefore, assess the impact of a particular resource on the mean costs, the variance in costs, and the difference in costs between Journal of the National Cancer Institute Monographs No. 24, 1998 ''treatment arms before eliminating it from the list of data ele- ments to be collected. Computerized cost-accounting systems, which are increas- ingly being adopted by health care provider organizations, make it possible to retrieve information on resource use and costs for a variety of resource-use categories. For example, the Decision Support System used by Group Health Cooperative of Puget Sound contains information on resource use and costs for such categories as inpatient days, surgeries, outpatient/short-stay hos- pitalizations, primary care visits, specialty care visits, mental health visits, emergency room visits, community health services, diagnostic radiology, laboratory services, occupational/physical therapy, respiratory therapy, and pharmacy (37). Direct Medical Costs: Inpatient and Outpatient Resource Utilization Hospitalizations Hospitalizations typically represent the most intense period of resource utilization by clinical trial patients. Resources received during a hospitalization include hospital *‘hotel services’’ them- selves, physician services, nursing services, laboratory and pa- thology services, blood work, operating rooms, and pharmaceu- tical products. Hospital services are provided based on the type of unit to which the patient is assigned, e.g., intensive care, high care, telemetry, general, and rehabilitation. Thus, descriptions of re- source utilization for hospital *‘hotel services’’ could be a de- scription of length of stay or a description of length of stay by unit type (e.g., a 10-day admission of which 4 days were spent in the oncology unit and 6 were spent on the general medical floor). One distinguishing feature of the different unit types within a hospital is the level of nursing support services avail- able to patients. Thus, location within the hospital is often also used as a description of the intensity of nursing care. Physician Services Physician services in the hospital setting include those pro- vided by the attending physician (either medical or surgical) and by consulting physicians. In the outpatient setting, they can in- clude oncology services as well as services provided by primary care and consulting physicians. In an economic investigation, measures of physician effort can be characterized in terms of time (a 30-minute consultation) or workload units (resource- based relative value units [RVUs]) (38,39). These data can be collected for all physicians treating patients in the protocol or by the type of physician providing the service. Surgical Procedures Surgical procedures are an important component of physician services. Performed for diagnostic and therapeutic purposes, sur- gical procedures can occur in and out of the hospital setting. The most global measure of surgical services is a description of the procedure itself (e.g., laparoscopy, colectomy). This description can be text based, or it can be coded using a standardized system such as the International Classification of Diseases (ICD) or the Common Procedural Terminology (CPT). Surgical procedures can be further described using a second level of detail—for example, the time in the operating room or the use of specific Journal of the National Cancer Institute Monographs No. 24, 1998 supplies, including blood products or isotopes. A third measure, which is especially important for outpatient surgical services, is a description of the location where the service is performed (e.g., hospital, outpatient surgical center, free-standing surgical center, physician’s office). These levels of detail can be used to describe resource utilization by different treatment modalities. Diagnostic Tests Some of the most important services provided to patients both in and out of the hospital are diagnostic tests. As mentioned above, economic investigators are interested in recording the number and types of services provided to patients. These re- sources can be captured in a variety of ways, including listing the number of radiologic or laboratory tests as either an aggre- gate measure or an itemized listing of the occurrence of specific types of tests and procedures. Pharmaceutical Services Pharmaceutical utilization during hospitalization periods may already be captured within clinical CRFs’ concomitant medica- tion sections. If not, investigators may wish to detail the amount of medication received by patients during a hospital stay. Medi- cation use can be characterized by medication type and dose (total daily dose) or by total dose received during a specific time period (i.e., daily dose, total dose during a course of chemo- therapy, total dose during a hospitalization). In addition, eco- nomic investigators should record the route of administration, because intravenous administration may be more costly than oral administration. Data collection can include all concomitant medications or a subset of specific medications of interest to the investigators. Radiation Oncology Physician services for radiation oncology treatment can be captured as described in the Physician Services section above. Use of radiation treatment facilities can be tracked by recording the types of services received by patients, the personnel effort required to provide such services, the duration of treatment, the dose of isotope delivered, and the setting where the treatment occurred. In radiation oncology protocols, more detailed treat- ment or service data may be collected. Ancillary Services Many other services are available to patients in and out of the hospital. These services can be especially important to the care of oncology patients, although they are not delivered by physi- cians. Social work, speech therapy, pain management, occupa- tional therapy, physical therapy, and clinical pharmacology ser- vices, to name a few, are usually measured by tracking the occurrence of specific consultations or by measuring the amount of time the provider spent in consultation with study patients. Subacute-Care Facilities Cancer patients may utilize a variety of subacute-care facili- ties, including nursing homes, rehospitalization centers, and in- patient hospices. These resources may be quantified by the type of institution and the length of stay in the institution. For more specific studies, data can be collected on specific services re- ceived during the stay. ''Direct Nonmedical Costs Direct nonmedical costs include the costs of transportation to and from physician visits or hospitalizations and other costs related to receiving medical care. For extended stays at a treat- ment center, these costs can include the cost of hotel services or of moving families to a treatment center. Investigators should determine the specific resource quantities of interest to be cap- tured as a direct nonmedical cost. These can include the number of hours of parking at specific treatment centers or the number of days spent at a hotel adjacent to a treatment center. Measuring Resource Utilization Data on resource measures can be collected from medical records and medical bills and through patient self-report. In this section we review the data sources available to economic inves- tigators. Medical Records Medical records include source documents, such as medical charts and abstracted flow sheets. These documents may contain data on all elements of resource utilization during a specific time period, such as the detailed information found on a hospital chart, which can be abstracted by data collectors to provide information for each of the resource categories reported in the previous section. Interpretation may be required for some data elements, such as unit type by day of hospitalization. Medical records are limited in that they often are not linked between inpatient and outpatient treatment facilities. Patients typically have multiple providers and, thus, multiple records, making it difficult to follow patients from one site to another. Administrative Data Sources Many types of administrative data are available to economic investigators, including hospital and physician billing records and datasets from large managed care organizations (MCOs) or the Medicare program. Administrative datasets include informa- tion on resource quantities and can be used as a direct measure of resource utilization or abstracted to track utilization of spe- cific services. As with medical records, certain administrative datasets may be available only for individual episodes of care (hospital datasets) and may not be available as linked records of inpatient and outpatient services. Also, Medicare and managed care administrative data may be available only for defined popu- lations of patients and usually do not include information on noncovered services and specific service elements (for example, they may include little information on resources utilized during a hospitalization). Patient Self-Report Patient self-report enables data to be collected for all medical care received by patients, whether or not it is provided at the study site. Patient self-report data can be used in several ways in an economic study. First, it may be the only means of collecting certain data items (direct nonmedical costs, productivity costs, intangible costs). Second, patients are often the only people with complete knowledge of all of the services they have received. Thus, patient self-report can be used to assess direct medical costs received by patients. However, patient self-report data may 10 not be reliable for several types of information, including data on resource utilization during acute-care hospitalizations or for de- tailed information on treatments received. In these cases, patient self-report information should not be used as the sole data col- lection strategy for economic studies. Often, patient self-report is combined with source-document validation. In this strategy, patients report the occurrence of a specific service at a facility, which then leads to follow-up data collection with that facility. In addition to direct medical resource-use data, patient self- report may include assessment of the amount of caregiver time required for care at home and the number of days lost from work or other activities. Alternatively, work loss could also be ab- stracted directly from patients’ work records, though it is un- likely that this would be feasible in a typical clinical trial unless the trial were restricted to patients with a single employer (for example, in a workplace cancer screening trial). Several methods exist for collecting patient self-report data for resource use items. Patients may be interviewed, either dur- ing a protocol visit or over the telephone. Alternatively, spe- cially designed questionnaires may be mailed to patients at regu- lar intervals. Problems occur with patient self-report data for a number of reasons. Patient recall of events becomes problematic when the recall period is extended, if the patient is a heavy user of medical care services, and/or illness interferes with the pa- tient’s mental status. Thus, scheduled visits or telephone or mail contact must be sufficiently frequent to avoid recall problems. An alternative way to avoid recall problems is to ask the patient to complete a diary at home as care is received. The diary should then be brought to the study site at each visit, used by the patient as a reference during telephone follow-up, or used as a reference when completing a mail questionnaire. Diary problems can be minimized through reminder telephone calls to the patient at regular intervals between visits and by means of a letter sent before each study visit reminding the patient to complete the diary. For patients who are illiterate or mentally incompetent, resource data can be obtained from a proxy, such as a family member or close friend. For an example of the use of combined patient interviews and medical billing records for obtaining direct nonmedical costs, see Bennett et al. (40). Probably the most detailed and compre- hensive survey instrumentation developed for the purpose of obtaining information on medical resource by way of patient interview is that developed for the National Medical Expendi- ture Survey (4/). Assigning Unit Costs: Direct Medical Costs Data on the costs of resources used by patients may be avail- able from standard costing data, from sources of care for indi- vidual patients, or from administrative data sets. Most U.S. hos- pitals maintain billing systems that can be used to record the costs of resources consumed by patients during a hospitalization. However, hospital billing information will only include a record of the specific services supplied by the hospital and may not include the cost of physician services during the hospital period. Except in specific settings, hospital billing information does not include information on the care received by patients on an out- patient basis. Thus, if a protocol follows patients over an ex- tended period of time, data collection mechanisms should be Journal of the National Cancer Institute Monographs No. 24, 1998 ''available to collect both inpatient and outpatient resource use as well as the cost of these resources. Administrative databases, such as the claims payment records of an MCO or large health plan, are becoming increasingly important sources of resource-use data in clinical protocols. However, given the fragmented nature of the U.S. health insur- ance system, the use of administrative datasets is not a feasible means of tracking resource use unless the study is designed around specific patient populations, such as patients more than 65 years of age (Medicare) or patients enrolled in a specific health plan. Hospital Costs The most basic description of costs for hospital care is the cost of an entire hospital period. For example, a study in which the measure of resource utilization is hospital admission will assign an overall cost to each admission. The most common means of developing an estimate of aggregate cost is by first classifying the admission by diagnosis-related group (DRG), which com- bines the primary diagnosis and procedure with information about comorbidities. Medicare or other insurance payment in- formation can then be used to assign a proxy for cost by the specific DRG. Another simple costing method tracks length of stay and assigns a cost per day, or a ‘‘per diem.” Generally, more specific information is collected on hospital resource utilization. Hospital financial information is available in the form of hospital charges. Because hospital charges are thought to overstate the actual costs of service, separate analysis of these data must be undertaken (42). Hospitals in the United States must report their overall costs and charges to the Health Care Financing Administration (HCFA) on an annual basis. This Medicare ‘‘cost report’’ has been used by investigators to develop a ‘‘cost-to-charge ratio”’ that is used to convert hospital charges to hospital costs for the purpose of economic analysis. These cost-to-charge ratios can be developed either at an aggregate level for the institution or on a more specific level based on the hospital departments providing the relevant services. The relationship between costs and charges at hospitals actually varies considerably between hospital de- partments. When available, departmental cost-to-charge ratios may be a better proxy for costs of specific services than total cost-to-charge ratios. One severe limitation to this more detailed approach to assessment of hospital costs is that each hospital may have its own system of assigning costs to uniform billing (UB-92) categories. Thus, fresh frozen plasma may be included in one institution’s blood bank cost center report but in the operating cost center in a different institution. These differences make aggregation of departmental cost-to-charge ratios a labor- intensive task. A study developed by the Cancer and Leukemia Group B (CALGB) is currently attempting to evaluate the ca- pabilities of administrative datasets within CALGB member institutions. The group is expected to develop recommendations for strategies of requesting cost information from CALGB member institutions for economic evaluations within group studies. Given the increasing financial pressures resulting from changes in the health care system, many hospitals have devel- oped their own cost-accounting systems to provide more de- Journal of the National Cancer Institute Monographs No. 24, 1998 tailed cost information for management decisions. However, not every institution has such an accounting system in place. Where these datasets exist, they can be used to assess the costs of services and may offer a more accurate reflection of the true costs of services than the cost-to-charge ratio (43). It is difficult to develop costs for specific services from de- tailed hospital bills. Hospitals often keep track of every service received by an individual patient on a disaggregated basis. Thus, for a specific procedure or treatment—for example, an intensive care unit day or an hour of operating room time—the hospital may bill for each of the hundreds of components of that service separately. In fact, it may be impossible to develop an overall cost for specific services for an institution. One approach to resolving this issue is to develop a regression-based model that will use the resource counts in the economic protocol as predic- tor variables in decomposing the overall hospital bill. This tech- nique could result in the development of bundled costs for the specific clinical services collected within a CRF. More recently, hospitals have revised their billing procedures due to changes in market forces. For example, hospitals may agree to a contract price for procedures that are part of the research protocol, resulting in a gap between the actual bill and the contract price. Furthermore, some hospital accounting sys- tems simply record the contract price. In these institutions, the process of tracking costs is predominantly being used for inter- nal cost accounting but not for generating bills to external or- ganizations. Some hospitals continue to generate bills but apply a contractual allowance to represent the difference between the charge and the contract price. Other hospitals have a separate accounting system for procedures priced through research con- tracts. If the data collection strategy for a proposed protocol is based solely on the use of administrative datasets, investigators must ensure that facilities are continuing to track resource uti- lization by specific patients in either the billing or cost- accounting systems at each center. In a multi-institutional study, the costing strategy may be complicated by the inability to collect cost data at some study sites (most often because of lack of sufficient investigator re- sources to collect these data). In these cases, cost data may be obtained from a subset of study institutions. Statistical tech- niques to reflect this data collection strategy are discussed in Part V. Physician Costs Physicians assign CPT codes when billing for their services. In a manner analogous to hospitals, charges for specific physi- cian services may not be directly related to the costs of providing those services. However, there is no physician cost-to-charge ratio. In 1992, HCFA developed the Resource-Based Relative Value Scale (RBRVS) as a measure of the resource intensity of the specific physician services for each CPT code. The Medicare system currently implements this resource intensity measure in physician payment. RVUs may be used to calculate standard costs for physician services (44). It should be noted that there are facility charges associated with many outpatient medical proce- dures that are not included in the RBRVS fee schedule. These facility costs may need to be developed from other data sources. This is especially true for outpatient surgical services. ''Nursing Services Costs for nursing services are usually developed on an hourly basis using an average wage rate for the type of nurse (registered nurse, nurse practitioner, etc.). Diagnostic Tests Costs for diagnostic tests and procedures can include the costs of physician services as well as the costs of conducting the tests. The costs of physician services for diagnostic tests are reported as described above. The costs of conducting laboratory tests are available using a set of standard laboratory workload units (Lab RVUs). The costs of diagnostic tests include the facility fees for the service. The facility costs for radiology services may need to be developed from other data sources. Pharmaceutical Services Pharmaceutical charges also vary depending on the pharmacy. One way of standardizing pharmaceutical costs is to use the average wholesale price available for specific pharmaceutical products (45), with the addition of a standard dispensing fee. Subacute-Care Facilities Costs for subacute-care facilities may be developed based on published price lists using per diem rates. For studies that focus on the use of subacute-care facilities, a more detailed costing approach may need to be developed for the study. This would be the case in a study of alternative hospice treatment programs. Capital Costs Investigators should also consider that some treatments re- quire the purchase of capital equipment; for example, cancer centers may already own much of the equipment required for new treatments, but community hospitals may have to purchase equipment to implement certain services. Capital equipment pur- chases become fixed costs that are independent of the standard process of costing for an individual patient. In other words, there may be costs associated with a program of treatment that are shared by many patients, and this cost would not appear on patient billing data (this may especially be the case in phase III trials of experimental, unreimbursed technologies for which there is no market price for the service). Derivation of these costs requires specific costing efforts to be conducted in con- junction with the clinical trial. Protocol Costs The costs of data collection in a research study should not be included in the economic assessment of the therapy. In addition, the costs of research-related archived materials not used for clinical care should be excluded from the assessment. Assigning Unit Costs: Direct Nonmedical Costs Often, these costs are measured from the patient’s perspective and include the costs of direct nonmedical services purchased by patients. While these data may be captured as both resources and costs, direct nonmedical costs are more often captured solely as costs of resources. This is usually assessed by asking patients to record their out-of-pocket expenses for these services. 12 Aggregating Price and Quantity There are four general approaches to data analysis that inves- tigators may consider in developing the economic component of clinical trials. These include resource use collected for all trial patients and costs collected for all trial patients; resource use collected for all trial patients and costs collected in a subset of study patients; resource use collected for all trial patients and costs developed from administrative data sets; and prospective collection of cost data only (resource utilization would then be derived from the cost data). In economic evaluation of cancer clinical trials, the second and third strategies are the most com- mon. Some economic evaluation strategies actually omit measure- ment of resource consumption and concentrate instead on eco- nomic measures of resource use, such as hospital bills. These studies are undertaken when collecting resource consumption measures may not be feasible or may be prohibitively expensive, forcing the investigator to choose an alternative ‘‘billing’’ ap- proach. Generic Economic Evaluation Strategies Resource categories in clinical protocols must be specifically enumerated for both prospective and retrospective studies. The level of precision in the costing exercise is dependent upon the type of study being implemented. Example. In a trial of low-molecular-weight heparin (outpa- tient) versus intravenous heparin (inpatient), no difference is expected in treatment outcome; however, a large difference is expected in the utilization of inpatient hospital resources. A list of resource categories to be collected in the clinical protocol might include hospital days by unit type, as well as physician Visits, Example. In a comparison of hospital settings in the admin- istration of a SHT3 odansetron bolus push versus intravenous administration, it is necessary to microcost the administration of the drugs. Such an approach represents a higher level of preci- sion in the data than is required for the previous example. This study requires a microcosting exercise to track nursing time (and intensity) in providing antiemetic services to patients, with an effort to develop specific prices for these services. Example. In a trial of adjuvant therapy for breast cancer assessing four versus eight cycles of chemotherapy, there is similar outpatient use of drug administration, pharmacy charges, diagnostic testing, and professional fees. The difference may lie in the hospitalization utilization rate for febrile neutropenia and in outpatient care for nausea, mucositis, dehydration, etc. Thus, investigators must evaluate outpatient resource utilization and occurrence of hospitalizations. Inpatient Studies Inpatient studies are clinical trials that either occur within the hospital setting or enroll a high proportion of patients that are expected to receive hospital treatment during the course of the clinical trial. Examples include trials of new cytokine therapies as supportive care for bone-marrow transplantation, and studies of new surgical techniques for solid organ tumors, studies of intensive chemotherapy regimens administered on an inpatient basis but potentially having different effects on bone-marrow Journal of the National Cancer Institute Monographs No. 24, 1998 ''suppression. Data collection would focus on the intensity of treatment during the hospitalization, including tracking resource utilization by unit type within the hospital, physician visits, sur- gical procedures, diagnostic tests and procedures, and high-cost medications. Outpatient Studies Outpatient studies are clinical trials that either occur outside the hospital setting or enroll a high proportion of patients who are not expected to receive hospital treatment during the course of the clinical trial. Data collection would focus on the utiliza- tion of services on an outpatient basis, including physician ser- vices, nursing services, surgical procedures, diagnostic tests and procedures, and use of high-cost medications (including chemo- therapy). Information about acute-care hospitalization would be recorded. However, unless there were an explicit hypothesis about the intensity of the hospital services by treatment arm, data collection on use of resources within the hospital setting could be minimized. For example, data could be collected on length of stay by unit type but not on diagnostic tests or procedures or medication use. Subacute-Care Studies Studies of patients within subacute-care facilities (i.e., hospice care, nursing home care, or rehabilitation care) require informa- tion about the use of the subacute-care facility but often do not require information about other outpatient treatments. Informa- tion about acute-care hospitalization would be recorded. How- ever, unless there were an explicit hypothesis about the intensity of the hospital services with treatment, data on use of resources within the hospital setting could be minimized. Conclusion In summary, this section addresses some of the major issues in designing and implementing an economic evaluation within a clinical trial. Issues highlighted in this section include the po- tential direct medical and direct nonmedical resources that can be captured in a clinical trial, the assessment of productivity and intangible costs, the sources of data for capturing resource uti- lization information, the methodology for assigning unit costs to the resources collected in the protocol, and methods of refining data collection instruments for different types of clinical trials. There is no ‘“‘boiler-plate’’ economic evaluation. Clinical and economic investigators must review the requirements for the different data elements within their protocol. They must also assess alternative strategies for collecting cost information for the resources collected within the economic protocol. Part IV: Designing an Effective Data Collection Strategy Designing Data Collection Forms When integrating economics into an existing clinical trial de- sign, investigators should attempt to collect only those items that are necessary for the economic analysis. Existing forms, such as clinical trial report forms and flow sheets, often contain much of the required information (see Appendix I). It may be necessary to add only a few variables to collect sufficient information to conduct the study. Variables should include any subset of those Journal of the National Cancer Institute Monographs No. 24, 1998 listed below, depending on the type of trial and the study analy- sis plan. When collecting data centered around actual resource use, the CRFs used to collect the data must be designed and pilot tested prior to data collection. They should allow the investigators to record core areas of resource use required for the study. Specific forms are developed to track major resource use by study pa- tients. These forms often are divided into hospital resource use, outpatient resource use, other institutional resource use, care- giver support, and covariates. Hospital Resource Data The following are important data elements to consider for collection when patients are hospitalized during the clinical trial: * facility name and location; ¢ admission and discharge dates; * reason for admission (e.g., protocol treatment, elective admis- sion, and urgent admission); * hospital room type (e.g., ward, step-down, intensive care unit); * surgical procedure (type as well as duration of procedure); * transfusions of blood or blood products (date, type, number); ¢ major laboratory tests (date, type, number); * major radiologic tests (date, type, number); * medications (date, type, route, dosage); ¢ chemotherapy (dates, types, routes, dosage); ¢ radiotherapy (dates, fractionation, site, dosage); and ¢ physician visits (date, type, duration in minutes). Outpatient Resource Data The following are important data elements to consider for collection when patients are treated in an outpatient setting: ¢ date, type, and duration of visit; * type of facility; ¢ reason for visit (routine, elective follow-up, emergent follow- up): * surgical procedure (type and duration of procedure); * major laboratory tests (date, type, number); * major radiologic tests (date, type, number); ¢ medications (date, type, route, dosage); ¢ chemotherapy (date, type, route, dosage); ¢ radiotherapy (date, fractionation, site, dosage); * physician visits (date, type, duration in minutes); and * home care (dates/frequency of visits, skill requirements). Subacute-Care Institutional Resource Use The following are important data elements to consider for collection when patients are treated in subacute-care institutions: * nursing home (date of admission, date of discharge, level of care); and * hospice care (date of admission, date of discharge, level of care). Caregiver Support The following are important data elements to consider for collection when assessing caregiver burden: ''* caregiver burden (number of hours of paid and unpaid care- giver support): ¢ days of usual activity missed (work, school, home activities); * out-of-pocket costs; * transportation costs; and * caregiver quality of life. Covariates The following are important covariates to be collected within the economic trial: * comorbidities; * patient’s demographic characteristics; * geographic proximity of patient to site of care; * insurance coverage status; * caregiver status; * prior hospitalizations (within the past year); * employment status; ¢ disease severity; ¢ prior physician visits (within the past year); and ¢ cancer history (prior chemotherapy, prior radiation therapy, prior surgery, date of diagnosis). Collecting Data Beginning Data Collection There are essentially three approaches to collecting trial data on a prospective basis—1) collection at a fixed time period; 2) collection at the beginning of a designated medical event; and 3) collection after a treatment cycle or hospitalization. The strategy for data collection should be planned well in advance and will depend on the needs of the specific trial (for example, whether outpatient treatment is being compared with inpatient treatment). Collection at fixed time periods. Most economic studies query patients about resource utilization at fixed time periods— for example, on a monthly or quarterly basis. This method as- sures that economic data are not omitted in the study protocol and that follow-up is comparable across the study areas. This data collection strategy is also the best method to assess quality- of-life data for study patients. Starting from an event. Clinical trials that measure resource utilization from the time of a predesignated event often use toxicity as a marker. For example, a data collection form might instruct, ‘‘In the case of toxicity, please indicate (on another form) what resources were used to treat the toxicity.’’ The event- triggered method is limited in that a significant amount of re- source utilization may not be linked to toxicity. Starting after a treatment cycle or hospitalization. Data can also be collected at the end of a treatment cycle, such as a cycle of chemotherapy, or after a computed tomography (CT) scan or other procedure or test. This method provides reasonable certainty—if interviews are conducted at least quarterly—that all instances of resource utilization and the reasons for each will be captured. More frequent interviews may be required for sicker patients, though the data collection strategy should be standardized to keep costs low. 14 Why Collect Nonstudy-Site Data? Although nonstudy-site data are difficult to collect, the data do reflect resource utilization at sites outside the treatment cen- ter. The costs of these resources can be substantial when sig- nificant portions of care are provided in these settings. For ex- ample, bone-marrow transplant patients may receive much of their follow-up care at nonstudy sites. Obtaining Informed Consent Primary economic data collection that requires access to medical records or other confidential sources must have the clearance of the study subject. Trials that follow patients to nonstudy centers must develop more elaborate consent proce- dures to ensure that all patient records are included in the con- sent. In general, a consent form should be designed to encom- pass as many potential sources of information as possible, particularly if the investigators anticipate having to collect in- formation from outside the study center. The consent form should allow the study investigators access to the patients’ fi- nancial records at whatever sites the patients receive care. Medical resource-use data collection may involve abstraction by trial personnel or data from the patient’s medical records or medical bills. To obtain a patient’s medical bill in the United States, the principal investigator must include a release-of- information statement in the patient consent form. This can be a requirement for enrollment in the study or optional depending on the design of the study. Patient trial numbers should be substi- tuted for patient names to maintain anonymity for all confiden- tial data sources for analysis, especially on financial data. Strict confidentiality procedures should be applied to all economic data. Maintaining Communication Between Research Teams Most large, multisite clinical trials are monitored at the sites by a team of clinical research associates. These individuals play an important role in maintaining high data quality for both clini- cal and economic data. Often, they are more experienced at reviewing clinical data than economic data; therefore, it is usu- ally important to conduct training sessions at the beginning of the trial in both the general requirements of the economic study and the specific review that should be conducted. Those items for which source-document review can and should be done must be specified in advance; often the clinical research associate will be a good judge of what is feasible. This training must be fol- lowed by regular communication between the economic inves- tigators and the clinical research associate during the data col- lection period to correct problems as they occur. Conclusion In this section, we review in more detail the specific data elements that often are considered for inclusion in an economic evaluation. We further discuss procedural issues, such as the need for informed consent and for maintaining effective com- munication within the research team. Economic evaluation re- quires a comprehensive approach to the development of data collection instruments and a data collection strategy. This sec- tion presents a general framework for this assessment. Journal of the National Cancer Institute Monographs No. 24, 1998 ''Part V: Statistical Issues In this section, we aim to describe the unique characteristics of economic data as they are encountered in controlled clinical trials; we intend, furthermore, to identify a number of the sta- tistical complications associated with such data. We suggest a variety of statistical methods that might be used to address these complications. This chapter is not meant to be taken as a cook- book that offers a standard menu of tools that can be uniformly applied to any economic data collected in the course of con- trolled clinical trials. Rather, it is meant to alert trial planners and analysts to the statistical concerns that may need to be addressed in the design, implementation, and analysis of trials oriented toward economic issues and to provide some guidance to the rich and growing statistical literature that has been engendered by the increasing practice of collecting economic data alongside clini- cal trials. The Importance of a Statistical Analysis Plan The same care and rigor must be applied to the statistical analysis of economic data as is routinely applied to clinical endpoints in clinical trials. Detailed analysis plans are as impor- tant for economic evaluations as they are for clinical evaluations. The primary economic outcome should also be specified as closely as possible, as should covariates that are intended to be included in analysis for descriptive or predictive purposes. In- cluded in the study protocol should be a specification of the Statistical strategies that will be used. In deciding on appropriate statistical methods to use, such features of the data, such as its distributional characteristics, the degree and type of censoring, and other types of missing data, should be considered. A prop- erly conducted sample-size analysis should be conducted for economic data. Also, the analyst should consider whether eco- nomic data from a clinical trial alone are sufficient for address- ing the economic question of interest or whether such data must be supplemented with data derived from other sources or from modeling or extrapolation techniques. Many of the statistical issues that apply to clinical endpoints are also applicable to economic data. However, economic data are in some ways different and more complicated than clinical data, and additional issues may arise in regard to the analysis of economic data. These characteristics of economic data should be considered before deciding on a strategy for statistical analysis. Important Properties of Economic Data Anticipated Uses of Economic Data Clinical endpoints often are reported as ratio terms; for ex- ample, treatment A resulted in a 30% mortality reduction com- pared with treatment B. It is also possible to report costs such that average medical costs for treatment A were 20% greater than for treatment B. This approach may be appropriate when the only aim of the study is to test the null hypothesis of equal costs between two treatment arms. It is often anticipated, however, that the economic results of clinical studies will be used in economic analyses in which the absolute difference of cost is of interest. So, in a cost- effectiveness or cost-utility analysis, average medical costs for treatment A might be $10000 greater than cost for treatment B. The same point applies to the clinical outcome in cost- Journal of the National Cancer Institute Monographs No. 24, 1998 effectiveness and cost-utility analyses. That is, an estimate of absolute, rather than relative, benefit is required for the denomi- nator of the cost-effectiveness or cost-utility ratio. For example, patients receiving treatment A survived 2 years longer than pa- tients receiving treatment B. The anticipated use, in economic analysis, of a ratio estimate, such as the cost-effectiveness ra- tio—consisting of the ratio of the absolute cost difference di- vided by the absolute treatment effect difference—may have implications for the statistical design of a trial in which eco- nomic considerations are prominent. Economic analysis may be focused on estimating the eco- nomic impact of a treatment for a patient group or treatment setting that is not typical of the trial setting. In such cases, covariate analysis becomes essential. Because different covari- ates may be more important for economic effects than for clini- cal effects, it may not be possible to control for these covariates through such strategies as block randomization. Covariates that are of interest in economic analysis and may or may not be controlled for in clinical trials include those reviewed earlier, including sociodemographic variables (sex, age, race, immigra- tion status, income, education, occupation, employment status, census tract of residence, and marital status), comorbidity status, functional status, treatment setting prior to randomization, treat- ment setting before initial trial care, cost prior to randomization, availability of caretaker at home, availability of social support networks, type of health insurance, and structure of health care provider organization. The Complexity of Economic Data While a variety of clinical measures may be recorded in trials and used in complex ways in covariate analysis, primary out- come measures in clinical trials often are measurable by a single index, such as death from or recurrence of cancer, etc. By com- parison, economic data—not unlike quality-of-life data—tend to be complex. First, economic data usually are not measurable as a single event but as an accumulation of resource use over time. Second, economic data may consist of a variety of components that may, in turn, serve as substitutes or complements to each other (i.e., positively or negatively correlated). For example, costs may consist of the components of hospital-based costs, outpatient costs, and family costs. Costs may be equivalent for patients A and B, but patient B’s outpatient or family costs may have been substituted for hospital costs. Third, the longitudinal nature of costs present specific analytic issues. For example, if certain components of cost have a systematic tendency to be measured with less complete data in treatment arm B, and this is not recognized in the analysis, an incorrect inference could be drawn that costs are lower for B. Within these broad components of cost, cost is measured as a collection of finer components. For example, hospital costs may consist of the summation of many procedure or service compo- nents, such as operating room use, intensive care days, normal hospital days. The case is similar for outpatient care. Because cost components may be substitutes or complements, it is im- portant that collection of all relevant cost data be as comprehen- sive as possible. In addition, it follows from this property of costs that tests of statistical significance should not be performed on individual components of cost, nor should individual cost 15 ''components be excluded from the summation of costs because of statistical insignificance (46). The second way in which economic data are complex is that cost is the sum of products of resource use (quantity) and unit cost (price). Cost may not always be separated into its compo- nent quantity and price vectors, but often it will be, because there is great uncertainty about the validity of the price vectors that are embedded in the usual accounting systems available from hos- pitals and physician office records. When the quantity and price vectors that compose cost are obtained separately, the question of how to treat the separate and joint variability of these quantity and price vectors arises. Because economic data tend to be complex, the statistical analyst is more likely to have to deal with the problem of miss- ing data. This is due both to the variety and complexity of data elements necessary to measure economic variables and to the fact that economic data elements may be drawn from adminis- trative data systems that are not designed primarily for research purposes. Also, to the extent that some data elements are derived from questionnaires administered to patients or caregivers, miss- ing data will be a problem. Distributional Characteristics of Economic Data Cost data may have a relatively skewed distribution. An idea of what the distributional characteristics of cost data from cancer clinical trials might look like can be inferred from observational cost data from the SEER'—Medicare database (47). This data- base links records of incident cancer cases from NCI’s SEER system to claims data from the Medicare system maintained by HCFA. Tables 5 and 6 show the distributional characteristics of actual payments made by Medicare for treatment of colorectal cancer by stage of diagnosis, covering all costs from the date of diag- nosis to 6 months following those dates (only patients surviving at least 6 months from the date of diagnosis are included in the analysis). Table 5 shows data using charges submitted to Med- icare as a measure of cost, and Table 6 shows data using actual reimbursements paid by Medicare as a measure of cost. In all cases, these data do not fit the assumptions of normality, and for all cases except one—reimbursements for distant-stage dis- ease—the data tend to be more highly skewed and dispersed than data from an exponential distribution. Actual economic data encountered in clinical trials may be less skewed than the data displayed in Tables 5 and 6 because the population and treatments studied in the context of clinical trials may be more homogeneous. On the other hand, economic Table 5. Distributional characteristics of SEER/Medicare Data: submitted charges for colorectal cancer—cumulative charges to 6 months from diagnosis* Standard Stage Mean Median Maximum Skewedness Kurtosis deviation In situ $13009 $ 6072 $300670 4.68 40 $18 943 Local $23 821 $18765 $665 256 4.47 50 $23 237 Regional $29336 $22961 $730504 4.60 54 $25 909 Distant $33207 $26731 $525 191 4.25 43 $26 731 *Source: SEER—Medicare database, Applied Research Branch, National Can- cer Institute. 16 Table 6. Distributional characteristics of SEER/Medicare Data: Medicare reimbursements for colorectal cancer—Cumulative reimbursements to 6 months from diagnosis* Standard Stage Mean Median Maximum Skewedness Kurtosis deviation In situ $ 8529 $ 4244 $13599]1 2.74 18 $10 141 Local $16049 $16082 $384 138 3.12 61 $11 030 Regional $19707 $18605 $295 032 2.82 33 $11 297 Distant $21425 $20461 $105975 1.02 4 $12 221 *Source: SEER—Medicare database, Applied Research Branch, National Can- cer Institute. data encountered in clinical trials may be more skewed because, unlike the SEER—Medicare data, it may be derived from a wide variety of accounting systems. Also, the inclusion of additional components of cost, such as patient out-of-pocket or time costs, may introduce additional components of variance and skewedness. Dudley et al. (48) present another example of highly skewed cost data from a clinical trial of coronary artery bypass graft surgery. Statistical Methods for Analyzing Economic Data Descriptive Methods Mean costs are perhaps the most important univariable sum- maries, and in the absence of censoring problems, they can be simple arithmetic means. Means are important because their multiplication by population size is an estimate of total system costs, and the comparison of total system costs conditional on comparison treatments can have economic meaning for policy evaluation. However, when cost data are heavily skewed, mean costs may not be descriptive of typical patients, so median costs and other quantiles should always supplement the presentation. The standard deviation of costs may not be very informative, as its interpretation depends on normality of costs (one fre- quently sees illogical published figures such as ‘‘the mean cost was $10000 plus or minus $12000’’). An alternative set of summary statistics would be the mean, median, and quantiles such as 0.05, 0.1, 0.25, 0.75, 0.9, and 0.95. Graphical presenta- tion of the empirical cumulative distribution function of costs is even more informative. Univariable Methods Analysis of economic data without informative censoring. In some trials, the economic data of interest derives from a single event or a series of events unrelated to the subsequent flow of costs or survival. Also, cumulative costs over a fixed duration from the date of randomization are of interest, and observations on costs up to this date are complete. In cases such as these, there will be no informative censoring. A variety of statistical methods applicable to cost data are available in this case. Testing the entire distribution. As stated above, the differ- ence in mean costs will be of most interest. However, it is useful to conceptualize cost comparisons by comparing the entire cost distribution between treatment groups. The most general method for comparing distributions is the Kolmogorov—Smirnov test, which is based on the maximum absolute difference between Journal of the National Cancer Institute Monographs No. 24, 1998 ''two cumulative distribution function estimates. If the sample is large enough to make relative efficiency a nonissue, the robust Kolmogorov—Smirnov test would not have many competitors. However, for moderate to small studies, this general nonpara- metric test does not have as much power as tests directed at detecting differences in specific parameters of the distribution, such as the mean or median. Tests for difference on means, medians, and other param- eters. For directed tests, one can use either parametric or non- parametric tests. The latter may be preferred because 1) if nor- mality holds, nonparametric tests such as the Wilcoxon test still have 0.96 efficiency with respect to the optimal test (Student’r test ); and 2) if normality does not hold (the likely condition for cost data), nonparametric tests can be much more efficient than the Student’r test. For symmetric distributions, the Wilcoxon two-sample test addresses whether the mean or median cost differs between groups, because under symmetry the mean equals the median. For the general case in which the distribution may be asymmetric, the Wilcoxon test is still a valid test for differences in central tendency. As such, the Wilcoxon test is most sensitive in test of the pseudomedian, defined as the me- dian of midpoints of all possible pairs of costs. More important, however, the Wilcoxon two-sample test is a valid test of the null hypothesis that the probability that costs in group A exceed costs in group B, for randomly chosen pairs of patients, regardless of the distribution. The Wilcoxon statistic is a simple translation of an estimate of the probability that a random A patient has incurred costs ex- ceeding those of a random B patient. Thus, the Wilcoxon test has many advantages for univariable resource-use comparisons. Other nonparametric tests, such as the log-rank test, can also be considered. The log-rank test has the advantage of being a spe- cial case of the Cox semiparametric regression model, which may come into play when covariable adjustment is needed. When the log of costs is thought to be normally distributed, log-transformed comparisons testing for differences in the me- dian cost (equivalent to testing for differences in mean costs if the group variances are equal) have often been used. Zhou et al. (49,50) have developed a method, the Z-score test, to test for differences in means when group variances are not equal by recognizing that the log of the mean of the untransformed data is equal to the mean of the log-transformed data plus one half the variance of the transformed data. The Z-score test gives type I errors closest to nominal levels when variances differ between the two distributions being tested. The validity of parametric tests, such as the Z-score, depend on the correctness of the assumption about the distributional characteristics of the data, which may be difficult to establish with relatively small samples. One way to obviate this difficulty is to use tests whose validity does not depend on the distribution of the data, such as permutation f tests (5/) or bootstrap tech- niques (52). These approaches are preferred over the Z score when it is uncertain that the distribution of log-transformed cost data approximates a normal distribution. The disadvantage of permutation tests and the bootstrap technique is that these meth- ods generally are not available in the form of ‘‘canned’’ software packages and, therefore, require a moderate amount of custom programming by the analyst. The bootstrap technique is further described later in this section. Journal of the National Cancer Institute Monographs No. 24, 1998 Covariable Adjustment and Multivariable Modeling Methods In general, two reasons for using covariable adjustment in the context of a randomized clinical trial are to gain power to shorten the confidence interval (CI) for the treatment effect and to estimate costs and health effects for important subgroups when trial participants are heterogeneous. Some authors have found that for binary endpoints or censored failure time out- comes, covariable adjustment may not result in any appreciable gain in precision or power (53,54). However, in heterogeneous patient populations, estimating crude effect ratios (unadjusted odds or hazard ratios) can result in different estimates than ad- Justed estimates, and the crude estimates may not apply to any patient. Also, unadjusted models may lack fit. For example, the treatment may operate in a nonproportional hazards fashion without adjustment but satisfy the Cox model’s proportional hazards assumption with adjustment (55,56). When patients have a moderately wide prognostic spectrum, unadjusted tests (e.g., unstratified log-rank test) may have poor power in com- parison with adjusted tests (e.g., from the Cox model) (57). In economic analysis, the outcomes of interest are the absolute cost difference and the absolute effects of treatments. Even when power and precision are not issues, modeling multiple patient descriptors or prognostic factors is necessary when one wishes to translate relative treatment benefit into absolute benefit. It is easy to demonstrate that, even when relative treatment effects are constant across all types of patients, absolute effects vary when the variation of prognostic factors is moderate across en- rolled patients. The absolute survival advantage of a treatment is strongly dependent on initial survival in the absence of treatment as well as the relative survival advantage associated with the treatment.” Thus, covariable modeling is necessary in estima- tions of absolute effects of treatment on event probabilities or survival time that comprise the denominators of cost- effectiveness ratios, and covariables may also be needed to in- crease power. Similarly, for cost endpoints, one expects that covariable ad- Justment will almost always increase power, and it will also be useful for estimating cost distributions (or point estimates and confidence limits) for different types of patients. Regression models. Ordinary multiple regression currently is the most commonly employed regression model for analysis of costs. These models can be used to derive unbiased estimates, but when the distribution of cost data is highly skewed, the estimates may have high variance, and confidence limits for mean costs may not have the desired coverage probabilities. As discussed briefly above, one of the traditional methods for re- gression analysis of costs is to model the log of costs. For ex- ample, Rutten-van Moken et al. (58) use a log transformation of costs in their analysis of economic data after first applying a linear transformation to deal with the problem of zero costs. They perform regression analysis on the log-transformed costs and then use the Duan ‘“‘smearing’’ estimator (59) to retransform the regression-adjusted cost estimates back to the linear domain. This method may not be entirely satisfactory because the residu- als may still not be normally distributed, and the assumed mul- tiplicative relationship between patient variables and cost (on the original scale) may not be sensible. For example, analyses of data from a clinical trial on the cost of coronary artery bypass 17 ''graft surgery found that one would have to take sixfold loga- rithms of charges to achieve normality (48). An alternative to using log transformation models is to use robust regression techniques. The Cox semiparametric regres- sion model (60,6/) has demonstrated several advantages in analyses of resource-utilization data (48,62,63) characterized by a skewed distribution. Regression coefficient estimates for the Cox model are identical whether one analyzes costs on the origi- nal scale or takes any monotonic transformation of costs (such as the logarithm). Antilogs of regression coefficients are hazard ratios, but these can be converted to cost ratios (64). Once re- gression coefficients are estimated, the entire cost distribution, or the mean, median, or any other quantile of cost, can easily be estimated for individual types of patients. By assigning treat- ment as one of the predictors, one can readily estimate differ- ences in mean or median costs using the fitted model. As the Cox model is distribution-free for any single type of patient, there is no a priori assumption about how the mean and median are related mathematically. Model fit can be checked using stan- dard methods, i.e., smooth-scaled Schoenfeld residual plots for checking the proportional hazards assumption, and smoothed martingale residual plots or regression spline fits for determining linearity of predictors (64). Note that the Cox model in no way assumes that any censoring is present, so it can be used to analyze continuous response variables of many types.” It should be recognized that while the standard Cox model is often used to analyze clinical outcome data in the presence of censoring, this approach to censoring may result in bias in the cost realm (although the bias resulting from ignoring censoring may be even greater). This issue is discussed in more detail below. Analysis in the Presence of Informative Censoring Cumulative costs, which are the integral of the flow of costs over time, often are the economic data of primary interest in clinical trials. This flow may not be constant over time, and in many cases, the flow will increase beginning sometime prior to death. Observations of this cumulative flow from a trial may include patients with various times accrued since the time of randomization and may include some patients who have died, others who are close to death, some who will not die for many years, and some who are lost to economic follow-up at various times from randomization. Survival Techniques Standard techniques of survival analysis have been recom- mended for analyzing cost data in clinical trials in the presence of censoring. For example, Fenn et al. (62) have recently advo- cated the use of the Kaplan—Meier estimator for comparing costs in clinical trials. This approach has obvious appeal because it is parallel to the usual method of analyzing clinical survival in cancer trials. However, the scale of cumulative cost is not strictly equiva- lent to the scale for the clinical endpoint of time. Time has a constant rate of flow while cost does not. In addition, changes in the flow of costs are not random; rather, increases in the flow of costs tend to be indicative of death. Thus, when cost data are censored prior to death, this censoring is informative with regard to costs and survival. Even when no deaths are present in the 18 data, the different scales for cost and censoring can cause infor- mative censoring. A number of variations on the Kaplan—Meier estimator have been proposed to take this complication into account. Etzioni et al. (65) have proposed the Kaplan—Meier sample average (KMSA) estimator. For each interval, the KMSA estimates the sample average cost for a given time interval as the average cost of all individuals who were alive and observed at the beginning of that period. Total cumulative cost is then given as the weighted sum of all interval costs where the weights are the Kaplan—Meier estimates for the probability of survival up to that interval (since the true probabilities are not known because of censoring). This estimator works well when 1) cost data are available at frequent intervals; and 2) the censoring mechanism is independent of survival and cost in the sense that individuals are not censored because they appear to be at especially high or low risk of death. In addition, individuals are not censored be- cause of accumulation of especially high or low costs relative to uncensored cases. A variation on the KMSA estimator is the Pathwise estimator (66), which requires only data on cumulative costs for individu- als who have died (or reached some other ‘‘terminal’’ follow-up point, such that the flow of costs at that point are not correlated with the probability of death). The Pathwise estimator of average cumulative cost is the weighted sum of the average cumulative costs (of individuals who died) in each interval, where the weights are the Kaplan—Meier estimates for the probability of survival up to that interval. The Pathwise estimator does not depend on the availability of cost data for short intervals (data based on annual intervals can be used), and it yields an estimate of the distribution of cumulative costs as well as the mean. The efficiency of the Pathwise estimator is limited by the proportion of cases in which death is observed. A variant of the KMSA/ Pathwise approach has been used to estimate cumulative cancer treatment costs using Medicare data linked to the SEER tumor registry system by treatment phase (67). Efforts are currently underway to extend the KMSA/Pathwise approach to Cox re- gression (66). Etzioni et al. (68) use SEER—Medicare data com- bined with simulated censoring to demonstrate the bias in esti- mating costs when using standard Kaplan-Meier and Cox regression techniques and the improvement achieved by using the KMSA and pathwise methods. Etzioni et al. (65) has dis- cussed the bias that may be introduced in cost extrapolations using the KMSA approach when costs are discounted. Parametric survival models have also been advocated as a method for extrapolating costs beyond the endpoint of trial ob- servation (69,70). Fenn et al. (62) describes this approach using standard survival techniques. The results of parametric extrapo- lation may be very sensitive to the specification of the survival function and, within trial data, may be inadequate to allow strong discrimination between specifications. In this case, the variance introduced into cost predictions by model specification uncertainty should be recognized (77). Missing Data Several types of missing-data problems are likely to occur in relation to economic data collected in clinical trials. The types of missing data discussed in this section are different from the types of missing data that can be dealt with using the survival methods Journal of the National Cancer Institute Monographs No. 24, 1998 ''reviewed above. This section discusses data that are missing due to nonresponse or a failure to collect or record specific data items due to technical or logistic problems—not as a result of censoring inherent to the study’s time frame. Item Nonresponse Item nonresponse occurs when one or several of the elements of an outcome variable at a specific point in time are missing but when there is still a great deal of data available on the patient. For example, data on the number of physician visits for a patient might be missing for a specific week but be available for all other weeks; or unit cost data for a specific procedure may be missing from the accounting system of one study hospital but available at others. [tem nonresponse can occur at intermediate points in time, in which observations exist before and after the missing item, as well as at the point of a final outcome. Because item nonresponse implies that there is sufficient information on the patient already available, it is reasonable to include the pa- tient in the analysis despite the missing elements. It is acceptable to impute missing data when the analyst believes that the study will not be biased by imputing the missing values. Unit Nonresponse Unit nonresponse occurs when all or most of the information is missing on the unit of observation, such as all hospital days for a particular patient or all unit cost data for a particular study hospital. One type of unit nonresponse, in contrast to censoring caused by the time frame of the study, is the type of missing data not handled well by the survival methods discussed above (i.e., when the cost histories of patients who ‘‘drop out’’ are corre- lated to the reason for dropping out and are very different from patients who do not drop out). Imputation Methods for Missing Data Methods of imputation for missing unit data are reviewed by Little and Rubin (72) and Rubin (73), and specific application to health care data is described by Rubin and Shenker (74). Impu- tation strategies range from using the last observed value of the missing item or unit to using mean values of the missing item or unit to modeling the missing values using regression or simula- tion models that link the missing values to observed data. The appropriate imputation strategy depends on the analyst’s judg- ment regarding the nature of the missing data. If the missing data are believed to be missing completely at random (MCAR) (-.e., missing data are independent of observed data or unobserved data if the latter were known), rather simple methods of impu- tation are appropriate. If data is not MCAR, but the probability of missing is related only to observed data, then various struc- tural imputation models may be appropriate. Lavori et al. (75) apply Rubin’s method of multiple imputation to unit data miss- ing from a clinical trial due to patient nonadherence. They then compare the results to those obtained using the simpler last value approach and using only complete data records. Multiple impu- tation gives values of clinical outcomes that are between the extreme values of the other two methods and yields higher (more realistic) variances. Bootstrapping can also be used to compute variance estimates and CIs that take into account the increased variance resulting from having imputed missing data (76). Mc- Guigan et al. (77) compared the results of the two-stage Heck- Journal of the National Cancer Institute Monographs No. 24, 1998 man sample selection model and the results of inverse sample weights based on logistic regression to results actually obtained when additional follow-up data were obtained for a cohort study on a school-based drug prevention program. The weighting strat- egy performed better than the sample selection model and better than analysis of data without the additional follow-up. Perfor- mance of the sample selection model was very sensitive to the particular specification of the model used. When billing or cost information are not available for all patients, cost data can be collected on a subset of patients with resource utilization data being collected for the entire sample. Regression techniques can then be used to impute charges and predict bills. It should be remembered that the selected institu- tions from which bills are obtained are not necessarily represen- tative of other institutions. For example, institutions with the best billing systems may also have more efficient delivery care systems. Or, the billing systems upon which unit costs are im- puted may have higher unit costs, because they are less efficient and have less advanced billing systems. To prevent bias, re- source utilization must be compared across institutions. Although much time and energy are involved in collecting bills at every institution, it is recommended that an effort be made to obtain complete billing data for major resource utiliza- tion events when possible. The likelihood that the distribution of expenses for an institution with missing billing data are different than other institutions should be addressed. If a difference in re- source use is found, costs from the hospitals that seem to be driving those differences in resource utilization should be obtained. How should the collection of data for both resource consump- tion and pricing studies be handled for hospitals that may be in pricing consortia and will not disclose the prices of some re- sources or certain pharmaceuticals? A price may have to be assigned. For example, a military institution, which is a large accruer to a number of trials, will have a different price on almost every resource than a nonmilitary site. Access to tech- nology and length of stay may radically alter pricing information among institutions. Therefore, one may only be able to assign a reasonable across-the-board price in the estimate. This must be explicitly stated in the analysis. Speaking directly to Internal Review Boards (IRBs) concerning the financial use of the data, especially in terms of confidentiality, may assist in disclosure of needed information. Variance of Price and Quantity Vectors In some studies, direct measures of monetary cost will be available. In others, costs will be constructed by multiplication of price (unit cost) vector and a quantity (resource utilization) vector. In the latter situation, data on resource utilization often will be obtained from observations on study subjects, while data on the price vector will be obtained from data sources not di- rectly related to these observations. In this case, it will be dif- ficult or impossible to determine the variance of the price vector and the covariance between the price and quantity vectors. In this case, it is permissible to treat the price vector as if it were a parameter of fixed values. But, it should be recognized that this will result in an optimistic estimate of the variance of cost. One way to account for this is to perform a sensitivity analysis in which the values of the price vector are varied over a *‘plausi- bility interval.’’ Upper and lower bounds for the price values 19 ''might be determined from observational data, such as billing, reimbursement, and cost-accounting records of various health care providers or third-party payers. Statistical Issues Related to Cost-Effectiveness Ratios What Should Be in the Denominator? The typical quantity to use in the denominator of a cost- effectiveness ratio is the increase in expected (mean) life length, considering death due to all causes. There is nothing inherently wrong with using the increase in median life length, and this quan- tity requires less extrapolation and in some cases does not require parametric survival modeling. Another choice is the mean restricted life length, the area under the estimated quality-adjusted survival curve from zero until the end of the follow-up period of interest (78,79). This measure also eliminates the need for extensive ex- trapolation. Another choice for the denominator is the difference in t-year survival probabilities. Mark et al. (SO) compared many of these measures in one setting. Similarly, the denominator for a cost-utility ratio is the life length adjusted for quality. Variance of the Cost-Effectiveness Ratio When economic and clinical outcome data from clinical trials are used to construct a cost-effectiveness (or cost-utility) ratio, the statistical properties of this kind of ratio estimator should be appreciated. The full variance/covariance structure of costs and clinical outcomes will usually not be known. A variety of tech- niques have been developed to provide estimates of the variance of the cost-effectiveness ratio in this situation. These are re- viewed by Manning et al. (8/) and Polsky et al. (82,53). These techniques include univariate sensitivity analysis, multivariate sensitivity analysis, computation of confidence regions, and overall CIs using the delta method or simulation techniques. Naive use of univariate analysis will most often result in esti- mates of CIs that are unrealistically narrow, and multivariate sensitivity analysis may result in CIs that are too wide if only extreme bounds are examined. Manning et al. (8/) compare the results of univariate and multivariate sensitivity analysis, the delta method and Monte Carlo simulation using an empiric ex- ample of the cost-effectiveness of breast cancer screening, where cost-effectiveness is a function of nine epidemiological, techni- cal, and economic parameters. Application of the bootstrap method to obtain joint variance plots of the numerator and denominator of the cost-effectiveness ratio and to perform threshold analysis is described by Hlatky et al. (84). The bootstrap can also be used to simultaneously take into account other sources of variation such as missing data and model selection (85). Interpretation of the confidence interval from a bootstrap as- sessment is based on the proportion of times a result falls into each of the four possible quadrants of the cost-effectiveness plane, as well as an assessment of how often the cost- effectiveness ratio is below a prespecified threshold (e.g., $50000 per QALY gained). Fig. | provides an example of the results of a bootstrap pro- cedure that was used to estimate the confidence intervals for the comparison of costs and effects of a new therapy and usual practice. The x axis represents the difference in mortality be- tween the two therapies. Points to the right of the line represent- 20 BOOTSTRAP METHOD 9000 gous | 12 $50,000 6 +f 7 H+ + ot 3000 fF te + Tote 4 at +b 4 oe EP t+ 4, 104 e a + ei + tf HE HE + 2 +4 tag + O° | fot + S — -3000 8 cm Hoty. -6000 + tae Hehe 7 . + ea at + te Re ‘a -9000 | +ty aft 120 1+ + ian -12000 : : -0.20 -0.10 0.00 0.10 0.20 0.30 Costs>0: 122 Deaths Averted (Probability) osts>0: Deaths averted>0: 865 Fig. 1. Imputation of missing data. ing zero (1.e., no difference) indicate that the new therapy in- creased the probability of survival, while points to the left of the line indicate that the new therapy decreased the probability of survival. The y axis represents the difference in costs between the therapies. Points above the line representing zero indicate that the new therapy increased costs, while those to the right indi- cate that it decreased costs. The diagonal line represents the com- binations of changes in costs and effects that bear a ratio of $50 000 per death averted, one example of a prespecified level of cost- effectiveness that might be proposed to justify new investments. In quadrant I of the Fig. 1, the 104 points below and to the right of the line represent cases in which the new therapy has a ratio below $50 000 (1.e., satisfy the acceptability criterion), while the 12 points above and to the left of the line represent cases in which the new therapy has ratios that fail to satisfy the criterion. In quadrant III, the new therapy is less costly and less effec- tive. The 120 points below and to the right of the $50000 line indicate instances in which the use of the more expensive, more effective usual practice has ratios that exceed the $50000 ratio (i.e., the new therapy would save enough money to potentially justify the trade-off in health), while the 8 points above the $50 000 line indicate instances in which the adoption of more expensive, more effective usual care would have ratios that sat- isfy the criterion. In all, in 974 of the 1000 replications, the new therapy had a change in costs and effects that satisfied the $50000 criterion (1.e., given a two-tailed test, we can conclude that there was just over a 5% chance that the new therapy satisfied the acceptability criterion). While there is evidence that the therapy satisfied the $50 000 acceptability criterion, the statistical tests for the indi- vidual comparisons of cost and mortality differences between the two therapies indicate that there were no differences between the therapies at the 95% CI (in 122 of 1000 cases, the costs were increased by the new therapy, while in 878 they were decreased; in 865 of 1000 cases, mortality was reduced by the new therapy, while in 135 it was increased). Stinnett notes that the bootstrap method can also be used to correct for bias in the cost-effectiveness ratio that may be present when the ratio is based on the mean cost and effect estimates derived from relatively small samples (86). Journal of the National Cancer Institute Monographs No. 24, 1998 ''Sample-Size Analysis Since the outcome of interest for economic analysis is the absolute magnitude of cost difference, it is appropriate to con- duct sample size analysis from the perspective of gauging the likely precision of a CI (87,88). The results of economic studies from clinical trials may also be used to compute cost- effectiveness or cost-utility ratios. It is important, then, that clinical trials with important economic components be designed so that the clinical benefit, cost or resource utilization, and cost- effectiveness ratios can be estimated reliably. This can be ac- complished by determining the sample size that will yield ac- ceptably small confidence limits for the three quantities of interest (89). The limiting factor in most cases will be the pre- cision of the cost-effectiveness ratio, as this ratio inherits noise from both numerator and denominator, and imprecision in the denominator will cause great fluctuations in the ratio (8/). Other factors to consider in sample size analysis are the magnitude of the cost difference that is considered economically meaningful, the distributional characteristics of cost data, and the efficiency of estimation methods used to analyze cost data. A naive evalu- ation of cost data like those in Tables 5 and 6 might indicate that a few hundred subjects per arm are adequate to establish cost differences of a few thousand dollars with 95% confidence. But lower precision would be expected for a cost-effectiveness ratio, and precision would be expected to deteriorate because of nonnormality of the data and missing and censored data. Appli- cation of Fieller’s Theorem may be useful in estimating sample sizes based on confidence intervals for cost-effectiveness ratios (90). Because trials tend to be underpowered for some or all eco- nomic outcomes, an alternative approach to demanding strict adherence to conventional hypothesis testing is to establish a sample size that is sufficient for establishing that the specified economic outcome is true with some probability. For example, one could establish the probability that the cost-effectiveness ratio of a treatment is less than $20 000 per life-year saved. Defini- tive probabilities of effects such as this can only be achieved when the Bayesian posterior distribution is sufficiently sharp or tradi- tional confidence limits are sufficiently narrow (9/). Example. Distributions of charges and reimbursements for cancer treatment can exhibit strong positive skew and, thus, cannot be assumed to be normally distributed. A similarly shaped distribution with positive skew is the log-normal. While the log-normal may only provide a first-order approximation to the true underlying distribution of charges, reimbursements, or costs, it can in some limited examples provide a useful method to estimate sample sizes needed to test for differences in cancer treatment costs between two samples. To calculate sample size for tests of differences in costs, it is possible to convert everything so that calculations can be per- formed with normal distributions.* One usually generates a log- normal distribution by first generating the Zs (for some wu and a) and then exponentiating to get X = exp(Z). This is the reverse problem, in that the moments of the log-normally distributed Ys are given and it is desired to return to the Zs to conduct the power calculations. The following simplifying assumptions are made: the costs have a two-parameter log-normal distribution, and CV, = | Journal of the National Cancer Institute Monographs No. 24, 1998 (where CVy is the coefficient of variation or the standard devia- tion of X divided by the mean of X). We then solve following equality: sqrt{exp(20°) — exp(o")} = exp(o7/2), so that B can then be calculated as 8 = E(X)/exp(o7/2), with the result that p = In B. In Fig. 2, we fix Type I error level, a, as 0.05 (one-sided) and power, or I-B as, 0.80. The expected difference between groups, A, is specified as a percentage of the mean. Note that these calculations do not depend on a particular value for the mean but rather depend on the ratio of the standard deviation relative to the mean and A as a function of the mean. Price Adjustments Unit prices change over time and vary by geographic location. When trial costs occur over a significant period of time and/or multiple locations, price adjusters should be used to allow costs to be expressed in a common “‘base.’’ Even when trial costs are incurred over a short period of time and/or at a single location, adjustments should be made if costs are to be related to a dif- ferent time or place (e.g., update of 1990 trial costs to 1997 costs or extrapolation of trial costs conducted in Olmstead County, Minnesota, to the United States overall). Methods of adjusting prices are readily available (92) and include the medical component of the Consumer Price Index (CPI), which tends to overestimate overall inflation of health care costs, and the Fixed Weight Index (FWI) from HCFA, which results in lower estimates of inflation than the CPI. The FWI relies more on changes in the input prices of goods and services that go into the production of health care services. Two components of the FWI that may be of particular interest are the Prospective Payment System (PPS) input price index for hospi- tal costs and the Medical Economic Index (MEI), which mea- 2500 4 10% 2000 = 1500 | = 4 15% 1000 — am —j 20% 500 — 30% 0.8 0.9 1.0 1.1 1.2 CV Fig. 2. A case study of sample size requirements. Sample size required for a = 0.05 (one-sided), | — 8 = 0 for comparing costs between two samples. N refers to the combined sample sizes of the two equally sized groups. CV refers to the coefficient of variation. Lines indicate sample size need for A% differences over ranges of CV. (Unpublished figure and data provided by Kimberly McGuigan and R. D. Fricker at the Rand Corporation.) 21 ''sures changes in the cost of physicians’ time and operating ex- penses. Price adjustment has also been accomplished using the Geographic Adjustments Factors (GAF) for hospital wage costs and physician practice costs, including costs at the county level.° Discounting the Costs and Benefits of Cancer Care Economic theory indicates that assessments that look at costs and benefits of medical therapies over periods of time longer than a year should incorporate discounting. This concept em- bodies the fact that the costs and outcomes may occur at differ- ent points in time and that their value changes with time. Costs incurred and effects realized in the present are generally greater than costs and effects of similar nominal value incurred later—if not used in the present, the resources could otherwise be invested in the interim. Hence, it is necessary to express all costs and benefits at one point in time, usually the present. This is done by ‘“‘bringing back’’ all future costs and effects of a therapy and expressing them in terms of the present value. Conclusion Resource utilization and cost data collected within cancer clinical trials pose a series of important analytic challenges for economic investigators. These challenges stem from the need to analyze the data collected on a longitudinal basis within clinical trials as compared with dichotomous outcomes such as survival. Given the nature of data collected within economic studies, the potential for missing data is greatly increased over that usually seen in clinical trials without economic components. This sec- tion presents a state-of-the-art review of methodologic issues in economic evaluation of cancer clinical trials and highlights the analytic issues for investigators pursuing analysis of economic data in clinical trials. The section also services as an important reference to help all members of the economic evaluation team better understand the complexity of economic study design. Part VI: Economic Evaluation in the Cooperative Group Setting This section addresses practical issues that affect the ability of cooperative groups to collect data in multisite clinical trials. Because data will be gathered at different sites, clear and spe- cific arrangements must be made from the beginning of the trial regarding data collection processes. It is recommended that dis- cussion about an economic component—including how appro- priate, complicated, and extensive it is—should be part of the concept review of a trial. The following issues are relevant to the design of an economic study in a clinical trial. Are the Data Required? If resource utilization or cost data are included in a trial, they should have the status of required clinical data and be part of any monitoring and reporting system. But should the commitment to collecting resource utilization or cost data be required for patient registrations? The best way to address this problem is to link therapeutic trial registration to completion of ancillary outcomes such as the economic study (i.e., submission of resource data is designated as a formal eligibility requirement in the protocol). Most resource-utilization data required for economic analysis are routinely collected for clinical outcomes. A resource- 22 utilization strategy should not, therefore, be burdensome to clini- cal research associates (CRAs) or investigators and should not result in substantial biases in recruitment to the economic study. A separate consent for release of billing data may be required. If resource-utilization data are being collection, the patient can be informed in the main consent form that the medical charts also will be reviewed for cost outcomes. However, even in a single cooperative group trial, some cen- ters do not enter patients into every part of a trial. Some groups, for example, do not require that patients participate in ancillary studies (e.g., provide quality-of-life outcome data); a patient can still be eligible for the treatment component of the trial as long as the ancillary study is offered to the patient. This approach has the potential to bias the ancillary study outcomes, as physicians might only offer the quality-of-life study to a less ill subset of patients. Economic investigators are also concerned about this potential for bias. If it is not possible to include all patients, one compromise is to select a priori subsamples of patients; for example, include the first 500 patients randomized to the trial in the cost study (given that they meet the eligibility requirements). Data obtained with this approach can also be biased depending on historical cohort factors, such as the introduction of a new supportive care treatment midway through the trial. In this case, the cost-saving or cost-inducing effects will be missed. Who Will Collect the Data at Cooperative Group Institutions? CRAs collect and submit clinical data in most cooperative group settings. Will CRAs also collect the cost data, or should these data come directly from patient accounts offices at the institutions? It is also possible to collect cost data directly from patients. If collection of data from patients is necessary, this should be done by the local site and not the central data center (unless outside support is obtained for this effort, and then only with a communication linkage with the clinical investigator). Collection of data directly from patients by central office staff may also involve additional informed consent issues requiring local IRB approval. A combination of these data sources can be considered for cooperative group research. How Will Data be Collected, Monitored, and Submitted? The mechanism for data collection should be specified in the protocol. Part IV of this workbook presents a number of data collection sources for this effort. Forms represent the most com- mon method used in cooperative group trials, and they can be mailed, faxed, or electronically submitted to the coordinating center. In addition, data can be submitted to the central office on diskette or tape. A number of procedural issues must be speci- fied in the design of the economic study: Highlight up front the importance of monitoring and data quality. Set standards for data collection. Include all resource-utilization and cost data in the group’s system for monitoring data submission (e.g., “expectation sys- tems’’). This may require early input from programming staff. To ensure that the necessary data are collected, be certain that protocols are written clearly and that terms and procedures are defined precisely. Develop an analysis plan for the economic study. Journal of the National Cancer Institute Monographs No. 24, 1998 ''Example. Detailed data collection procedures should be de- veloped for each component of the protocol. An example of the issues to be addressed in developing these procedures follows for the collection of economic information from patients by telephone. Who will be the contact person? Is the contact person from the local hospital or a centralized location? Are sufficient staff available? Is there an informed consent issue? Does the clinical trials consent form require special permission for this data collection approach? Does it require the consent of the physician contact? How will confidentiality be monitored? Are there additional IRB issues to consider in economic studies? Does clinical trial medical chart access obtained in consent forms for clinical data needs cover the level of information required for economic stud- ies? At the very least, amend consent forms to include the cost objectives of the trial. If patient bills are evaluated, additional IRB approval may be required, depending on the local IRB. Speaking directly to IRBs concerning the financial use of the data especially in terms of confidentiality may assist in disclo- sure of needed information. Do existing quality control procedures implemented in coop- erative group trials address data quality issues for economic studies, particularly for billing or resource-utilization data col- lection from the patient? Economics investigators, coordinating center programmers, and statisticians should communicate early on about any particular features of economic data that may need to be incorporated into data entry verification systems and track- ing systems. Procedures for standardizing the collection of all economic data are necessary but this is particularly important for data obtained for care delivered ‘‘off-site’’ (e.g., at other hos- pitals, outpatient clinics, nursing homes, in-home). Data that require additional documentation at the institution level (e.g., data obtained through a telephone interview) and allowable source documentation must be clearly specified. Given these arrangements and approvals, the monitoring plan can incorporate early and ongoing review of the correctness of economic data as it is received at the coordinating center. This would allow changes to be made in the data collection process if appropriate data are not being collected. It is important to specify how long economic data will be collected and monitored. Training: An Important Quality-Control Procedure Training at cooperative group meetings is one way to achieve standardization in procedures for economic studies. Explanation of economic study objectives and definition of terms are impor- tant when new research outcomes are introduced in cooperative group trials. Roundtable sessions on various research topics are held for CRAs at cooperative group meetings, and these meet- ings could easily incorporate sessions on economic evaluation. These training sessions can also provide a vehicle for obtaining feedback on data collection forms for economic studies. Inves- tigator meetings, group meetings, or nurse/CRA training ses- sions can also provide an opportunity for reviewing data collec- tion forms. All of these meetings provide the opportunity to ask volunteer institutions to conduct pilot tests for the economic study forms and procedures. If possible, it is ideal to have sepa- rate training sessions for economic studies of cancer treatment Journal of the National Cancer Institute Monographs No. 24, 1998 protocols. However, time is very limited at these meetings, and sessions separate from the main training sessions may not be well attended. It is advisable to include at least limited training on economic studies during the regular training sessions at co- operative group meetings. Consider terminology in training efforts. Confusion can arise around semantics in collecting quantities of resource use. It is important for the staff to recognize that they will be collecting familiar information; for example, numbers of procedures and surgeries. Spreadsheets, made up before the economic study is implemented, should list all major categories of data collection for economic studies. These can be used to explain the economic study to the CRAs. Probably 90% of the data for the economic study are already being collected for the clinical study—but in a different format. Few additional data items really need to be checked. Unit price information, beyond hospital bills, usually is not obtained at the institutional level. If patient bills are required for the study, they could be sent from the hospital finance center directly to the data center, given that appropriate arrangements have been agreed upon (e.g., confidentiality safeguards). If resources allow, in-depth, on-site training can occur for CRAs at selected institutions (e.g., institutions where more de- tailed cost data might be collected). Cooperative group newslet- ters can provide another vehicle for training. It may be useful to add economic outcomes to the training provided for new study coordinators for cooperative group trials. A video tape is a useful approach for communicating economic study procedures and, if distributed to all participating institutions, can address staff turn- over occurring between group meetings. Small-Scale Pilot Studies Small-scale pilot studies for data collection forms can be con- ducted to develop standard data collection forms for economic evaluations and to identify data collection problems. These small-scale pilot studies would address feasibility issues at the institutional level, as well as statistical center issues (e.g., inte- gration into the current database system, extra programming requirements). Logs of time and effort required to use the forms and collect the data should be completed at institutions and statistical centers. Quantification of the extra effort involved, in terms of time and money (e.g., amount of time required to re- view charts, the source of the data required, and the cost in terms of staff time) can identify ways to reduce the data collection burden and to prepare budgets for future funding requests. Several questions should be addressed in a pilot study. An assessment of the acceptability of the data collection forms can be done to ensure that the forms will be completed. The wording and structure of the questions in the forms must be assessed to ensure that the desired information is obtained. Who Will Analyze and Share the Data? Sharing of Data For most cooperative groups, all data flow through a central statistical office. There are many safeguards to prevent the es- tablishment of ‘‘shadow’’ databases. The clinical trials group may insist that all data remain at the statistical center, with output required by the economics group generated by the central office. However, this may cause economic investigators some 23 ''concern. With the economics expertise in a separate environ- ment, biostatisticians at the statistical center probably do not have an analysis plan appropriate for economic outcomes or have access to the full extent of available related data. Even in this context, however, most statistical centers will prefer main- taining the database for the trial. One solution is to allow for sub- stantial input from the economists regarding data analysis. In these collaborative relationships, therefore, communication between eco- nomic investigators and group biostatisticians should occur early in the design of a trial to address appropriate statistical methods for the cost data and the use of clinical data for the cost analyses. Statistical center personnel often collaborate with outside economists, and some arrangements for data sharing may al- ready have been discussed. Also, there may be a person who has the appropriate expertise for these analyses at the coordinating center, and there will be no need to split up the information. It may well be the case that the economic investigators will need all of the data to integrate resource consumption for patients with specific clinical practice outcomes. If these arrangements can be agreed upon, it is important to establish some mechanism to make sure that both groups are given continuous and consistent data. Before the clinical study begins, a decision should be made concerning the amount and type of data that will be given to the economic investigators and the publications rules for the eco- nomic study. Decisions need to be specified clearly and within the constraints established for that cooperative group. If a decision is made to have economic data analyzed else- where, most cooperative groups will prefer to have all data sent to the statistical or coordinating center for forwarding to the economic investigators. If data are to be submitted directly to an economics center, the procedures for this separate submission should be clearly stated in the protocol. Data Analysis Issues Economic analysts should remain blinded until the last step in the analysis. This will avoid implicit conflict-of-interest issues and maintain the integrity of the overall trial. If data analysis involves the clinical data, the analysis ideally should not be done until the data are mature. Any use of major clinical endpoints will require waiting until data are mature. These decisions will be made by the data monitoring committee established for the particular trial or by arrangements established during the design phase of the trial. Intergroup Collaboration Issues Specific to Cost Studies It is generally unworkable to have one cooperative group execute and coordinate an ancillary study that is being coordi- nated by a different cooperative group because of the compli- cations associated with transfer of large amounts of data. There- fore, it is reasonable to expect that cost studies will usually need to be executed and coordinated by the group that is coordinating the therapeutic trial. There are, of course, potential difficulties associated with this policy: * negotiations of ‘ownership of economic data’’; ¢ no salary support for economists; *noncoordinating group often does not receive appropriate credit (i.e., publications as primary author for economic study results); and 24 ¢ awkward when the participating group may have more exper- tise in the area (quality of life, economics) than the coordinat- ing group. There are ways to encourage collaboration: the use of outside or separate funding (grants) to help support various areas of the trial; subcontracting of specific, external (to the group) staff to ob- tain required expertise; development of a liaison committee of economic experts among cooperative groups; attendance at each other’s group meetings; individuals that focus on specific sites (1.e., breast or lung) can attend those committees at other cooperative group meetings to monitor upcoming trials that might be candidates for an eco- nomic study; the standardization of data collection instruments for economic protocols across groups, whenever possible; and it is recommended that discussion about an economic compo- nent including how appropriate, complicated, and extensive it is should be part of the concept review of an intergroup trial. Quality Control Quality control issues become more challenging in the inter- group trial context because of the variation in procedures em- ployed by the different groups. Quality-control procedures sug- gested for single group trials are equally relevant to the inter- group context. In particular, training at the meetings held by each group would be very important to standardize data collec- tion procedures. Regular communication between intergroup staff associated with the cost study would help identify problems with forms and procedures and allow for improvements in pro- cedures. Intergroup Pilot Test Clinical and economic investigators across cooperative groups interested in a specific protocol could identify clinical research areas with important economic issues and identify trials planned in those areas; the cooperation of the Cooperative Group Chairs Committee would assist this process. Pilot work is particularly important for intergroup trials, given existing varia- tions in procedures. Each group could evaluate the acceptability of the data collection forms, particularly with respect to access to required information and the time required to collect the infor- mation. Quantification of the extra effort involved, in terms of time and money (e.g., amount of time required to review charts, the source of the data required, and the cost in terms of staff time) could document whether or not intergroup economic stud- ies were feasible. The intergroup pilot study could identify ways to reduce the data collection burden and to prepare budgets for future funding requests. Data Issues Information-Sharing The same questions addressed for single-group economic studies involving outside economic expertise apply to intergroup studies. Who will have access to cost (and clinical) data? What Journal of the National Cancer Institute Monographs No. 24, 1998 ''is the process for sharing data, and how will the information be copied and distributed to others? Which data are relevant to the analysis of economic and clinical parameters? How Will Economic Studies Be Funded in Cooperative Group Trials? The conduct of economic studies requires expenditure of con- siderable staff resources, both at the coordinating center and at individual institutions. Because these studies are new, case re- port forms, data collection procedures, data base preparation, and data analysis all require considerable start-up effort. Eco- nomic outcome data must be accorded the same importance as clinical data, meaning that, if collected, they are subject to the same strict quality control procedures that drive collection of clinical endpoint data. Funding is required to support this effort. Currently, we recognize the importance of including economic outcomes in selected clinical trials. Introduction of quality-of- life outcomes to cooperative groups is a case in point, where substantial central staff time for quality control and monitoring has been necessary (93). However, identifying funding sources Appendix I. Excerpt from Clinical Trial Case Report Form SOUTHWEST ONCOLOGY GROUP PROTOCOL S9509 A Randomized Phase Ill Trial of Paclitaxel Plus Carboplatin Versus Vinorelbine and Cisplatin in Untreated Non-Small-Cell Lung Cancer SPECIAL INSTRUCTIONS: RESOURCE UTILIZATION SUMMARY FORM The purpose of tracking resource utilization is to see if patients on one treatment arm require more medical resources than those on the other arm (i.e., to see if the treatment arms differ with respect this information from the billing office. to cost of care). We will be tracking medical resource use more intensively during the first 25 weeks postrandomization. We will monitor medical care between months 6 and 24, but less intensively. We have targeted medical resources that are particularly expensive, and for that reason, many commonly used tests or treatments may not be listed on the forms. Completion of the summary forms requires chart review. It may be necessary to consult physician progress notes, physician orders, and nursing notes. Since chart review is normally done at the end of each treatment cycle, we suggest that you collect information for the Resource Utilization Summary Forms each cycle. Funding has been provided for the extra chart review time required for the medical resource use study. If you are in a managed care setting, you may find it easier to obtain Il. SUPPORTIVE CARE MEDICATIONS/TREATMENT Were supportive care medications/treatment utilized during this period? CL] No { Yes 5HT3 Antagonists: (metoclopramide, rperazine, haloperidol, diphen-hydramide dexmethasone Bisphosphonates: etidronate, um (G-CSF/filgrastim, GM- CSF/. e Journal of the National Cancer Institute Monographs No. 24, 1998 25 ''that fit the specific timing and procedural constraints of coop- erative group research is a challenge. To date, several approaches to funding economic studies have been tried in cooperative group trials. Pharmaceutical industry funding has supported several economic studies in the coopera- tive groups: CALGB 9411 (an acute lymphocyte leukemia trial) and SWOG 9509 (an advanced-stage non-small-cell lung cancer trial) are examples. Third-party payer examples are less common, but one cur- rently in place supports an intergroup economics investigation in a metastatic breast cancer trial by U.S. Healthcare. The extent to which other large managed care organizations can be convinced to support these studies is yet to be determined. The NCI has two potential mechanisms for supporting eco- nomic studies: direct grants, and credits based on accrual to either treatment or cancer control trials. Through a request for applications, NCI is supporting trials of minimally invasive sur- gery that incorporate an economic component. An example is an intergroup trial (INT 0146) of open colectomy versus laparo- scopic-assisted colectomy for colon cancer. Pursuing the RO1 mechanism involves delays between the time applications are submitted and funds are awarded. The cooperative group will not be able to delay activation of a trial while funding for the economic component is being sought. Cancer control and treatment credits represent another poten- tial avenue for funding. Treatment credits translate into data management support based on accrual effort during the previous grant cycle. Cancer control credits are assigned to cancer pre- vention and control clinical trials following review and approval by the Division of Cancer Prevention (DCP) Cancer Control Protocol Review Committee (CCPRC). The process is analo- gous to the review of treatment protocols in the Cancer Treat- ment Evaluation Program. Credit is a measure of the data management burden needed to implement an intervention in a clinical trial. Most treatment clinical trials receive one credit/accrual. Credits for cancer pre- vention and control clinical trials vary depending on the nature of the intervention, the amount of data collection, and the length of follow-up. Community Clinical Oncology Programs (CCOPs) receive budgets based in part on their projected accrual as measured in treatment and cancer control credits. Cooperative groups serving as CCOP research bases receive ‘‘pass-through’’ finding for mem- bers and affiliates that accrue to cancer control clinical trials. To date, economic evaluation in cooperative group treatment studies have not received cancer control credits. Quality-of-life studies have received cancer control credit when they were stand-alone protocols or when the outcomes have been incorpo- rated into treatment trials. Economic outcomes could follow the same model, but this will require a reassessment of DCPC policy. Another possibility is award of additional treatment cred- its for registrations to companion economic trials or treatment trials with an economic outcome. As can be seen by this brief summary, routine funding mecha- nisms are not available to support economic studies in coopera- tive group trials. Access to stable funding for this research is a first step in our efforts to encourage cooperative groups to in- corporate economic outcomes data in cancer clinical trials. This section addresses specific issues related to the imple- mentation of economic evaluations within the NCI cooperative 26 group setting. The NCI cooperative groups are unique and spe- cialized research institutions with well-established protocols for collection of clinical data. However, economic evaluation is a new endpoint within cancer clinical trials for which investigators may have to modify their study procedures. Given the need for change, training and communication become key factors for suc- cess in the implementation of economic evaluation in the coop- erative group setting. Finally, given the current structure of funding for NCI clinical trials, economic evaluation often will require external funding arrangements, including peer-reviewed grants or industry support. Conclusion Economic evaluation of cancer clinical trials is a new and evolvy- ing discipline. As outlined in this workbook, implementation of economic evaluation requires a multidisclipinary research team and extensive planning and forethought analogous to the process of designing a study to assess the clinical endpoints of a clinical trial. Economic evaluation requires training of cooperative group per- sonnel as well as training of site personnel including physicians and the clinical research study team. Through this collaboration, eco- nomic data can be successfully collected within cancer clinical trials. References (1) Gold MR, Siegel JE, Russell LB, Weinstein MC, editors. Cost- effectiveness in health and medicine. New York: Oxford University Press, 1996. Drummond MF, O’Brien B, Stoddart GL, Torrance DW. Methods for the economic evaluation of health care programmes: second edition. New York: Oxford University Press. Parker SL, Tong T, Bolden S, Wingo PA. Cancer statistics, 1997. Cancer 1997;47:5-27. (4) Brown ML. The national economic burden of cancer. J Natl Cancer Inst 1990;82:181 1-4. Rundle RL. Salick pioneers selling cancer care to HMOs. The Wall Street Journal 1996;August 12, 1996, B1-B2. (6) Brown ML, Hodgson TA, Rice DP. Economic impact of cancer in the U.S. In: Schottenfeld D, Fraumeni J, editors. Cancer, Epidemiology, and Pre- vention. New York: Oxford University Press, 1996. Callahan D. Controlling the costs of health care for the elderly: fair means and foul. N Engl J Med 1996;335:744-6. (8) Levinsky NG. The purpose of advance medical planning: autonomy for patients or limitation of care? N Engl J Med 1996;335:741-3. Hillner BE. Potential evaluation of the incremental cost-effectiveness of paclitaxel in advanced non-small-cell lung cancer (Eastern Cooperative Oncology Group 5592). Monogr Natl Cancer Inst 1995;19:65—7. (10) American Society of Clinical Oncology. Recommended breast cancer sur- veillance guidelines. ASCO Breast Cancer Surveillance Expert Panel. J Clin Oncol 1997;15:2149-56. (11) American Society of Clinical Oncology. Clinical practice guidelines for the use of tumor markers in breast and colorectal cancer. ASCO Tumor Marker Expert Panel. J Clin Oncol 1996;14:2843-77. (12) American Society of Clinical Oncology. Recommendations for the use of hematopoietic CSFs: evidence-based clinical practice guidelines. ASCO Ad Hoc Colony-Stimulating Factor Guidelines Expert Panel. J Clin Oncol 1994;12:2471-508. (13) Peters WP, Ross M, Vredenburgh JJ, Hussein A, Rubin P, Dukelow K, et al. The use of intensive clinic support to permit outpatient autologous bone marrow transplantation for breast cancer. Semin Oncol 1994;21(4 suppl 7):25-31. (/4) Bennett CL, Westerman IL. Economic analysis during phase III clinical trials: who, what, when, where, and why? Oncology 1995;9(suppl):1—7. (15) Hillner BE, Kirkwood JM, Atkins MB, Atkins ER, Johnson ER, Smith TJ. Economic analysis of adjuvant interferon alfa-2b in high-risk melanoma based on projections from Eastern Cooperative Oncology Group 1684. J Clin Oncol 1995;15:2351-8. (16) Bochner F, Martin ED, Burgess ND, Somogyi AA, Misan GM. How can (2 (3 <= Nn (7 SS (9 Journal of the National Cancer Institute Monographs No. 24, 1998 ''(17) (18) (19) (20) (24) (25) (26) (27) (28) (29) (30) (31) (32) (33 (34) (35) (36) (38) (39) (40) (41) hospitals ration drugs? Drug rationing in a teaching hospital: a method to assign priorities. Br Med J 1994;308:901-5S. Smith TJ, Bodurtha JN. Ethics in oncology: balancing interests of the patient, oncologist, and society. J Clin Oncol 1995;13:2464—70. Eisenberg JM. Clinical economics: a guide to the economic analysis of clinical practices. JAMA 1989;262:2879-86. Bennett CL, Matchar D, McCrory D, McLeod DG, Crawford ED, Hillner BE. Cost-effective models for flutamide for prostate carcinoma patients: are they helpful to policy makers? Cancer 1996;77:1854-61. Hillner BE, Smith TJ, Desch CE. Efficacy and cost-effectiveness of au- tologous bone marrow transplantation in metastatic breast cancer: estimates using decision analysis while awaiting clinical trial results. JAMA 1992; 267:2055-62. Davies E, Clarke C, Hopkins A. Malignant cerebral glioma—I. Survival, disability, and morbidity after radiotherapy. Br Med J 1996;313:1507-12. Davies E, Clarke C, Hopkins A. Malignant cerebral glioma—II. Perspec- tives of patients and relatives on the value of radiotherapy. Br Med J 1996;313:1512-6. Drummond MF. Survey of cost-effectiveness and cost-benefit analyses in industrialized countries. World Health Statistics Quarterly 1985;38: 383-401. Drummond MF, O’Brien B. Economic analysis alongside clinical trials: practical considerations. The Economics Workgroup. J Rheumatol 1995; 22:1418-9. Schulman KA, Dorsainvil D, Yabroff KR, DiCesare J, Feltzer J, Glick HA, et al. Prospective economic evaluation of rhGM-CSF/rhIL-3 bone marrow transplant for leukemia and lymphoma (manuscript in preparation). Lerman C, Narod S, Schulman KA, Hughes C, Gomez-Caminero A, Bon- ney G, et al. BRCAI testing in families with hereditary breast-ovarian cancer: a prospective study of patient decision-making and outcomes. JAMA 1996;275:1885-92. Haioun C, Lepage E, Gisselbrecht C, Coiffier B, Bosly A, Tilly H, et al. Comparison of autologous bone marrow transplantation with sequential chemotherapy for intermediate-grade and high-grade non-Hodgkin’s lym- phoma in first complete remission: a study of 464 patients. J Clin Oncol 1994;12:2543-S1. Peters WP, Ross M, Vredenburgh JJ, Meisenberg B, Marks LB, Winer E, et al. High dose chemotherapy and autologous bone marrow support as consolidation after standard-dose adjuvant therapy for high-risk primary breast cancer. J Clin Oncol 1993;11:1132—-43. Feuer EJ, Frey CM, Brawley OW, Nayfield SG, Cunningham JB, Geller NL, et al. After a treatment breakthrough: a comparison of trial and popu- lation-based data for advanced testicular cancer. J Clin Oncol 1994;12: 368-77. Goldhirsch A, Wood WC, Senn HJ, Glick JH, Gelber RD. Meeting high- lights: international consensus panel on the treatment of primary breast cancer. J Natl Cancer Inst 1995;87:1441-5. Drummond MF, Jefferson TO. Guidelines for authors and peer reviewers of economic submissions to the BMJ. Br Med J 1996;313:275-83. Luce BR, Manning WG, Siegel JE, Lipscomb J. Estimating costs in cost- effectiveness analysis. In: Gold MR, Siegel JE, Russell LB, Weinstein MC, editors. Cost-effectiveness in Health and Medicine. New York: Oxford University Press, 1996. Schulman KA, Glick HA. Economic evaluation of high dose chemotherapy and BMT for patients with stage 2 and stage 3 breast cancer with greater than ten positive nodes at the time of surgical excision. J Natl Cancer Inst 1995;19:45-S0. Olivotto IA, Bajdik CD, Plenderleith IH, Coppin CM, Gelmon KA, Jack- son SM, et al. Adjuvant systemic therapy and survival after breast cancer. N Engl J Med 1994;330:805-10 Mauskopf JK, Schulman K, Bell L, Glick H. A strategy for collecting pharmacoeconomic data during phase II/III clinical trials. PharmacoEco- nomics 1996;9:264-77. Kukull WA, Koepsell TB, Conrad DA, Immanuel V, Prodzinski J, Franz C. Rapid estimation of hospitalization charges from a brief medical record review: evaluation of a multivariate prediction model. Med Care 1986;24: 961-6. Nefcy P. Challenge for health care: information for managing costs, utili- zation and quality. In: QQPHS 1992 Conference Proceedings. Atlanta: Institute of Industrial Engineers, 1992. Hsiao WC, Braun P, Dunn D, Becker ER. Resource-based relative values: an overview. JAMA 1988;260:2347-53 Federal Register. FR 58. Washington (DC): National Archives and Records Administration, 1993. Bennett CL, Cvitanic M, Pascal A. The costs of AIDS in Los Angeles. J Acq Immune Defic Syndr 1991;4:197-203. Edwards WS, Berlin M. National Medical Expenditure Survey: Question- naires and Data Collection Methods for the Household Survey and the Journal of the National Cancer Institute Monographs No. 24, 1998 (49) (50) (51) (52) (53) (58) (59) (60) (61) (62) (63) (64) (65) (66) (67) (68) (69) (70) (71) (72) (73) (74) Survey of American Indians and Alaska Natives, Methods 2. DHHS Publ No. (PHS) 89-3450. Rockville (MD): DHHS, 1989. Finkler S. The distinction between cost and charges. Ann Intern Med 1982;96:102-9. Finkler SA. Cost Accounting for Health Care Organizations: Concepts and Applications. Gaithersburg (MD): Aspen Publishers, 1994. UB-92 Editor: A Guide to Medicare Billing. Falconer C, editor. Reston (VA): St. Anthony, 1994. Red Book 1995. Drug topics. Montvale (NJ): Medical Economics Data, 1995, Potosky AL, Riley GF, Lubitz JD, Mentnech RM, Kessler LG. Potential for cancer related health services research using a linked Medicare-tumor reg- istry database. Med Care 1993;31:732-48. Coyle D. Statistical analysis in pharmacoeconomic studies: a review of current issues and standards. PharmacoEconomics 1996;9:506-16. Dudley RA, Harrell FE Jr, Smith LR, Mark DB, Califf RM, Pryor DB, et al. Comparison of analytic models for estimating the effect of clinical factors on the cost of coronary artery bypass graft surgery. J Clin Epidemiol 1993;46:261-71. Zhou XH, Gao S. Confidence intervals for log-normal mean. Stat Med 1997;16:783-90. Zhou XH, Gao S, Hui SL. Methods for comparing the means of two independent log-normal samples. Biometrics. In press. Edgington ES. Randomization Tests. New York: Marcel Dekker, 1986. Efron B, Tibshirani RJ. An Introduction to Bootstrap. New York: Chapman and Hall, 1993. Robinson LD, Jewell NP. Some surprising results about covariate adjust- ment in logistic regression models. Inter Stat Rev 1993;59:227-40. Schoenfeld DA. Sample size formulae for the proportional hazards regres- sion model. Biometrics 1983;39:499-503. Anderson GL, Fleming TR. Model misspecification in proportional hazards regression. Biometrika 1995;82:527-41. Ford IJ, Norrie J, Ahmadi S. Model inconsistency, illustrated by the Cox proportional hazards model. Stat Med 1995;14:735—46. Akazawa KT, Nakamura T, Palesch Y. Power of log-rank test and Cox regression model in clinical trials with heterogeneous samples. Stat Med 1997;16:583-97. Rutten-van Moken MP, van Doorslaer EK, van Vliet RC. Statistical analy- sis of cost outcomes in a randomized controlled clinical trial. Health Eco- nomics 1994;3:333-45. Duan N. Smearing estimate: a nonparametric retransformation method. J Am Stat Assoc 1983;78:605—10. Cox DR, Oakes D. Analysis of survival data. London: Chapman and Hall, 1984. Lin DY, Wei LJ. The robust inference for the Cox proportional hazards model. J Am Stat Assoc 1989;84:1074-8. Fenn P, McGuire A, Backhouse M, Jones D. Modeling programme costs in economic evaluation. J Health Economics 1996;15:115—25. Smith TJ, Desch CE, Hillner BE. Ways to reduce the cost of oncology care without compromising the quality. Cancer Invest 1994;12:257-64. Harrell FE Jr, Lee KL, DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 1996;15:361-87. Etzioni RD, Urban N, Baker M. Estimating the costs attributable to a disease with application to ovarian cancer. J Clin Epidemiol 1996;49: 95-103. Lin DY, Feuer EJ, Etzioni R, Wax Y. Estimating medical costs from incomplete follow-up data. Biometrics. In press. Riley GF, Potosky AL, Lubitz JD, Kessler LG. Medicare payments from diagnosis to death for elderly cancer patients by stage at diagnosis. Med Care 1995;33:828-41. Etzioni RD, Feuer RJ, Sullivan SD, Lin D, Hu C, Ramsey S. On the use of survival analysis techniques to estimate medical care costs. Working Paper, Fred Hutchison Cancer Research Center, Seattle, 1997. Cole BF, Gelber RD, Anderson KM. Parametric approaches to quality- adjusted survival analysis. Biometrics 1994;50:621-31. Cole BF, Gelber RD, Kirkwood JM, Goldhirsch A, Barylak E, Borden E. Quality-of-life—adjusted survival analysis of interferon alfa-2b adjuvant treatment of high-risk resected cutaneous melanoma: an Eastern Coopera- tive Oncology Group study. J Clin Oncol 1996;14:2666-73. Draper D. Assessment and propagation of model uncertainty. J R Stat Soc 1995;57:45-97. Little T, Rubin D. Statistical analysis with missing data. New York: John Wiley & Sons, 1987. Rubin DB. Multiple imputations for nonresponse in surveys. New Work: John Wiley & Sons, 1987. Rubin D, Shenker N. Multiple imputation in health-care data bases: an overview and some applications. Stat Med 1991;10:585—98. 27 ''(75) Lavori P, Dawson R, Shera D. A multiple imputation strategy for clinical trials with truncation of patient data. Stat Med 1995;14:1913-25. (76) Efron B. Missing data, imputation, and the bootstrap. J Am Stat Assoc 1994;89:463-79. (77) McGuigan KA, Ellickson PE, Hays RD, Bell RM. Adjusting for attrition in school-based samples: bias, precision, and cost trade-offs of three methods. Eval Rev 1997;21:554-67. (78) Karrison TG. Restricted mean life with adjustment for covariates. J Am Stat Assoc 1987;82:1169-76. (79) Karrison TG. Use of Irwin’s restricted mean as an index for comparing survival in different treatment groups—Interpretation and power consider- ations. Controlled Clinical Trials 1997;18:15 1-67. (SO) Mark DB, Hlatky MA, Califf RM, Naylor CD, Lee KL, Armstrong PW, et al. Cost effectiveness of thrombolytic therapy with tissue plasminogen activator as compared with streptokinase for acute myocardial infarction. N Engl J Med 1995;332:1418-24. (S81) Manning WG, Fryback DG, Weinstein MC. Reflecting uncertainty in cost- effectiveness analysis. In: Gold MR, Russell LB, Siegel JA, Weinstein MC, editors. Cost-effectiveness in health and medicine. New York: Oxford Uni- versity Press, 1996. (82) Polsky D, Glick HA, Willke R, Schulman K. Confidence intervals for cost-effectiveness ratios: a comparison of four methods. Health Economics 1997;6:243-52. (83) Willan AR, O’Brien BJ. Confidence intervals for cost-effectiveness ratios: an application of Fieller’s theorem. Health Economics 1996;5:297-305. (S84) Hlatky MA, Boothroyd DB, Johnstone IM, Marcus FI, Hahn EH, Hartz V, et al. Long-term cost-effectiveness of alternative management strategies for patients with life-threatening ventricular arrhythmias. J Clin Epidemiol 1997;50: 185-93. (S85) Chaudhary X, Mohammad A, Stearns SC. Estimating confidence intervals for cost-effectiveness ratios: an example from a randomized trial. Stat Med 1996;15:1447-58. (S6) Stinnett AA. Adjusting for bias in C/E ratio estimates. Health Economics 1996;5:470-2. (S87) Hornberger JC, Brown BW, Halpern J Jr. Designing a cost-effective clini- cal trial. Stat Med 1995;14:2249-59 (88) Borenstein, M. The case for confidence intervals in controlled trials. Con- trolled Clin Trials 1994;15:41 1-28. (S9) Borenstein M. Planning for precision in survival studies. J Clin Epidemiol 1994;47:1277-85. (90) Willan AR, O’Brien BJ. Confidence intervals for cost-effectiveness ratios: an application of Fieller’s Theorem. Health Economics 1996;5:297— 305. (91) Spiegelhalter DJ, Freedman LS, Parmar MK. Bayesian approaches to ran- domized trials. J R Stat Soc 1994;157:357-416. (92) Sensenig AL, Heffler SK, Donham CD. Health care indicators: hospital, employment, and price indicators for the health care industry—third quarter 1995. Health Care Financing Review 1996;17:269-79. 28 (93) Moinpour CM. Costs of quality-of-life research in Southwest Oncology Group trials. J Natl Cancer Inst 1996;20:1 1-6. (94) Smith TJ, Desch CE, Hillner BE. Efficacy and cost-effectiveness in on- cology: rational allocation of cancer care resources. J Natl Cancer Inst 1996;85: 1460-71. (95) Smith TJ, Hillner BE, Schmitz N, Linch DC, Dreger P, Goldstone AH, et al. Economic analysis of a randomized clinical trial to compare filgrastim- mobilized peripheral blood progenitor cell transplantation and autologous bone marrow transplantation in patients with Hodgkin’s and non- Hodgkin’s lymphoma. J Clin Oncol 1997;15:5—10. (96) Elit LM, Gafni A, Levine MN. Economic and policy implications of adopt- ing paclitaxel as first-line therapy for advanced ovarian cancer: an Ontario perspective. J Clin Oncol 1997;15:632-9. Notes 'Editor’s note: SEER is a set of geographically defined, population-based central tumor registries in the United States, operated by local nonprofit orga- nizations under contract to the National Cancer Institute (NCI). Each registry annually submits its cases to the NCI on a computer tape. These computer tapes are then edited by the NCI and made available for analysis. These indices are available from Carolyn S. Donham, Office of the Actuary, Health Care Financing Administration, telephone 410-786-7947. ‘For example, the absolute increase in 5-year survival probability, when a proportional hazards model holds, is S'—S. Here, S$ is the 5-year survival for control subjects, and r is the experimental control hazard ratio. This absolute survival difference is strongly influenced by S. If the survival distribution for control patients is exponential with a constant hazard rate / and proportional hazards model holds, the difference in mean life length due to treatment is w(/—-r)/r, where pt is the control group’s mean life length (//h). When one wishes to estimate absolute treatment effects for patients different from those enrolled, these effects will depend on prognostic factors. ‘The more general case is a three-parameter log-normal distribution, Y ~ LN(a«, B, o), with location parameter a, scale parameter B, and shape parameter o. The expectation is E(Y) = a + B exp(o’/2), and the variance, var(Y) = B? exp(o”) [exp(o*) — 1]. For the two-parameter log-normal distribution, the loca- tion parameter is assumed to be 0. *In limited experience thus far, the proportional hazards assumption has been remarkably well satisfied. A case study of the Cox model in estimating mean and median costs along with methods for checking model assumptions may be ob- tained from the World Wide Web: http://www.med.virginia.edu/medicine/ clinical/hes/heshp4.htm We thank the many people who participated in this project, including the writing committee, the workshop participants, and outside reviewers at the NCI and other institutions across the country. We also thank Joy Lopez for her assistance in coordinating this project and the Amgen Corporation for its gen- erous support of this activity. Journal of the National Cancer Institute Monographs No. 24, 1998 ''If you're not already subscribing, here’s what you're missing... The Journal of the National Cancer Institute remains the Number 1 journal in the field (/S/ Journal Citation Reports, October 1997), publishing the best original research papers in oncology from around the world. Internationally acclaimed as the source for the most up-to-date news and information from the rapidly changing fields of cancer research and treatment, the Journal of the National Cancer Institute is now available by subscription from Oxford University Press. Each issue is packed with extensive news about important cancer-related science, policy, politics, and people. Plus, editorials and reviews by experts in the field, book reviews, and commentary on timely topics. Become a subscriber, today. www.oup.co.uk/jnci/ And there’s more! Immediately upon subscribing, you will become a member of the NCI Information Associates Program, which includes these outstanding benefits: * One copy of each new Journal of the National Cancer Institute Monographs* * 24-hour access to PDQ®, the NCI’s comprehensive cancer information database ¢ Electronic Bulletin Board System Wa * ProtoCall™ Newsletter INSTITUTE ¢ Trained Member Service ; Representatives eww For more information about the Information Associates Program, call 1-800-624-7890 (+301-496-7600 outside the USA). *Free to members upon request prior to publication. I wish to subscribe to the i Journal of the National Cancer Institute. l Name (please print) Address Country i i I Zip/Post code TI enclosed the correct amount (check payable to Oxford University Press). I Please bill me. i Please charge my I J MasterCard visa TJ American Express Tpdiners JCB card. I Card number Pitti ti ttt ttt ttt i Exp. date Signature 1998 Subscription Rates volume 90 (24 issues) Individuals US & Canada UK & Europe Elsewhere *Institutional subscribers will now receive all monographs automatically as they are published with no need to order separately. If, however, you would prefer not to receive this automatic service, please contact your nearest Journal Subscriptions Department at Oxford University Press. Personal subscribers will have the opportunity to order monographs free of charge prior to publication. J One year: US$132 J One year: £127 J One year: US$180 Institutions US & Canada UK & Europe Elsewhere J One year: US$190* J One year: £170* J One year: US$240* Single issues are available upon request. Air-expedited delivery fee is included for subscriptions outside North America. Individual subscriptions must be paid for by personal check or credit card and sent to private addresses only. Canadian residents add 7% GST. To begin your subscription, call 1-800-852-7323 (USA & Canada). Or call 919-677-0977. Fax: 919-677-1714. Outside the Americas, ring +44 (0)1865 267907 or fax +44 (0)1865 267485. OXFORD JOURNALS DOMIPMINA| NVS| TIO LLV|MEA In the Americas, please return to: Oxford University Press, Journals Marketing (f), 2001 Evans Road, Cary, NC 27513, USA. Elsewhere, please return to: Oxford University Press, Journals Marketing (f), Great Clarendon St., Oxford, OX2 6DP, UK. Us pe imme some, me: Sm DS jm ij iT i Wa a i Si i a al '''' '''''' U. C. BERKELEY LIBRARIES COb0917401 ''