United States General Accounting Office Report to the Chairman, Committee on Science, Space, and Technology, House of Representatives May 1994 NUTRITION MONITORING Progress in Developing a Coordinated Program > - GAO/PEMD-94-23 '' ''KR A GAO United States General Accounting Office Washington, D.C. 20548 Program Evaluation and Methodology Division B-249872 May 27, 1994 The Honorable George E. Brown, Jr. Chairman, Committee on Science, Space, and Technology House of Representatives Dear Mr. Chairman: The United States has one of the most comprehensive nutrition monitoring programs in the world today. Data from the current monitoring activities, conducted primarily by the Departments of Agriculture (UsDA) and Health and Human Services (HHS), serve a multiplicity of users in government, academia, and private industry. However, several problems have been identified over the past two decades concerning the consistency, quality, and cost of the various nutrition monitoring activities. These problems eventually led to passage of the National Nutrition Monitoring and Related Research Act of 1990 (P.L. 101-445), which requires USDA and HHS to develop and implement a 10-year comprehensive plan for the National Nutrition Monitoring and Related Research Program (NNMRRP). NNMRRP is intended to enhance the benefits of current and future nutrition monitoring activities. You asked us to perform the following work: (1) an examination of current monitoring activities and planning efforts for NNMRRP, to include a review of statistical weighting issues associated with the 1987-88 Nationwide Food Consumption Survey (NFCS) data; (2) a definition of a model program or system, and viable options for such a program; and (3) a comparison of the methodological strengths and weaknesses and the potential costs associated with a model program and other viable options. This interim report covers the first component of the work requested by the Committee. It provides a descriptive overview of current monitoring activities, summarizes the major findings and recommendations of previous studies of nutrition monitoring activities, reviews the act and planning activities by HHs and usDA, and evaluates statistical weighting issues associated with the 1987-88 NFcs conducted by UsDA. Our discussion also considers the validity of the data released to the public from NFCcs, given concerns raised regarding the low rate of response (34 percent) to that survey. We plan to complete the other parts of our work and report on alternative approaches to nutrition monitoring early in 1995. Page 1 GAO/PEMD-94-23 Nutrition Monitoring '' Background B-249872 Nutrition monitoring is a critical governmental activity. Its principal goal is to accurately measure and survey the dietary and nutritional status of the U.S. population, as well as the quality, quantity, and safety of the food it consumes. Observing trends in the health of the population and linking nutritional intake to health outcomes are important elements of effective monitoring. Nutrition monitoring should provide information on a regular basis about the kinds and amounts of foods eaten by Americans; shifts in people’s knowledge about, and preferences for, certain foods (both of which influence food choices); the composition of the foods eaten, including their content of essential nutrients, as well as the presence of any contaminants that may affect food quality or safety; and the availability of food for consumption—which may in turn be affected by such factors as food production practices, commodity prices, and government farm support policies. These activities are clearly important in and of themselves, but the monitoring program also serves to provide data for several public policy uses. For example, data from the program are used in determining benefits in food assistance programs (such as food stamps). Data are also used to formulate national nutrition and health policies (such as national initiatives that seek to lower the fat content of diets or to educate the public about cholesterol), devise food labeling regulations (such as determining serving sizes and defining criteria for the qualification of nutrient content and health claims), and evaluate nutrition and health programs (such as studies to determine factors affecting participation in food programs and studies of the relationship of calcium intakes to increased risk of osteoporosis, hypertension, and colon cancer). Finally, the data are used to determine the adequacy and safety of the food supply. The FDA uses the data to assess the need for appropriate and safe levels of food fortification and the levels of dietary exposure to food additives and contaminants. In addition, pesticide residue estimates are calculated by the Environmental Protection Agency (EPA) using data from the monitoring program. Appendix IV contains more detail on how the data are used. Over 70 different federal data collection activities, developed over the past six decades, presently comprise the nation’s nutrition monitoring, surveillance, and research activities. However, three nationwide surveys constitute the heart of the program: the HHS National Health and Nutrition Examination Survey (NHANES), the UsDA Nationwide Food Consumption Survey (NFCS), and the uspa Continuing Survey of Food Intakes by Individuals (csFi1). A series of NHANES studies have been conducted since Page 2 GAO/PEMD-94-23 Nutrition Monitoring '' B-249872 1971 to collect health and nutrition data through direct physical examinations and interviews of individuals. NHANES III, the largest and most recent survey, is being conducted in two 3-year segments over the period 1988-94. NFcs, conducted every 10 years, was last conducted in 1987-88. It is designed to collect data on household food use and individual food consumption. csFi, while originally intended to be an annual supplement to NFcs data, has been administered twice (1985-86 and 1989-91), with the most recent survey started in 1994. (See table I.1.) Principal Findings Difficulties With the Current Set of Activities Even though a considerable amount of information is provided by current federal nutrition monitoring activities, they do not constitute a well-integrated system. They are, instead, a kind of patchwork of federal activities that have evolved over a 60-year period. Integrated approaches that specify measures and collect data to answer the public policy needs of federal, state, and local governments are not currently in place. Those activities that do exist have not been jointly planned so as to collect consistent and comparable data. The agencies involved in nutrition monitoring have been criticized in the past for not coordinating their data collection activities. Our review of the literature also revealed specific concerns that included the cost of redundant activities, the difficulty of comparing data across surveys, and data gaps (for example, the lack of information on some population subgroups that are at high risk for nutrition-related health problems). In addition, issues concerning data quality, response rates, frequency of data collection, timeliness of reporting, and dissemination of nutrition monitoring information have been raised. These problems have led to a reexamination of the relationship between the NHANES and USDA surveys, and the collection of data from other federal nutrition monitoring activities, so that these data will be most helpful to users and amenable to integration into the new NNMRRP. The 10-Year Plan As already noted, the National Nutrition Monitoring and Related Research Act required that a 10-year comprehensive plan be developed for establishing and implementing the coordinated NNMRRP. This plan was published in June of 1993 and is currently the centerpiece of HHS and USDA Page 3 GAO/PEMD-94-23 Nutrition Monitoring '' B-249872 planning efforts. It lays out a broad set of activities that are important and necessary for addressing known problems, and when implemented, such planned activities should go a long way towards achieving the goals and objectives for improving the nutrition monitoring system. The plan, however, is weak in several respects. It does not establish priorities and does not provide a framework for maintaining, deleting, or adding new monitoring activities. The plan relies largely on what already exists without placing monitoring activities in a new coordinated context and critically evaluating which activities are essential, which should be eliminated or modified, and which new ones are needed. In addition, no attention is given in the plan to assessing the likely costs and the feasibility of implementing monitoring activities. Further, many of the specific planning details normally present in a comprehensive plan are deferred to future planning by working groups or committees. In sum, the plan lacks important ingredients essential for the successful implementation of a new NNMRRP designed to resolve the problems of the past. On the positive side, the enactment of the legislation and the development of the 10-year comprehensive plan have led to a number of agency activities that address at least some of the identified weaknesses in nutrition monitoring. (See table 1.) Perhaps most encouraging is the fact that coordination between the two major departments involved, HHS and usDA, has improved in recent years, with the establishment of better collaboration and communication as well as mechanisms to facilitate continued improvements. Not only did the agencies successfully work together in drafting the plan, but the creation of the Interagency Board for Nutrition Monitoring and Related Research has also provided a forum for discussion, review, and implementation of NNMRRP. The Interagency Board, for example, has recently proposed options for prioritizing planning activities set forth in the 10-year plan and prepared reports on the progress of the coordinated program. Furthermore, various workshops have also been conducted that have linked federal with state officials and with other nutrition monitoring users from the academic and industry sectors, and additional studies have been undertaken to assess survey sampling designs and methodologies as well as improve compatibility between data collection and reporting activities. Page 4 GAO/PEMD-94-23 Nutrition Monitoring '' B-249872 BAAN LLAMA MENTE AIS ALLIAGE ALLELE LLL LE LALLA ALE LILLE DLA LD, Table 1.: Criticisms of Nutrition Monitoring and Agency Responses Criticism of nutrition monitoring Agency response Lack of coordination among nutrition Interagency Board and Working Groups created* and progress made towards monitoring activities coordinating the activities of HHS and USDA Lack of compatibility in methods for Common methods being assessed by HHS and USDA for gathering information on assessing dietary intake dietary intake Specific population groups not covered by NCHS initiated study to identify and evaluate design approaches for sampling major surveys population subgroups in the next NHANES; supplement planned for CSFII (1994-96) to increase collection of dietary intake information on infants and children Specific geographic areas not represented State-based surveillance systems expanded in recent years as a source of some by major surveys geographically-specific data; CSFII (1994-96) includes Alaska and Hawaii, which were previously excluded from the NFCS and CSFII surveys Reporting by national surveys not Revised directory of federal and state nutrition monitoring activities published in 1992; integrated first chartbook of selected findings from NNMRRP published in 1993; and third scientific report on the dietary and nutritional status of the U.S. population currently being prepared under contract (guidelines for reporting dietary intake data developed by NCHS/HNIS working group and being used by agencies in preparing data for the report) Core set of standardized measures not yet Working group on comparability developed common set of population descriptors in developed for major surveys 1992 (which were incorporated into the CSFII 1994-96 questionnaires); two other working group efforts begun to assess similarities and differences of survey questions on nutrition and nutrition-related health, as well as on nutrition knowledge, attitudes, and behavior; CDC/NCHS also funded study to evaluate what core nutrition and health indicators to include in NHANES and to determine the feasibility of applying a core unit in alternative settings, such as nursing homes, schools, and so on Compatible sampling techniques not used Study of NFCS/CSFIl and NHANES sampling designs conducted in 1991; NCHS and for national surveys HNIS funded contract to consider ways in which the next implementations of NHANES and CSFIl can link sampling Information needs of users not Interagency Board provides forum for identifying federal agency data needs; workshop systematically determined convened by HNIS to review the objectives of the NFCS/CSFIl surveys and, in part, to determine whether the surveys meet the needs of users;° in planning NHANES, NCHS formally solicited input from various federal agencies and other researchers on which topics to include in the survey Data not collected continuously CSFIl intended to be operated continuously; to date, CSFII has been conducted at periodic intervals (next series to begin 1998) Improvement needed in methodology for Several workshops and studies conducted to assess the tradeoffs and the strengths assessing dietary intake and nutritional and weaknesses of different methods (for example, the appropriate number and type of status recalls to use) Need for more timely dissemination of NCHS released NHANES III, Phase | (1988-91) data on topics of public health survey information importance (for example, cholesterol levels of the population) beginning in 1993; HNIS has automated certain data collection and processing activities, which may help speed release of the CSFII 1994-96 data Sampling problems with NFCS HNIS has separated the household and individual portions of the NFCS survey in order to reduce respondent burden and improve response rates (NFCS will become the Household Food Consumption Survey, scheduled to be conducted in the late 1990s, and CSFIl will provide individual level data); in addition, HNIS’signed interagency agreements with the Bureau of the Census for assistance in designing and conducting the individual and household surveys, as well as for research on improving methods for collecting household food use data (continued) Page 5 GAO/PEMD-94-23 Nutrition Monitoring '' B-249872 Criticism of nutrition monitoring Agency response State and local data needs not fully addressed Through state-based CDC/NCCDPHP surveillance systems, CDC assisting states in collecting some nutrition-related data through program records of participants in maternal and child health programs, as well as telephone interviews with randomly selected residents : *Three formal working groups on survey comparability, food composition data, and federal-state relations and information dissemination were established in 1989 and operate under the guidance of the Interagency Board for Nutrition Monitoring. They meet regularly through the year to provide oversight for implementing planning activities in the 10-year plan and to facilitate better communications and coordination among agencies. Research Triangle Institute, “Sampling Designs and Population Descriptors of Nationwide Food Consumption Surveys and National Health and Nutrition Examination Surveys,” report prepared for HHS and USDA, July 1991. °“Report of a Workshop to Review the Nationwide Food Consumption Survey Conducted by USDA,” seminar convened by HNIS at USDA headquarters, Washington, D.C., July 29-30, 1991. Important methodological problems remain, however. Many geographic areas and specific population groups are not covered by the major surveys. Data collection and reporting by national surveys remain to be integrated. No core set of standardized measures has yet been developed for the major surveys, and other measures are still not interlocking across the three national surveys. There is still a need for compatible sampling techniques and for data that are collected continuously. A further weakness continues to exist in the methodologies for assessing dietary intake and for measuring the nutritional status of the nation’s population. In addition, state and local information needs are not yet being fully addressed in survey designs. Finally, we found that the methodology used to design weighting equations for the 1987-88 NFcs data would have been technically correct (that is, within the constraints of standard survey analysis) if the data had been missing at random. However, with no nonresponse data available, it is impossible to know whether the missing data are randomly distributed. Therefore, there is no way to truly test the accuracy of the weighting, which must therefore remain uncertain until data are collected on the characteristics of the nonrespondents. Warnings on the data are provided by uspA, and backup documentation is delivered upon request to users. However, given that the data from the last NFCS may be problematic, we believe that they should be used as a last resort and only after all other usable data sources have first been identified. Several major users of the data have decided not to use the 1987-88 nFcs data. For example, EPA could not reliably estimate dietary Page 6 GAO/PEMD-94-23 Nutrition Monitoring '' B-249872 exposure to pesticide residues using the NFcs data because the sampling error ranged from 70 percent to up to 175 percent of the estimate for various subpopulation groups. A telephone survey we performed of NFCS users revealed a general awareness of the data problems as well as a conservative approach to their use. (See appendix IV.) Conclusions We conclude that (1) a coherent, consistent system or program for nutrition monitoring is not yet in place, and (2) although the current 10-year plan reflects some progress in planning for a program, several important aspects of the plan are incomplete. We therefore believe the Committee should continue to closely monitor the development of NNMRRP in order to ensure its success. USDA and HHS provided written comments on a draft of our report. (See appendixes V and VI.) Officials from these departments agreed in general with our principal findings and conclusions. They did, however, provide additional detail about activities under way within their departments that they believe reflect further progress in meeting the goals of NNMRRP. We have incorporated these comments in the report where appropriate. As agreed with your office, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days from its date of issue. At that time, we will send copies to interested congressional committees and government agencies, and we will make copies available to others upon request. If you have any questions or would like additional information, please call me at (202) 512-2900 or Kwai-Cheung Chan, Director of Program Evaluation in Physical Systems Areas, at (202) 512-3092. Major contributors to this report are listed in appendix VII. Sincerely yours, Eleanor Chelimsky Assistant Comptroller General Page 7 GAO/PEMD-94-23 Nutrition Monitoring '' Contents Letter Appendix I Overview of the National Nutrition Monitoring System Uses of System Data Nutrition Monitoring Activities 10 10 11 Appendix II Previous Reviews of the Nutrition Monitoring System Six Expert-Panel Evaluations GAO Reports 18 18 23 Appendix III Recent Legislative Changes The 1990 Legislation 26 26 Appendix IV The 1987-88 Nationwide Food Consumption Survey: Quality and Uses of the Data Background Impact of Nonresponse on Dietary Data From the 1987-88 NFCS Uses of the 1987-88 NFCS Data Users’ Awareness of 1987-88 NFCS Data Limitations Results of Telephone Survey Using the Weighted Data From the 1987-88 NFCS 29 29 29 30 32 32 34 Appendix V Comments From the Department of Agriculture 37 Page 8 GAO/PEMD-94-23 Nutrition Monitoring '' Contents Appendix VI 44 Comments From the Department of Health and Human Services Appendix VII 51 Major Contributors to This Report Tables Table 1.: Criticisms of Nutrition Monitoring and Agency 5 Responses Table I.1: Principal Federal Nutrition Monitoring Activities 12 Abbreviations CDC Centers for Disease Control and Prevention CSFII Continuing Survey of Food Intakes by Individuals DHKS Diet and Health Knowledge Survey EPA Environmental Protection Agency ERS Economic Research Service FDA Food and Drug Administration GAO General Accounting Office HHS Department of Health and Human Services HNIS Human Nutrition Information Service NCCDPHP National Center for Chronic Disease Prevention and Health Promotion NCHS National Center for Health Statistics NFCS Nationwide Food Consumption Survey NHANES National Health and Nutrition Examination Survey NNMRRP National Nutrition Monitoring and Related Research Program NNMS National Nutrition Monitoring System USDA Department of Agriculture Page 9 GAO/PEMD-94-23 Nutrition Monitoring ''Appendix I Overview of the National Nutrition Monitoring System Uses of System Data Nutrition monitoring refers to a broad range of activities designed to periodically and systematically assess the dietary and nutritional status of the American people, the conditions in this country that affect the nutritional status of individuals, and the relationship between diet and health. The primary nutrition monitoring activities consist of assessments of dietary and nutritional status obtained through three nationwide surveys. Supporting activities include continuous updating of food composition data, as well as research on human nutritional requirements and nutritional assessment methods. The current set of activities identified as nutrition monitoring, often referred to as the National Nutrition Monitoring System (NNMS), provide data for a broad range of goals and purposes. ! These cover problem identification, policy making and program planning, program evaluation, and related research areas. Data from NNMs have been used to develop the Dietary Guidelines for Americans and the nutrition and related health objectives included in Healthy People 2000, as well as to evaluate progress towards achievement of the 1990 Health Objectives for the nation.” Other uses of data from the system include the development of the Recommended Dietary Allowances and the identification of areas of nutrition research that are needed to increase the knowledge base and revise standards pertaining to human nutrient requirements. Data from NNMs are also important for defining the prevalence of nutrition-related health problems, developing practical program strategies, and determining what changes, if any, are occurring over time following the implementation of program interventions. For example, the National Institutes of Health have used NnMs data in establishing clinical and population guidelines for the detection, evaluation, and treatment of hypertension and high cholesterol, as well as in assessing what progress has been made in lessening both of these risk factors following the creation of the National High Blood Pressure and Cholesterol Education Programs. Similarly, the Occupational Safety and Health Administration and the Environmental Protection Agency (EPA) have used NNMs data in ‘Since enactment of the National Nutrition Monitoring and Related Research Act of 1990 PL 101-445), the system has been called the National Nutrition Monitoring and Related Research Program (NNMREP). 2USDA and HHS, Nutrition and Your Health: Dietary Guidelines for Americans, 3rd ed. (Washington, D.C.: 1990); and HHS, Healthy People 2000: National Health Promotion and Disease Prevention Objectives (Washington, D.C.: Public Health Service, 1990). Page 10 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix I Overview of the National Nutrition Monitoring System Nutrition Monitoring Activities establishing regulations aimed at combating elevated blood lead levels in the population and in tracking the extent to which reductions have occurred. Furthermore, NNMS data have been analyzed by the Food and Drug Administration (FDA) to identify how, and what types of, food should be fortified with nutrients such as iron, zinc, and vitamin A. In addition, nutrition monitoring data have been important in developing the Thrifty Food Plan, which forms the basis for determining benefit levels for food stamp recipients, and in evaluating various UsDA food assistance programs (for example, determining the factors that affect program participation, as well as the extent to which participation affects food consumption and expenditures).° NNMs also provides information to estimate the impact that consumer demand and spending have on commodity supplies and prices, which in turn are important factors in the management of government farm commodity policies and programs. Furthermore, food consumption data are used by FDA to estimate dietary exposure to food additives, toxicants, and contaminants, as well as by EPA to estimate dietary exposure to pesticide residue levels. FDA also uses the data to develop food labeling regulations, such as establishing the reference standard for determining serving sizes for nutrition labeling purposes and defining the criteria for the qualification of nutrient content and health claims. Beyond federal needs for information, state and local governments require access to the data on food consumption to aid them in the areas of resource allocation in public health policy, budget justification in programming, identification of problems for legislative or regulatory intervention, and determination of evaluation needs. Scholarly research is also conducted with the data to better understand the relationship of diet to health, as well as to understand the relationship between food supply and demand. NNMs is defined to include all data collection and analysis activities of the federal government associated with five traditional categories: nutritional and health status measurements, food consumption measurements; food composition measurements and nutrient data banks; dietary knowledge, behavior, and attitude assessments; and food supply and demand determinations. 3Further discussion of the Thrifty Food Plan is presented in appendix IV. Page 11 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix I Overview of the National Nutrition Monitoring System The Directory of Federal and State Nutrition Monitoring Activities lists more than 70 separate survey, surveillance, and research activities conducted by 22 different agencies of the federal government charged with covering these areas.‘ (See table I.1.) Only about one half of these activities, however, represent specific monitoring of food and nutritient content, food consumption, nutritional status, or relationships between diet and health or food supply and demand. Of the major ongoing nutrition monitoring activities described in table I.1, there are three national surveys that make up the core of NNMs: (1) the Nationwide Food Consumption Survey (NFCSs), (2) the Continuing Survey of Food Intakes by Individuals (csFil)—both sponsored by Usba—and (3) the National Health and Nutrition Examination Survey (NHANES) supported by HHS. Table 1.1: Principal Federal Nutrition Monitoring Activities Nutrition monitoring component Monitoring activity Agency Description 1. Nutritional and health status National Health and Nutrition Examination Survey (NHANES) HHS/CDC/NCHS Nationwide data gathered on the health and nutritional status of the population through physical examinations, clinical and laboratory tests, and traditional survey methods; third NHANES currently underway and scheduled for completion in 1994 Pregnancy Nutrition Surveillance System HHS/CDC/NCCDPHP/states Participating states use data on low-income, high-risk pregnant women who participate in government nutrition and food assistance programs, in order to monitor nutrition-related problems and behavioral risk factors associated with low birth weight Pediatric Nutrition Surveillance System HHS/CDC/NCCDPHP/states Participating states use data on low-income, high-risk children who participate in government health, nutrition, and food assistance programs, in order to monitor nutritional status among children (continued) ‘Prepared by the Interagency Board for Nutrition Monitoring and Related Research, HHS Publication No. (PHS) 92-1255-1 (1992). Page 12 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix I Overview of the National Nutrition Monitoring System Nutrition monitoring component Monitoring activity Agency Description 2. Food consumption Nationwide Food NFCS/USDA/HNIS Information collected from Consumption Survey (NFCS) nationwide sample of households and individuals within households on food consumption behavior (including where foods are purchased and consumed and what they cost) and the nutritional content of diets; NFCS conducted every 10 years since 1936 Continuing Survey of Food ~USDA/HNIS Designed to supplement NFCS and Intakes by Individuals collect on a more regular basis (CSFIl) information on food consumption and nutritional content of diets; CSFII conducted twice since the mid-1980’s Total Diet Study HHS/FDA Key foods are purchased from stores and restaurants, prepared for consumption, and then analyzed to determine nutrient and contaminant levels in the food supply and in representative diets of specific population groups National Health and HHS/CDC/NCHS Dietary intake information on Nutrition Examination individuals collected as part of Survey (NHANES) overall NHANES survey design 3. Food composition and nutrient data National Nutrient Data Bank USDA/HNIS Data compiled from various sources on the nutrient composition of foods; used as a basis with food consumption data (from NFCS, CSFIl, and NHANES) to estimate nutrient intake Food Label and Package HHS/FDA Survey of retail packaged foods Survey conducted to monitor nutrition labeling practices 4. Dietary knowledge, behavior, and Diet and Health Knowledge ~USDA/HNIS Telephone survey conducted to attitudes Survey, follow-up to CSFIl assess individuals’ knowledge and attitudes about dietary guidance, food safety, and other food and nutrition issues Health and Diet Survey HHS/FDA Telephone survey conducted to assess public knowledge, attitudes, and practices concerning food and nutrition as they relate to health problems Behavioral Risk Factor HHS/CDC/NCCDPHP/states Participating states conduct Surveillance System telephone surveys to assess personal health practices that are related to leading causes of death; optional modules included for the assessment of dietary fat and for fruit and vegetable consumption (continued) Page 13 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix I Overview of the National Nutrition Monitoring System Nutrition monitoring component Monitoring activity Agency Description 5. Food supply and demand Food and Nutrition Supply © USDA/ERS/HNIS Used to estimate the levels of foods Series and nutrients available for consumption in the U.S. food supply by deducting data on exports, year-end inventories, and nonfood use from data on production, imports, and beginning inventories The Nationwide Food Consumption Survey (NFCS) The first national survey of household food consumption and dietary levels, called the Consumer Purchases Study, was conducted jointly in 1935-36 by several federal agencies. Since this survey, six national surveys (with different names) have been conducted by usDA, roughly every 10 years. The early surveys—in 1935-36, 1942, 1948 (urban only), and 1955—were designed to measure food used by the household as a whole, the costs of that food, and the dietary levels of household members. The 1965-66 survey, called the Household Food Consumption Survey, was the first to include data collection in all four seasons and on food intake by individual members of households as well as on food used by the household as a whole. Collection of information on the intake of individuals was included because of the emerging interest in diet and health. The 1977-78 and 1987-88 Nationwide Food Consumption Surveys (NFCS) continued to include both components: (1) household food use where the household food manager is asked to recall, with the aid of a food list, the kinds and amounts of food used from home food supplies during the previous 7 days and the cost of those foods; and (2) individual intakes where each household member is asked to recall the kinds and amounts of foods eaten at home and away during the previous day and to keep a record of the food eaten on the day of the interview and the following day (1-day recall/2-day record). NFCS was a multistage, stratified area probability sample that targeted households in the 48 contiguous states and individuals residing in those households. In 1977-78 and 1987-88, NFcs included two samples: a basic sample of all households and a low-income sample of households with incomes at or below 130 percent of the poverty threshold—a level consistent with eligibility for the Food Stamp Program. In 1987-88, the basic sample was 4,589 households and the low-income sample was 2,584 households. The basic sample included 10,172 individuals. USDA has replaced NFCs with two separate surveys—the Household Food Consumption Survey (planned for the late 1990s) and the Continuing Page 14 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix I Overview of the National Nutrition Monitoring System Survey of Food Intakes by Individuals (csFi1), which is described in the next section. Continuing Survey of Food Intakes by Individuals (CSFII) CSFII was initially designed to supplement data from NFCs annually. However, it has only been conducted in 1985-86, 1989-91, and is currently underway for 1994-96 (as the third in a series). The survey has now replaced the individual intake component of NFcs. The surveys provide information on diets of individuals in the United States, the diets of population groups of concern such as the low-income population, and an indication of changes taking place in the dietary status of Americans. The 1985-86 survey targeted persons (women aged 19 to 50 and their children aged 1 to 5, and men aged 19 to 50) in households with all levels of income as well as a separate sample with low income. The 1989-91 survey was redesigned to provide data in 3-year time periods and targeted all-income households and low-income households, as well as individuals of all ages and both sexes. Both surveys used multistage, stratified area probability samples. The first study included the collection of six 1-day recalls at about 2-month intervals during a 1-year period. The first 1-day recall was collected with an in-person interview; subsequent interviews were done by telephone whenever possible. The second survey included the collection of 3 days of intake data. In both surveys, respondents were asked to recall the kinds and amounts of foods eaten at home and away from home during the previous day. In the second survey, respondents were also asked to keep a record of foods eaten on the day of the interview and on the following day (1-day recall and 2-day record). Both surveys used the Nutrient Data Bank, developed by HNIS, to derive nutrients ingested by individuals. The Diet and Health Knowledge Survey (DHKS) was also initiated in 1989 as a follow-up to the csFu. It measures attitudes and knowledge about diet — and health among Americans. DHKS and CSFII together are designed to examine relationships between individuals’ attitudes and knowledge about food and nutrition and the same individuals’ food choices and nutrient intakes. The 1994-96 csFII/DHKS differs from the 1985-86 and 1989-91 surveys in several important ways. Specifically, it features a target population of individuals in all 50 states, rather than the 48 contiguous states; Page 15 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix I Overview of the National Nutrition Monitoring System the collection of 2 nonconsecutive days of food intake, rather than 3 consecutive days (as in 1989-91), or 6 nonconsecutive days as in 1985-86; the use of two in-person 24-hour recalls, rather than 1-day recall/2-day record as in 1989-91 or a combined in-person/telephone 24-hour recall for 6 days as in 1985-86; oversampling of the low-income population, rather than a separate low-income survey; a larger sample in selected sex-age categories—specifically, young children and elderly; subsampling within households, rather than the collection of information from all members of a household; collection of DHKs data from adults aged 20 and older, rather than from only main meal planners/preparers; and additional questions on attitudes and knowledge about using food labels. The National Health and Nutrition Examination Survey (NHANES) Since 1960, the National Health Examination Survey has collected data through interviews and direct physical examinations. Since 1971, when a nutrition component was added, the survey has been called the National Health and Nutrition Examination Survey (NHANES). The survey is conducted under the direction of the National Center for Health Statistics, which is part of the Centers for Disease Control and Prevention (cDc). NHANES III, the current, most recent, and largest survey, is the seventh in the series of surveys using a sample of approximately 40,000 individuals in communities throughout the country. The major goals of NHANES III are (1) to estimate the national prevalence of selected diseases, risk factors, and health conditions; (2) to assess the health and nutritional status of the nation’s population, as well as specific population subgroups, and estimate changes over time; and (3) to provide information on the interrelationships of health and nutrition variables. Survey participants are randomly selected. The interview includes demographic, socioeconomic, dietary, and health-related questions. Dietary intake data are collected from individuals using a single 24-hour recall.° The physical examination component consists of medical and dental examinations, physiological measurements, and laboratory tests administered by medical personnel. Examinations and interviews are conducted in specially-equipped mobile examination centers that travel to survey sites throughout the country. 5Nonrandom replicate recalls are also collected to adjust population distributions of nutrients, and two additional 24-hour recalls were collected by telephone for all examined persons aged 50 years and older in 1989-91 to estimate usual dietary intake in older persons. Page 16 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix I Overview of the National Nutrition Monitoring System The survey team consists of a physician, a dentist, medical and health technicians, and dietary and health interviewers. A large staff of interviewers conduct the household interview. The sample for the survey is selected to be representative of the U.S. population aged 2 months and older. In order to produce reliable statistics for children, the elderly, blacks, and Mexican Americans, these groups are oversampled for the survey. Data collection for NHANES III began in September 1988 and will conclude in 1994; the survey team will have traveled to 88 locations across the country by the time they complete their data collection. The survey is being conducted in two 3-year segments, with data collected and analyzed at the end of the segments as well as for the full survey. The National Center for Health Statistics (NCHS) is planning the next NHANES to begin in 1997. Page 17 GAO/PEMD-94-23 Nutrition Monitoring ''Appendix IT Previous Reviews of the Nutrition Monitoring System There is a substantial literature on nutrition monitoring, including six comprehensive studies by expert panels: an evaluation by a panel of the National Academy of Public Administration on improving the Health and Nutrition Examination Survey (1980); a study of food consumption data sources by the National Research Council (1981); a report on national survey data on food consumption by the National Research Council (1984); a progress report from the Joint Nutrition Monitoring Evaluation Committee, Department of Health and Human Services (HHS) and Department of Agriculture (USDA), (1986); a report on approaches to assessing nutrient adequacy by the National Research Council (1986); and an update report on nutrition monitoring prepared by the Life Sciences Research Office, Federation of American Societies for Experimental Biology, for HHS and UsDA (1988). In addition, the General Accounting Office (GAO) has reviewed the nutrition monitoring system many times and made a number of recommendations in consequence. We summarize these reports, including their findings, in the next two sections. Six Expert-Panel Evaluations National Academy of An evaluation panel of the National Academy of Public Administration Public Administration focused attention on improving the Health and Nutrition Examination (1980) Survey (now NHANES). Published in 1980, many of their conclusions and recommendations are still relevant today.! The study emphasized the importance of NHANES as the only means by which strictly standardized physical examinations of a representative sample of the population are conducted. However, the evaluation panel pointed to the essential need for a core set of standardized measures that would be repeated in every national population survey for planning health services and allocating public and private resources to health programs, facilities, and education. Once the need for standardized measures had been met, the panel concluded that other parts of the survey could assess selected conditions of special national interest, which might change from survey to survey. The evaluation found that the survey should be repeated every 5 years, with the midpoint coinciding as closely as possible with that of NFCS. Additionally, during the interval between the national population surveys, 'National Academy of Public Administration, Panel to Review the Health and Nutrition Examination Survey, Improving the Health and Nutrition Examination Survey: An Evaluation by a Panel of the National Academy of Public Administration (Hyattsville, Md.: HHS/NCHS, 1981). Page 18 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix II Previous Reviews of the Nutrition Monitoring System the panel recommended that one or more surveys should be conducted of special population subgroups, such as ethnic, geographic, or age groups. (This recommendation was partially fulfilled by the Hispanic Health and Nutrition Examination Survey conducted in 1982-84.) The report stressed the importance of interlocking data on the nation’s health, nutrition, diet, and food utilization between the Health and Nutrition Examination Survey and nFcs. Therefore, the panel recommended that comparable planning, scheduling, sampling, field procedures, and coding would render the Health and Nutrition Examination Survey data and the NFCs interview data complementary for given socioeconomic and demographic groups. Finally, the evaluation panel stressed the need for care in both operations and reporting to assure reliable and valid data that would be released within 12 to 15 months of the conclusion of the survey, with preliminary high priority data being released before fieldwork was completed. National Research Council (1981) The Food and Drug Administration (FDA) contracted with the Food and Nutrition Board of the National Academy of Sciences to develop recommendations for integrating sources of data on food consumption with other data on nutrition and health status. A Committee on Food Consumption Patterns was formed to study existing data sources and design a system to meet the needs of FDA and other agencies involved in monitoring the food consumption and nutritional and health status of the population. The committee concluded that available information from different sources was not adequate and could not be integrated because of differences in sampling designs, data collection methods, and measures. : They recommended a proposed system that would include continuous collection, processing, and analysis of food consumption data; a review of the precision of food identification and coding in the collection of food consumption data; the identification and transfer of existing sources of health status data to the proposed system, the use of other aggregate government and commercial databases to supplement information needed for an overall monitoring system, and interagency discussions to develop a coordinated system. 2National Research Council, Committee on Food Consumption Patterns, Assessing Changing Food Consumption Patterns (Washington, D.C.: National Academy Press, 1981). Page 19 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix II Previous Reviews of the Nutrition Monitoring System National Research Council (1984) In response to a request from the Human Nutrition Information Service (HNIS) of USDA (and supported by NCHS as well), a Coordinating Committee on Evaluation of Food Consumption Surveys was established under the auspices of the Food and Nutrition Board in the National Research Council’s Commission on Life Sciences. The purpose of the review was to consider ways in which food consumption data from NFCS and NHANES were used and to make recommendations on survey design that would facilitate wider application of survey data. The report found that both surveys were important to a multiplicity of users in government, the academic community, and industry, and concluded that the present system of two separate national surveys should continue.’? The committee recommended that, although the two surveys should continue to collect dietary intake data, a common methodological core should be developed and implemented. Furthermore, the National Research Council reported that the two surveys could be better linked through compatible sampling and common population descriptors. Finally, the review concluded that the NFcs Individual Dietary Intake component and NHANES should be redesigned as continuous survey processes with continuous data reporting to ensure timely data release and reporting. They recommended the Household Food Use components of the NFCS should continue on a regular, intermittent basis, unless future study demonstrated that some other design (for example, continuous basis) would be more advantageous. Joint Nutrition Monitoring Evaluation Committee (1986) The Joint Nutrition Monitoring Evaluation Committee was established as a federal advisory group (sponsored by USDA and HHS) to report on the nutritional status of the U.S. population. The committee examined the importance of different food components in the diet and their relation to nutritional and health status, relying heavily on the dietary and health measurements derived from the NFCS and NHANES data. It was the first time that a systematic effort was made to integrate and interpret data from these two major surveys.* . The committee found that differences between the surveys concerning sample design and population descriptors made comparability difficult 3National Research Council, Commission on Life Sciences, Food and Nutrition Board, National Survey Data on Food Consumption: Uses and Recommmendations (Washington, D.C.: National Academy Press, 1984). 4HHS, Public Health Service, NCHS, and USDA, Food and Consumer Service, HNIS, Nutrition Monitoring in the United States: A Progress Report From the Joint Nutrition Monitoring Evaluation Committee (Hyattsville, Md.: July 1986). Page 20 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix II Previous Reviews of the Nutrition Monitoring System and also that more valid methods for measuring dietary intake and health were needed to improve assessments of nutritional status. The committee further concluded that data gaps existed for several key food components in the diet, which prevented adequate assessments of their importance for public health. The committee was also charged with issuing recommendations to improve the National Nutrition Monitoring System. Such recommendations were made in four general areas, as follows: improve the information exchange between data users and gatherers by establishing a means to learn more about the information needs of users and how nutrition monitoring data are used, and by increasing the identifiability and availability of nutrition monitoring information for users; increase the use of existing data collected under the system by conducting more in-depth analyses for policy making and program management, providing better documentation of data files, improving the comparability of data so that information from different data sources can be integrated (for example, by developing “core questionnaires,” compatible sampling schemes, and similar definitions of terms), and improving the timely release of data; improve the methods and techniques for gathering information for assessing nutritional status by expanding efforts to study the factors that influence nutritional status, improving coverage of low-income population groups in the monitoring system, developing nutrition indicators to monitor changes in food consumption and nutritional status, and increasing research to improve methods for assessing dietary intake and nutritional status; and finally, increase resources for the monitoring system to implement the foregoing recommendations. National Research Council (1986) USDA asked the National Research Council to develop criteria and approaches for using survey data on dietary intakes (particularly the NFcs data) in order to estimate nutrient adequacy in the U.S. population. The study, undertaken by the Subcommittee on Criteria for Dietary Evaluation of the National Research Council, examined various methods used by USDA and other researchers for assessing nutrient adequacy. The subcommittee 5Subcommittee on Criteria for Dietary Evaluation, Coordinating Committee on Evaluation of Food Consumption Surveys, Food and Nutrition Board, Commission on Life Sciences, National Research Council, Nutrient Adequacy: Assessment Using Food Consumption Surveys (Washington, D.C.: National Academy Press, 1986. Page 21 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix II Previous Reviews of the Nutrition Monitoring System concluded that the use of fixed standards alone, such as the Recommended Dietary Allowances, did not take into account normal variability in nutrient requirements among individuals and could lead to imprecise estimates of nutrient adequacy. An alternative, probability-based approach for interpreting nutrient intake proposed by the subcommittee would incorporate information on the distribution of usual dietary intake among individuals. To facilitate the further development and use of this approach, the subcommittee recommended a number of design changes to NFCS, as well as improvements in other nutrition monitoring research activities. The subcommittee recommended that consideration be given to (1) having a sufficient number of days of intake data, (2) using a single method for obtaining dietary intake data, (3) collecting intake data on independent rather than on consecutive days, and (4) collecting information on dietary supplements as well as on food intake. The subcommittee also recommended continued research on dietary intake methods and the design of sampling strategies, as well as the development of methods to improve the reference tables on nutrient composition of foods. Update Report on Nutrition Monitoring (1989) Like the report by the Joint Nutrition Monitoring Evaluation Committee, this report was focused on an update of the dietary and nutritional status, as well as nutrition-related health conditions, of the U.S. population.® This report placed special emphasis on using nutrition monitoring data to examine two topics: the relationship between diet and cardiovascular disease and the assessment of iron nutrition deficiency in the population. It also addressed the strengths and weaknesses of existing data and the methodological issues associated with combining data from different components of the nutrition monitoring system. The committee identified several problems and limitations in the existing nutrition monitoring system and made the following recommendations for improvement: improve comparability of nutrient composition data and coding used in different dietary surveys; test the impact of methodological differences on survey results; ®This report built on the framework of the report from the Joint Nutrition Monitoring Evaluation Committee. Prepared by the Life Sciences Research Office of the Federation of American Societies for Experimental Biology and sponsored jointly by USDA and HHS, the report is entitled Nutrition Monitoring in the United States: An Update Report on Nutrition Monitoring (Hyattsville, Md.: September 1989). Page 22 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix II Previous Reviews of the Nutrition Monitoring System use a common core of sociodemographic descriptors (for example, age, race, income, and education) in all NNMS surveys; increase similarities in NNMS data reporting; investigate methods for assessing population groups currently excluded from NNMS, such as the homeless, migrant workers, military personnel, Native Americans, and those residing in institutions, improve coverage of some groups at nutritional risk, such as infants, pregnant women, lactating women, preschool children, adolescents, and the elderly; improve measures of usual dietary intake in NHANES; collect information for assessing the impact of knowledge and attitudes on patterns of food consumption and nutrient intake; ¢ obtain quantitative information on vitamin and mineral supplement use to better estimate total nutrient intake; ¢ improve estimates of alcohol consumption; ¢ improve survey response rates and analyze nonresponse; * educate data users on the proper use of data from complex surveys; and lastly, be responsive to the needs of state and local data users. We have issued several reports on nutrition monitoring, surveillance, and GAO Reports research, dating back more than 20 years. An early report was our 1971 review recommending that the Health and Nutrition Examination Survey (now called NHANES) and Household Food Consumption Survey (now called NFcs) be merged.” In 1977, we revisited NFcs and found need for improvement and expansion. Specifically, our report found the sample size to be too small to provide useful information in evaluation of food assistance programs and in identifying nutritional problems of low-income families. We therefore recommended that the Congress approve the requests for funds for an additional survey of low-income families and that the survey methodology be validated. In 1978, we reviewed a joint proposal developed and submitted to the Congress by the Departments of Health, Education, and Welfare (now HHS) and uspa for a comprehensive Nutritional Status Monitoring System. This proposal recognized that there was no adequate surveillance system and proposed to institute one. In our report, we established a set of criteria for 7Letter report to the Department of Health, Education, and Welfare and the Department of Agriculture, July 30, 1971. 8Nationwide Food Consumption Survey: Need for Improvement and Expansion, GAO/CED-77-56 (Washington, D.C.: March 25, 1977). Page 23 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix II Previous Reviews of the Nutrition Monitoring System an effective nutrition surveillance system, which were designed to (1) promptly identify nutritional needs; (2) pinpoint, within rather narrow geographic boundaries, the nutritional needs of specific target groups; (3) predict future areas of nutritional concern; and (4) provide data that federal agencies could use to monitor the effectiveness of programs to improve the nutrition, health, and food consumption of various population groups.? We found that a considerable amount of data was being collected that, to varying degrees, satisfied the above criteria, but that a number of weaknesses existed that prevented the programs from functioning as an effective nutrition surveillance system. Specifically, the system (1) was not always sufficiently specific to identify problems by narrow geographic area or did not always include important population groups; (2) did not produce information in a timely manner; and (3) did not provide information adequate for evaluating the effectiveness of programs designed to improve nutritional health. Many of the components included in the proposal by the Department of Health, Education, and Welfare and USDA were never adopted. We criticized the proposal for its lack of specificity and agreement between the Departments, the lack of consensus on collaboration for a decennial survey, the insufficient consideration of program evaluation, and inadequacies in coordination mechanisms. Later in the same year, we issued a report on the future of the National Nutrition Intelligence System.!° The report reiterated problems identified in earlier reviews, including untimely data reporting, insufficient geographic specificity, omission of important population groups, fragmentation and lack of integration for a coordinated system between Departments, and a lack of evaluation capability. However, the report pointed out the positive action taken by both uspDA and the Department of Health, Education, and Welfare in their jointly developed proposal for a comprehensive system of nutrition intelligence in response to the Food and Agriculture Act of 1977. We again reviewed the NFCS in 1991 and found that the most serious data quality problem in the 1987-88 NFCcs resulted from the low response rate of ®Joint Proposal for a Nutrition Surveillance System, GAO/CED-78-145 (Washington, D.C.: June 29, 1978). 10Future of the National Nutrition Intelligence System, GAO/CED-79-5 (Washington, D.C.: November 7, 1978). Page 24 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix II Previous Reviews of the Nutrition Monitoring System only 34 percent of the households in the basic sample.!! We consequently again made the recommendations concerning data validity, and suggested that a detailed sampling plan be developed and that better internal controls be developed to avoid problems with future surveys. We also recently reviewed UsDA’s procedures for evaluating the quality of its food composition data and found that specific quality assurance criteria needed to be developed to ensure data reliability.” Summary As is evident from this series of reviews, several problems with federal nutrition monitoring activities have been repeatedly identified over the past two decades. Although these problems have thus been known for some time, many still exist today. The problem area on which there has been greatest consensus is the lack of coordination between, and compatibility of, different data collection activities, particularly the USDA and HHS surveys. This encompasses differences across surveys in methods for assessing dietary intake and nutritional status, sampling designs, population descriptors and other measures, and the timing and reporting of results. In addition, attention has been focused on the need for more valid methods of measurement, better coverage of population subgroups and geographic areas, and improved survey response rates. ‘Nutrition Monitoring: Mismanagement of Nutrition Survey Has Resulted in Questionable Data, GAO/RCED-91-117 (Washington, D.C.: July 1991). Food Nutrition: Better Guidance Needed to Improve Reliability of USDA’s Food Composition Data, GAO/RCED-94-30 (Washington, D.C.: 1993). Page 25 GAO/PEMD-94-23 Nutrition Monitoring ''Appendix III | Recent Legislative Changes The 1990 Legislation Since the late 1970’s, USDA and HHS have jointly issued three separate plans to strengthen the nation’s nutrition monitoring system.! The lack of progress in implementing these planning efforts, along with growing recognition of the need to improve nutrition monitoring, led to enactment of the National Nutrition Monitoring and Related Research Act of 1990 (P.L. 101-445). As noted earlier, the legislation requires the Secretaries of the Departments of Agriculture (USDA) and Health and Human Services (HHS) to prepare and implement a comprehensive 10-year plan to assess the dietary and nutritional status of the U.S. population, support research and development of nutrition monitoring, foster national nutrition education, and establish dietary guidelines. This 10-year coordinated effort is called the National Nutrition Monitoring and Related Research Program (NNMRRP). The legislation delineates several activities, to be conducted periodically and systematically, that are meant to result in timely information about the role and status of factors bearing on the eating habits and health of the people of the United States. While the law places responsibility for coordinating NNMRRP with USDA and HHS, these activities cut across the jurisdictions of other federal departments and agencies, including Labor, Commerce, Defense, Veterans Affairs, and the Environmental Protection Agency (EPA). To facilitate coordination of nutrition monitoring activities and assist in implementing the program, the law requires the establishment of an interagency board. In addition, it establishes a National Nutrition Monitoring Advisory Council, consisting of independent experts (drawn from outside the federal government), to provide technical advice on the development and implementation of the program. The 10-Year Comprehensive Plan USDA and HHS published a draft of the Ten-Year Comprehensive Plan for the National Nutrition Monitoring and Related Research Program for public comments in the Federal Register in October 1991. The plan was subsequently revised, based on 53 sets of written comments received, and issued in final form in June of 1993. The plan provides an overview of the history of nutrition monitoring and extensive listings of past and current monitoring activities. It lays out a set of planning activities that are organized around three overall national objectives (achieve continuous and coordinated data collection, improve the comparability and quality of data across NNMRRP, and improve the “Proposal: A Comprehensive Nutritional Status Monitoring System,” 1978; “Joint Implementation Plan for a Comprehensive National Nutrition Monitoring System,” 1981; and “Operational Plan for the National Nutrition Monitoring System,” 1987. Page 26 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix III Recent Legislative Changes research base for nutrition monitoring) and three objectives addressing state and local nutrition monitoring efforts (develop and strengthen state and local capacities for continuous and coordinated data collection, improve methodologies to enhance comparability of NNMRRP data across federal, state, and local levels, and improve the quality of state and local nutrition monitoring data). In addition, the plan also emphasizes the need for better nutrition monitoring information about selected population subgroups and for more efficient and effective dissemination to, and exchange of information with, data users. Since both of these areas cut across other nutrition monitoring components, they are not discussed separately in the plan but rather are addressed in the sections devoted to the national and state objectives. Within each section, planned activities and agency responsibilities are described according to five component areas—nutrition and related health measurement; food and nutrient consumption; knowledge, attitudes, and behavior assessments; food composition and nutrient data bases; and food supply determinations. The plan recognizes the problems identified within the nutrition monitoring system over the past two decades and provides a broad, comprehensive set of activities to address them as well as other requirements of the act. Many of the activities are important and necessary for strengthening the nutrition monitoring system. A general time frame (specified by year) for when planned activities will be initiated and completed, as well as when expected products will be issued, is included in the plan. In addition, the plan identifies agency responsibility in terms of lead, collaborating, or contributing agency, for each planned activity. Taken together, these planned activities, when implemented, should go a long way towards achieving the goals and objectives for improving the nutrition monitoring system. One weakness of the plan, however, is that while existing problems are identified as issues, they are often responded to with plans for more planning under NNMRRP. That is, instead of the provision of specific solutions, there are calls to develop further plans, review existing programs and procedures, assess needs, and evaluate alternative methods. The plan also provides little evidence of the extent to which listed activities can or will be implemented in the future. Given the broad scope of activities included in the plan, several of which involve large research and data collection efforts, it is not clear that what is planned for can be accomplished. Further, no attention is given in the plan to prioritizing the myriad activities or assessing the likely costs and feasibility of implementing them. Instead, USDA and HHs acknowledge, in their foreword Page 27 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix III Recent Legislative Changes to the plan, that completion of planned activities will be heavily influenced by the availability of financial resources. They estimate that 20 to 40 percent additional funding will be needed to carry out the plan, amounting to over $200 million in additional appropriations through fiscal year 1998. Page 28 GAO/PEMD-94-23 Nutrition Monitoring if ''Appendix IV The 1987-88 Nationwide Food Consumption Survey: Quality and Uses of the Data Background Impact of Nonresponse on Dietary Data From the 1987-88 NFCS In this appendix, we review statistical weighting issues in the 1987-88 NFcs data set, as identified by the Joint Nutrition Monitoring Evaluation Committee. As we reported earlier, the most serious data quality problem in the 1987-88 Nationwide Food Consumption Survey resulted from the low response rate for the basic sample.! Only 34 percent of the households in the basic sample provided individual data. This raised the question of whether the data were representative of the U.S. population. USDA contracted with a group of statisticians to develop statistical weights to adjust for the high nonresponse rate. These weights are now included with the data tapes provided to users. In 1991, an expert panel was convened by HNIS to assess the integrity of the 1987-88 data. The panel’s report focused solely on the 1987-88 NFcs and examined the statistical design and survey execution.” The report placed particular emphasis on issues related to nonresponse, reviewed analyses of nonresponse conducted by HNIS, identified additional analyses needed to evaluate further the potential for nonresponse bias in NFCS, and reviewed and identified critical issues relating to the implications of potential nonresponse bias for possible inclusion by HNIS in formal publications of survey results and research analyses. The report concluded that it is not possible to definitively establish the presence or absence of nonresponse bias in the 1987-88 NFcs data. The possibility of nonresponse bias is suggested, however, by the analyses reviewed in the report. According to the panel, it is impossible to determine the extent to which nonresponse bias might influence the interpretation of analyses using these data. The report also concluded that it is questionable whether any weighting system could rectify the nonresponse and possible noncoverage (day of the week and month of the year) of the survey. Because of the high level of nonresponse, the report includes an opinion that no weighting procedure could give users the confidence that the low response rate had been successfully dealt with. The report to HNIS warned that HNIS should include strongly worded cautionary statements concerning the potential for nonresponse bias in all publications of the NFcs 1987-88 data, as well as with all public releases of information and data. We note that HNIS ‘Nutrition Monitoring: Mismanagement of Nutrition Survey Has Resulted in Questionable Data, GAO/RCED-91-117 (Washington, D.C.: July 26, 1991). *Life Sciences Research Office, Federation of American Societies, Impact of Nonresponse on Dietary Data From the 1987-88 Nationwide Food Consumption Survey (Bethesda, Md.: April 1991). Page 29 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix IV The 1987-88 Nationwide Food Consumption Survey: Quality and Uses of the Data Uses of the 1987-88 NFCS Data continues to make the 1987-88 data available to the public and does provide a notice to those users receiving the data set emphasizing its nonrepresentativeness. In addition, a nonresponse evaluation report is distributed by HNIS to data users who request it. In ongoing work on nutrition monitoring data uses and users, we are making a thorough study of NFcs data utilization. Here we want merely to mention two critical uses that underscore the importance of high quality data from NFcs. First, the Food and Nutrition Service, UspDA, makes benefit determinations for the Food Stamp Program based, in large part, on NFCS data. Second, the Environmental Protection Agency (EPA) establishes safe levels of pesticide residues for food based on data from NFcs. These uses make clear the importance of nutrition monitoring data not only to the understanding of health and nutrition, but also to food assistance and safety programs. Determining Food Stamp Benefit Levels While eligibility for food stamps is based primarily on income and assets, benefits are based on the cost of the uspa Thrifty Food Plan—a nutritionally adequate diet required to feed a family of four consisting of two adults and two children—adjusted for household size. The food stamp benefit, or allotment, is the difference between the cost of the Thrifty Food Plan for an eligible household and one third of its countable income. About 26 million people are recipients of food stamps. The cost of the Thrifty Food Plan originates from its predecessor, the Economy Food Plan. The Economy Food Plan was based on the 1955 Household Food Consumption Survey data and was used to develop official poverty thresholds. The Thrifty Food Plan replaced the Economy Food Plan in 1975. In the development of the Thrifty Food Plan in 1975 and its subsequent revision in 1983, the target cost for each was linked to the updated cost of the plan that was being replaced. Thus, the cost of the current Thrifty Food Plan is based on the inflation-adjusted cost of the original plan. An inflation adjustment is made monthly by HNIS using Bureau of Labor Statistics food inflation data. This adjustment has been applied to a food basket of 31 food groups based on consumption patterns by low-income households in the 1977-78 NFcs adjusted to meet cost and nutritional standards. The Thrifty Food Plan uses the consumption patterns of low-income households provided by the NFcs low-income sample. This survey collected data on the food consumption behavior of Page 30 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix IV The 1987-88 Nationwide Food Consumption Survey: Quality and Uses of the Data low-income households. The plan uses the NFcs household and individual intake data components. According to a usDaA official, questions about the reliability and credibility of the 1987-88 NFcs data set led to the internal decision not to revise the most recent food plan based on 1977-78 NFcs data. Given that these are 15 years old, they may not be representative of the consumption behaviors of today’s low-income households. According to this same official, there is no other source of household food consumption data available for revising the food plan. Therefore, if current methods are used, the plan cannot be revised until the data from the planned 1996 survey are available. This suggests that USDA will continue to determine food stamp benefits on data collected in 1977-78 through the year 2000. Pesticide Residue Levels for Food To establish safe levels of pesticide residues for food, EPA estimates dietary exposure to pesticide residues using data from NFCS. EPA registers pesticides and establishes maximum allowable pesticide residues (called tolerances) for food in accordance with the Federal Insecticide, Fungicide, and Rodenticide Act and the Federal Food, Drug, and Cosmetic Act. EPA registers pesticides for specific uses and takes regulatory action—such as denying, canceling, or restricting a pesticide’s use—if a pesticide presents a significant health or environmental risk. To determine potential health effects, EPA conducts risk assessments of pesticide products, based largely on laboratory tests and field trials of pesticides. In 1986, EPA began estimating dietary exposure to pesticide residues for tolerance assessments using a computerized system and data collected from the 1977-78 NnFcs. In addition to individual food intake information, EPA uses data such as the individual’s age, gender, weight, race, and place of residence. These data, combined with residue data (usually tolerance levels), allow EPA to estimate exposure for 22 distinct subpopulations who, because of their diets, may be exposed to unsafe pesticide levels. Because they are based on a sample of the population, EPA’s pesticide exposure estimates are subject to sampling error. Our analysis indicated that EpA’s ability to adequately base tolerance assessments on exposure estimates for the five smallest subpopulations—namely, nursing and non-nursing infants, nursing females, pregnant females, and non-Hispanic others (such as Asians and Native Americans)—could be compromised Page 31 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix IV The 1987-88 Nationwide Food Consumption Survey: Quality and Uses of the Data because the sampling error for these subpopulations, based on the 1987-88 NFCS data, can range from nearly 70 to up to 175 percent of the estimate.’ EPA was unable to update food consumption data with the 1987-88 survey results to reflect the current eating habits of Americans. The next major update is scheduled to occur after the 1996 survey has been conducted. This represents another example of an important government program affected by data from NFcs. In this case, a major government program is using noncurrent data and may continue to do so through the year 2000. Users’ Awareness of 1987-88 NFCS Data Limitations The weaknesses of the 1987-88 nFcs data were well-documented before we prepared this report. However, as already noted, use of the data continues, and the Committee asked us to consider whether users were aware of the data’s limitations. We contacted users, identified for us by HNIS, of both the Household Data and the Individual Intake Data from the 1987-88 NFCS. HNIS identified 61 users of the data tapes as of December 1992. We were given a list of users who obtained data tapes directly from HNIS or who purchased the data through the National Technical Information Service. We contacted 31 of those listed and conducted an informal telephone survey using a standardized interview schedule from April through May of 1993. We asked data users in government, academia, and industry the following questions: 1. Can you please describe your use of the 1987-88 NFcs data set? 2. What do you consider to be the strengths and weaknesses of the data? 3. Are you aware that, of the households selected in the basic sample, 34 percent responded to the survey? 4. In your analysis of the data, do you break the data out for subgroups? If so, what groups? Do you utilize the weighting formulae provided with the data by HNIS? 5. Are you comfortable with the results of your analyses using this data set? Why or why not? Results of Telephone Survey Government uses, according to our respondents, included determining whether the nutrient needs of children are being met and identifying the 3For the complete report on this issue see Pesticides: Food Consumption Data of Little Value to Estimate Some Exposures, GAO/RCED-91-125 (Washington, D.C.: May 22, 1991). Page 32 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix IV The 1987-88 Nationwide Food Consumption Survey: Quality and Uses of the Data barriers to a nutritious diet as well as the shopping habits of households. The Food and Drug Administration (FDA) uses the amount of food commonly consumed as a standard to determine serving sizes for labeling. FDA also uses NFCS to select core food samples for its Total Diet Study, a yearly monitoring program, and for assessment of the adequacy and safety of the U.S. food supply. Another agency conducted a study to identify the shopping habits, preparation skills, and dietary guidance of low-income persons in order to determine how many achieved the Recommended Daily Allowance of nutrients for the nation’s population. The Federal Trade Commission, which regulates food advertising, looks at how new types of advertising may change behavior. For example, the Commission looks at how consumption of cereal items and fat, as well as cholesterol claims, change after advertisers air messages linking fat consumption and heart disease. Any change in consumption, such as a reduction in fat, should be detectable in NFCs. Nonfederal government, academic, and food industry uses of the NFcs data identified by our respondents included the study of low-income households; food consumption patterns of various U.S. populations; exposure to pesticides; the prevalence of high fat diets in low-income populations; the impact of employment, age, and gender on eating patterns and nutrient profiles; nutritional adequacy profiles for food consumed away from home and money spent on food away from home; and finally, demand functions for dairy products, product development, serving sizes, labeling, nutrient composition data for product menu/recipe development and planning, product promotions, dietary guidelines, and consumption of meat products. Our respondents identified as the main strength of the NFcs data the fact that it is the only large household food use data set available. In addition, the broad variety of foods, with commodities delineated and the dollar value of the food items listed, is a unique data source. Information is also available in NFcs on household income, food expenditures, and demographics. Time and again, users called the survey results the “only data around” to approximate a national probability sample with household level information on food use. Overwhelmingly, respondents indicated that the chief weakness of the data was the low response rate. Only one user was unaware of this issue. Most of our respondents utilized the weights provided with the data by HNIS, and seemed to interpret their results with caution. When large aggregated data were analyzed, it appears our respondents were less Page 33 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix IV The 1987-88 Nationwide Food Consumption Survey: Quality and Uses of the Data Using the Weighted Data From the 1987-88 NFCS concerned that the results might be skewed. However, when breaking the data out into smaller cells (such as Hispanics, black males, young persons, or alternatively, infrequently consumed foods), the level of comfort with any result declined for our users. Several of them tried to match samples with data in csFu or other national data sets, to compare for similar trends. The original 1987-88 basic survey sample was designed as a self-weighting, stratified, multistage, area probability sample, representative of the households in the 48 coterminous states. There were 60 strata. Two primary sampling units were selected from each stratum, with replacement. Area segments were selected from each selected sampling unit; then housing units were selected from each selected area segment. Every household member of each selected housing unit was to be interviewed. The survey was planned to start in April 1987 and continue to March of 1988. The total number of area segments was 1,000. The total number of estimated sample housing units was 8,800, with each quarter including 2,200. Four interpenetrating samples were to be drawn—one for each quarterly survey. However, due to the high nonresponse rate of the first quarter, the sample sizes of successive quarters were increased, and the survey was extended to August of 1988. After March of 1988, the households interviewed were housing units from the four prior quarterly sample housing units that had not yet been interviewed. After sample housing units were selected, the interviewers contacted and screened them, and then made appointments with eligible households to conduct interviews. The interviews could not be conducted until at least 7 days after screening. During the interview, the interviewer obtained the household food use over the last 7 days and 1-day dietary recall from each member of the household, and left a 2-day dietary record for each member of the household to complete. The final sample for the 1987-88 NFCs basic survey deviated from the original design and was not a self-weighting sample. As has been stated earlier, in spite of efforts to raise them, the NFCS response rates were very low. A standard practice in survey research is to develop weights to be applied to the data in analyses which attempt to compensate for the nonresponse. Weights were developed to compensate for the NFCS nonresponse. However, given the high level of nonresponse in Page 34, GAO/PEMD-94-23 Nutrition Monitoring '' Appendix IV The 1987-88 Nationwide Food Consumption Survey: Quality and Uses of the Data the survey, we concur with the report that suggests there may be nonresponse bias in the data despite the weighting adjustments.’ We reviewed general weighting adjustment methods for NFcs, including sample-based adjustments, population-based adjustments, a regression adjustment procedure, combinations of sample- and population-based adjustments, and the combination of sample-based and regression adjustments. The weighting adjustment methods chosen depend on the survey design and what kinds of information are available. Generally, combination methods are better. The regression adjustment method was used to develop weights released with the NFcs data. In the combined method, subgroups called weighting classes are used for the weighting adjustments and are formed using a combination of control variables. The most important issue is to find weighting classes in which a missing-at-random assumption or a “similarity assumption” holds as accurately as possible. The missing-at-random assumption states that nonresponding units within weighting classes are a random sample of all units within the class. Similarity is a weaker assumption requiring only that responding and nonresponding units within a class be, on average, alike. Usually, the choice of classes is decided by experience and best judgment. The two major categories of missing units are “not-at-home” and “refusal.” The probability of not-at-home depends on a wide variety of situations. For example, a family with small children may be easy to find because they are at home more often than families without small children, whereas single people may not be easy to find. Alternatively, older or widowed “shut-ins” may be easy to find, while young married, childless couples may not. Other significant factors are housing unit location, household head’s age, and employment status. The probability of refusal depends on the patience of respondents, the number of surveys they have been exposed to, the skill of the survey staff, and other factors. Different survey subjects have different degrees of patience. For example, some studies suggest that higher income and busy people usually have a lower level of patience with surveys. The poor design of a questionnaire, which can cause heavy respondent burden, as well as poor interviewing skills, may increase the probability of refusal. ‘Life Sciences Research Office, Federation of American Societies, Impact of Nonresponse on Dietary Data From the 1987-88 Nationwide Food Consumption Survey (Bethesda, Md.: April 1991). Page 35 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix IV The 1987-88 Nationwide Food Consumption Survey: Quality and Uses of the Data Since unit nonresponse is affected by many different circumstances, it is hard to find weighting classes in which data are likely to be missing at random. However, it is easier to find the weighting classes in which data for responding and nonresponding units are similar. Usually, units with the same characteristics—such as age, sex, and level of urbanization—yield similar data. We conclude that the weighting techniques used for the 1987-88 NFcs data are technically correct within the constraints of standard survey analysis, assuming the data are missing at random. However, it is virtually impossible to determine whether the missing-at-random assumption is valid for these data, especially in view of the high level of nonresponse. Thus, results of analyses that use these data are questionable even when the regression adjustment weights are employed. There are other adjustments that could have been made that require weaker assumptions. Even these adjustment procedures are suspect since there is no way of knowing whether the findings are substantially biased due to the high level of nonresponse. Thus, these data should be used only as a last resort, when no other data are available. Even when used as a last resort, they should be used with caution and an awareness of the potential problems inherent in the data. Further, any findings from these data must contain a caution to the readers of the results concerning the potential for bias due to nonresponse. Page 36 GAO/PEMD-94-23 Nutrition Monitoring ''Appendix V Comments From the Department of Agriculture United States Agricultural Office of the Washington, D.C. Department of Research Administrator 20250 Agriculture Service April 29, 1994 Mr. Kwai-Cheung Chan, Director Program Evaluation in Physical Systems Areas General Accounting Office Washington, D.C. 20548 Dear Mr. Chan: Thank you for the opportunity to comment on the draft interim report, entitled "NUTRITION MONITORING: Progress in Developing a Coordinated Program." The Department of Agriculture is committed to a strong and well-coordinated National Nutrition Monitoring and Related Research Program, and we believe progress has been made toward that goal within current fiscal constraints. Our specific comments which are attached reflect that progress. Sincerely, Ws iy E. E. FINNEY, Acting Administrator Enclosures . an Tie, 2, 3 / CONCURRENCE : - aS DATE: wily R. D. PLOWMAN Acting Assistant Secretary Science and Education Agriculturat Research Service Page 37 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix V Comments From the Department of Agriculture HNIS, ARS, USDA COMMENTS ON GAO DRAFT REPORT "NUTRITION MONITORING: PROGRESS IN DEVELOPING A COORDINATED PROGRAM" rrec ions r 1987-88 NF Difficult The Human Nutrition Information Service (HNIS) has moved aggressively to resolve the difficulties with the execution of the 1987-88 Nationwide Food Consumption Survey (NFCS). Major areas of concentration have included management and quality control procedures, survey design and data collection methods, and technical support systems. The following provides an overview of the actions taken in these areas. ° Separation of NFCS into Individual and Household Surveys - HNIS separated the household and individual components in response to concerns GAO first raised in 1977 about the burden placed on the respondents in the NFCS and to facilitate data processing. The burden on respondents also was the chief GAO criticism of the 1987-88 NFCS. HNIS’ decision to conduct two separate surveys, the Continuing Survey of Food Intakes by Individuals (CSFII), and the Household Food Consumption Survey (HFCS) was documented in the June 11, 1993, Federal Register notice on the Ten-Year Comprehensive Plan for the National Nutrition Monitoring and Related Research Program, Vol. 58, No. 111, pages 32753 and 32767. ° Interagency Agreements with the Bureau of the Census - After the 1987-88 NFCS, HNIS signed two interagency agreements with the Bureau of the Census to assist in the planning and preparation for the next individual food intake survey as well as for the next household food consumption survey. Focus was placed on reducing respondent burden and collecting complete food intake data. Census Bureau recommendations were incorporated into the CSFII 1994-96, and cooperative efforts between the Census Bureau and HNIS continue for future individual intake surveys. Under the interagency agreements, the Census Bureau also conducted research to improve the list-recall methodology used to collect household food use data and to study alternative methodologies. The list-recall methodology had been criticized by GAO for the burden it placed on respondents. ° Sampling Design and Data Collection - Sampling changes made for the CSFII include (1) representation of all 50 States and Washington, D.C.; (2) oversampling of the low-income population rather than a separate low-income survey; (3) larger numbers of individuals in some age categories, Page 38 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix V Comments From the Department of Agriculture especially young children, teens, and elderly; and (4) subsampling within households rather than collecting dietary intakes from all household members to reduce burden. To reduce respondent burden and increase response rates. We are collecting 2 independent days of intake data through in- person interviews rather than 3 consecutive days using a 1-day recall and a 2-day record. OMB did not approve a request for collection of 3 nonconsecutive days of intake data citing the increase in respondent burden and decrease in response rates. Each question on every questionnaire has been thoroughly reviewed for its value and efficiency in obtaining the desired information. The Continuing Survey Users Group, comprised of representatives from Federal food consumption data users and the Census Bureau, provided input into revision of the questionnaires. To help increase response rates, survey publicity has been greatly expanded to include brochures and flyers in both English and Spanish, and press releases sent to over 500 newspapers in survey locations. Management and Quality Control Procedures - To further address GAO concerns after the 1987-88 NFCS, every facet of the CSFII 1994-96 was thoroughly examined and redesigned if necessary. The contract for the CSFII 1994-96 was awarded September 1992 to Westat, Inc. HNIS has aggressively implemented strong management and quality control procedures, both as part of the contract and in its in-house operations for the CSFII 1994-96. Staff have been assigned specific responsibilities for monitoring tasks in their areas of expertise. The survey contract contains specified response rates and penalties the contractor must pay if the response rates are not met. Pilot Study - A pilot study of all survey operations for CSFII 1994-96 was conducted in spring 1993. This study was conducted in 10 locations nationwide and included both rural and urban areas, as well as areas designated by the Census Bureau as low income. The results of the pilot study showed that the response rates specified in the contract were met or exceeded and that new operations implemented for the survey were successful. We are pleased to report that we are continuing to experience comparable results since the survey was fielded in January of this year. Technical Support Systems - From the contractor's automated field management system, HNIS receives status reports of field work to monitor the data collection efforts and reviews the progress of the survey with the contractor on a weekly basis. An automated forms tracking system provides weekly reports detailing the number of questionnaires received, processed, and transmitted weekly to HNIS. In Page 39 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix V Comments From the Department of Agriculture addition, HNIS receives daily "snapshots" from the contractor on the status of each document. HNIS is receiving survey data electronically on a weekly basis in order to identify problems or errors early and to speed the release of data. Much of HNIS’ in-house data processing has been automated to shorten processing time, to improve the efficiency of review, and to strengthen quality control. We have implemented an in-house tracking system to monitor the status of data from receipt to analysis. HNIS has developed, in collaboration with the University of Texas, an automated food coding system that improves the efficiency of our processing of food data. This software, SURVEY NET, has built-in quality control features and, as an additional measure of quality control, the software undergoes continuous review of recipes, food weights, and the inclusion of brand name and ethnic foods. ° Redesign of Household Food Consumption Survey - The plans and redesign for the HFCS are currently under study. During 1995 budget development, it was determined that this survey may duplicate data available from other sources or provide data that could be collected from other sources for far less than $15 million. Consequently, it was decided that the "Household" survey should be delayed until a compelling case can be made that it is worth the money. ti ing I iat - HNIS, as recommended by GAO, has taken strides to make all data users aware of the nonresponse issues associated with the 1987-88 NFCS. We are pleased to learn that your survey of those data users indicates success in this endeavor. We would like to note that we additionally distribute our nonresponse evaluation report to users who request it. We would like to clarify the GAO analysis of the statistical weighting issues. The NFCS weights were designed with the assumption that given a vector of characteristics, data are missing at random. The level of nonresponse has no bearing on whether this assumption is true. Page 40 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix V Comments From the Department of Agriculture rvey Facts imin £ the Cont ny urvey of Foo t. 8 by Individuals The original design of the CSFII was as an annual or periodic survey, but with the capability to provide a continuing source of data for comparability. Since implementation in 1985-86, the CSFII has been conducted regularly in order to provide continuing dietary intake data. Data collection for CSFII 1994-96 began in January 1994, and the start of the next series is planned for 1998. In addition, attached is a revision to the sections in Appendix I which describes the NFCS and the CSFII (Attachment A). We believe this updated version more adequately summarizes the development and current status of these surveys. a_ Ne r Env 1 Prot O} In response to recommendations from the National Academy of Sciences report on "Pesticides in the Diets of Infants and Children," HNIS chairs an interagency Food Safety Working Group that is addressing the specific data needs for estimating pesticide exposure. The group is developing two proposals that would utilize existing data from the Nutrition Monitoring Program. Funds have been included in the President's 1995 budget request for implementation of a dietary survey of infants and children. Assuming Congress will appropriate the increase in funds for this survey, detailed survey design and implementation planning has begun. USDA, EPA, and FDA officials will meet to define data requirements. An internal planning committee of ARS and HNIS personnel will meet to participate in survey design and planning. inin t efit Levels-- Thrifty Food Also attached is a revision to the section in Appendix IV "Determining Food Stamp Benefit Levels" (Attachment B). As described in this revision, the Thrifty Food Plan uses the consumption patterns of low-income households provided by the NFCS low-income sample, utilizing both household and individual intake data components. Food stamp benefits are based on the cost of the Thrifty Food Plan. The cost of this plan is based on the inflation adjusted cost of its predecessor, the Economy Food Plan, and demonstrates that a nutritionally adequate diet can be purchased given the inflation adjusted costs of the Thrifty Food Plan. The research conducted to determine the Thrifty Food Plan is important in defining nutritionally adequate diets for Americans based on current consumption patterns. Page 41 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix V Comments From the Department of Agriculture r id, rdization HNIS and the National Center for Health Statistics (NCHS), Department of Health and Human Services, have jointly funded research to explore the linkage of the National Health and Nutrition Examination Survey (NHANES) and CSFII sample designs as part of the NHANES 1997+ sample design research contract awarded on September 22, 1993. HNIS and NCHS are jointly developing a proposal for an automated dietary interview for use in both surveys. Work on comparability in key survey questions will continue to be addressed through the Survey Comparability Working Group. Progress has been made on developing core standardized measures for the major surveys. The Survey Comparability Working Group issued a report in July 1992, Improving Comparability in the National Nutrition Monitoring and Related Research Program {NNMRRP), that recommends specifically defined and measured sociodemographic descriptors for use in nutrition monitoring surveys to improve linkages among the surveys and achieve a more comprehensive and coordinated Nutrition Monitoring Program. These descriptors were incorporated into the CSFII 1994-96 questionnaires where appropriate. The group is currently working on a common core of nutrition and health-related variables to be used in NNMRRP surveys. An analytic working group comprised of staff from HNIS and NCHS was established in 1992 to coordinate the analyses and reporting of data from dietary intake surveys. Progress to date includes the use of the same basic sex-age categories and population descriptors for reporting nutrient intakes in publications issued by both agencies; greater similarity in calculating income for reporting and on the use of imputation procedures; development of a set of statistical guidelines for determining when a survey estimate is stable enough to be published; and a standardized methodology for calculating response rates. These guidelines were made available to Federal agencies for use in preparing data for the Third Scientific Report on Nutrition Monitoring. The group is now addressing research underway in both agencies for estimating usual nutrient intakes. Cc ehensiv -Year Pl tional ion Monito: d Re Res h We agree with GAO that implementation of the Ten-Year Plan activities will strengthen and advance nutrition monitoring. This was the underlying philosophy when the Plan was being developed. Technical staff working in nutrition monitoring across the Federal Government were asked to identify activities that should be accomplished to provide a stronger nutrition ‘ monitoring program. Budget and cost constraints were not to be a barrier in defining what should be done. Of course, in the fiscal environment of today, budget limitations are challenging Page 42 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix V Comments From the Department of Agriculture nutrition monitoring priorities. Options for prioritizing the Ten-Year Plan activities have been discussed widely, both at meetings of the Interagency Board for Nutrition Monitoring and Related Research and the National Nutrition Monitoring Advisory Council. The Ten-Year Plan has provided an excellent forum to enhance and expand coordination and communication across all agencies involved in nutrition monitoring. Prioritizing Ten-Year Plan activities must be accomplished without diminishing the progress that has resulted from the Plan's first 2 years of implementation. The Plan does provide a framework for evaluating and updating activities. Annual progress in implementing and accomplishing activities is summarized in a progress report yearly. The Plan is to have a thorough evaluation in 1997 and undergo revision, as appropriate. These requirements are discussed in section IV of the Plan, "Activities of the Interagency Board for Nutrition Monitoring and Related Research." Technical Correction - Page II-9, line 12: An additional survey of low-income families occurred with the 1977-78 NFCS, not the CSFII. Page 43 GAO/PEMD-94-23 Nutrition Monitoring ''Appendix VI Comments From the Department of Health and Human Services paericns, DEPARTMENT OF HEALTH & HUMAN SERVICES Public Health Service A MEAT, “ene Rockville MD 20857 APR 15 1904 Mx. Kwai-Cheung Chan Director of Program Evaluation in Physical Systems Areas Assistant Comptroller General General Accounting Office Washington, D.C. 20548 Dear Mr. Chan: Enclosed are the Public Health Service’s comments on your draft report, “Nutrition Monitoring: Progress in Developing a Coordinated Program." The comments represent the tentative position of the Public Health Service and are subject to reevaluation when the final version of this report is received. The Public Health Service appreciates the opportunity to comment on this draft report before its publication. Singerely co ’ 4 Anthony’L. Itte Deputy Assistant Secretary for Health Management Operations Enclosure Page 44 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix VI Comments From the Department of Health and Human Services We appreciate the opportunity to review this draft report because of the PHS’ commitment to improving nutrition monitoring in the United States. We offer the following comments for your consideration. As you are aware, several PHS agencies work in the area of nutrition improvement and monitoring, among them are the Centers for Disease Control and Prevention (CDC), the National Institutes of Health (NIH) and the Food and Drug Administration (FDA). Within CDC, there are three centers involved in this area: the National Center for Health Statistics (NCHS), the National Center for Chronic Disease Prevention and Health Promotion, and the National Center for Environmental Health. PRINCIPAL FINDINGS t n v. Collaborative working arrangements with federal agencies are established early in NCHS’ National Health and Nutrition Examination Survey (NHANES) planning process and continue through data collection, monitoring, evaluation, and data reporting. The survey is designed to meet as many public health and nutrition policy data needs as feasible. The NCHS currently collaborates with 27 other federal organizations on NHANES III. We agree that it is essential for joint planning in the National Nutrition Monitoring and Related Research Program (NNMRRP) so that comparable data are collected across surveys. The Interagency Board for Nutrition Monitoring and Related Research (IBNMRR) provides the appropriate forum for soliciting federal nutrition monitoring data needs for joint survey planning. ~Ye lan We consider that the 10-Year Plan for the National Nutrition Monitoring and Related Research Program (NNMRRP) sets the necessary framework for implementing specific actions to improve nutrition monitoring for the next decade. The annual reviews at the IBNMRR meeting provide the mechanism for maintaining, deleting and modifying monitoring activities. The IBNMRR has met quarterly over the past two years and holds annual progress reviews on the 10-Year Plan. In addition, these activities are planned as part of the formal mid-course review and revisions scheduled for 1997. As noted in Appendix III of the GAO report, Page 45 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix VI Comments From the Department of Health and Human Services the Plan includes accountability through delineation of agency responsibility, time schedules for each activity, and periodic progress reviews. The Plan was also designed to allow flexibility to the responsible agencies in implementing specific activities through the development of detailed work plans with contributing and collaborating agencies. This flexibility also allows for modifications to the Plan as priorities and resources shift over time. The interagency Implementation Working Group for the 10-Year Plan has established priorities for the Plan which have been discussed with the National Nutrition Monitoring Advisory Council. These top priority areas were also the subject of the 1993 progress review held at the January 1994 IBNMRR meeting. As part of the Plan, essential actions are required in the key areas of joint planning and coordination of the major national surveys that collect dietary intake information, i.e., NHANES and the U.S Department of Agriculture’s (USDA) Continuing Survey of Food Intakes by Individuals (CSFII). These actions include use of a common dietary methodology, use of common population descriptors, and exploration of a linked sample design to improve population subgroup estimates. We would like to provide additional details on HHS actions relevant to the past criticisms of nutrition monitoring noted in Table 1, page L-8. Criticism Additional Details Specific population In response to the need for improving groups not covered by coverage of specific population groups major surveys in surveys, NHANES III (1988-94) includes oversampling of African Americans and Mexican Americans, with each subgroup accounting for 30 percent. of the total sample. The NCHS has awarded a sample design contract in preparation of the next NHANES survey scheduled to begin in 1997. The contract includes conducting research on sample design issues for sampling population subgroups and investigating sample linkage possibilities between NHANES and CSFII. This latter objective is jointly funded by USDA’s Human Nutrition Information Service (HNIS). Page 46 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix VI Comments From the Department of Health and Human Services 3 Core set of In preparation for NHANES '97, CDC has standardized measures sponsored a contract to evaluate core not yet developed for nutrition and health indicators for major surveys inclusion in NHANES and the feasibility of conducting a core nutrition component, including dietary intake, in alternative settings such as households, nursing homes, schools, homeless shelters, and American Indian reservations. Development and dissemination of a core nutrition component for use in national surveys, surveillance systems, and State and local settings is a high priority activity for which NCHS has lead responsibility in the 10-Year Plan. Information needs Information needs of users are of users not systematically determined in planning systematically the NHANES survey. Through a broad determined solicitation process, the NHANES program has traditionally integrated survey planning with data needs for public health and nutrition policy and incorporated policy data needs into study objectives. In addition, NCHS sponsors a biennial data users’ conference to update data users about the availability and use of NCHS survey data and findings. Need for more timely In response to improving the timing of dissemination of data release, NCHS has placed high survey information priority on reporting NHANES III, Phase 1 (1988-91) data on topics of public health importance, such as serum cholesterol levels in the population (JAMA 1993;269:3002-14), dietary fat and energy intakes (MMWR 1994;43(7):116-25), and hypertension (National High Blood Pressure Education Program, The Fifth Report of the Joint National Committee on Detection, Evaluation, and Treatment of High Blood Pressure, 1993). Reports of overweight, prevalence of low bone density, HIV, and blood lead have been submitted for publication while NHANES III, Phase 2 (1991-94), is still in the field. Page 47 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix VI Comments From the Department of Health and Human Services 4 State and local data States do use the national surveys but needs not fully it is financially prohibitive to change addressed the design of any of the national surveys for them to provide detailed information at State and local levels. TECHNICAL COMMENTS a. Page L-3. In the sentence “Nutrition monitoring should provide information on a regular basis...the composition of foods eaten, including their content of essential nutrients and supplements, as well as the presence of any contaminants that may influence nutritional quality...." the phrase that appears in bold print is unclear and perhaps some punctuation is missing. We also suggest that you define the word "supplements". 2. Page L-3, second paragraph. Beginning with the sentence “Por example, data from the program are used in determining benefits in food assistance programs (such as food stamps, and the Special Supplemental Food Program for Women, Infants, and Children)...," we suggest it be revised to read as follows: "For example, data from the program are used in determining benefits in food assistance programs (such as food stamps, and the Special Supplemental Food Program for Women, Infants, and Children). Data are also used to formulate national nutrition and health objectives and policies (such as national initiatives that seek to lower fat intakes or to educate the public about cholesterol), to formulate food labeling regulations (such as determining serving sizes and defining criteria for the qualification of nutrient content and health claims), and evaluate nutritional and health programs (such as studies of the effects of second-hand smoke on the health of the U.S. population). Finally, the data are used to determine the adequacy and safety of the food supply. The FDA uses the data to assess the need for appropriate levels, and safety of food fortification and the levels of dietary exposure to food additives and contaminants. In addition, pesticide residue estimates are calculated...." 3. Page L-4. The following statement appears in the first paragraph: "NHANES collects data over a 3-year period, with a 3-year planning cycle between administrations, and focuses on questions of health and nutrition." This statement does not agree with information about NHANES III appearing in Appendix I, page I-12: "Data collection for NHANES III Page 48 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix VI Comments From the Department of Health and Human Services 10. began in September 1988 and will continue to 1994 ... The survey is being conducted in two 3-year segments, with data collected and analyzed at the end of the segments as well as for the full survey. The National Center for Health Statistics is planning the next NHANES to be fielded in 1997." APPENDIX I, page 1. We assume the appendix uses the term National Nutrition Monitoring System (NNMS) to distinguish between the nutrition monitoring system prior to the 1990 legislation, in which the name was changed to the National Nutrition Monitoring and Related Research Program (NNMRRP). APPENDIX I, page 3. We suggest the last sentence of the first full paragraph beginning with “Furthermore, data on food consumption..." be deleted and replaced with: "Furthermore, food consumption data are used by FDA to estimate dietary exposure to food additives, toxicants, and contaminants; and by EPA to estimate dietary exposure to pesticides. FDA also uses the data to develop food labeling regulations such as establishing the reference standard for determining serving sizes for nutrition labeling purposes and defining the criteria for the qualification of nutrient content and health claims." APPENDIX I, page 4. The Directory of Federal and State Nutrition Monitoring Activities, published in 1992 by the Department of Health and Human Services (HHS), is authored by the IBNMRR. APPENDIX I, page 10. At the end of the third line, add “add both sexes." APPENDIX I, page 11, line 2. CDC should be noted as the Centers for Disease Control and Prevention. APPENDIX I, page 11, first full paragraph. In NHANES III, dietary intake data are collected using a single 24-hour recall for all sample persons. In addition, non-random replicate recalls are collected to adjust population distributions of nutrients, and two additional 24-hour recalls were collected by telephone for all examined persons ages 50 years and older in 1988-91 to estimate usual dietary intake in older persons. APPENDIX I, page 12. NHANES III will have traveled to 89 locations by the close of the survey in 1994. Page 49 GAO/PEMD-94-23 Nutrition Monitoring '' Appendix VI Comments From the Department of Health and Human Services 1l. 12. 13. 14. 15. APPENDIX II, page 1. The update report on nutrition monitoring was published in 1989 (not 1991). APPENDIX II page 2. Recommendations to oversample population subgroups in HANES was partially met with Hispanic HANES (1982-94) as well as oversampling of African Americans (30 percent of total sample), Mexican-Americans (30 percent of total sample) and infants and children under 6 years, and persons ages 60 years and older in NHANES III (1988-94). APPENDIX II, page 3. The 1984 National Research Council’s Committee on Evaluation of Food Consumption Surveys specifically incorporated a review of NHANES food consumption data. This project was also supported by NCHS. APPENDIX IV, page 8. The sentence beginning with "FDA also uses the NFCS to select ..." should be revised to read: "FDA also uses the NFCS to select core food samples for its Total Diet Study, a yearly monitoring program, and for assessment of the adequacy and safety of the U.S. food supply." APPENDIX IV, page 8, Results of Telephone Survey. At the end of the seventh line, please add *, and for the assessments of the adequacy and safety of the U.S. food supply." Page 50 GAO/PEMD-94-23 Nutrition Monitoring ''Appendix VII Major Contributors to This Report : James Joslin Program Evaluation Jalna. Covaubaciin and Methodology Leslie Riggin Division Randall H. Wold : Maricela Camerena Denver Regional Art Gallegos Office (973382) Page 51 GAO/PEMD-94-23 Nutrition Monitoring '' '' C€O4?293%be Ordering Information The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent. Orders by mail: U.S. General onan este P.O. Box 6015 agate MD Te og Room Te 700 4th St. Nid ee a 4th and G Sts. NW) U.S. General or daeauia Office Washington, DC OCB ENA Cm Ee me ee (202) 512-6000 or by using fax number (301) 258-4066. OY. PRINTED ON (3) RECYCLED PAPER '' United States General Accounting Office Washington, D.C. 20548-0001 Official Business Penalty for Private Use $300 Address Correction Requested Bulk Mai! Postage & Fecs Paid GAO Permit No. (:100 ''