LC 90 .C2 C78 2000 [GSL UC B Crucial Issues in California Education 2000: Are the Reform Pieces Fitting Together? Policy Analysis for California Educaflon PACE .32! l‘ tlii Correction for Chapter 4, Figure 1, page 50 $7000 $6000 I 55000 / I“)! \U l \U l CO (J l \D U) \D l \I) t.) l \D \0 Ln I \U 0 1969—70 1 , I United States g: California Figure 1. Current Expenditures (in current dollars) per Pupil in Average Daily Attendance in Public Elementary and Secondary Schools. California and the United States: 1969-70 to 1995-96 Crucial Issues in California Education 2000: Are the Reform Pieces Fitting Together? POIiCy Elizabeth Burr Analysis for UC Berke/9y California Gerald C. Hayward Ed UCatlon Sacramento PACE Bruce Fuller UC Berkeley WNWEWM. SIIIIJIESIJBRMY Michael W. Kirst Stanford University m1 2 M‘ WHOFCMJW Preface and Acknowledgments Crucial Issues in California Education 2000 is a successor to Conditions ofEclucation, a PACE publication since 1984. Conditions combined up-to-date data and ongoing trends in a wide variety of indicators relevant to state education policy. In recent years, education in California has become more complex, undergoing both strident criticism and renewed support. To present a more analyti- cal overview of California education, this year PACE has asked experts around the state to con- tribute chapters based on in-depth research projects. Their contributions allow PACE to offer the latest data analysis around a wider variety of issues, while continuing to provide overall strategic recommendations. This volume provides a unique function in policy analysis because it brings together numerous reports on components of California education in one source. Moreover, the scope of Crucial I.\‘.\‘ll6.\‘ is the largest in the history of the series, spanning child care to universities. We have received generous financial support from several foundations. First, PACE could not survive without core support and advocacy from Ray Bacchetti at the William and Flora Hewlett Foundation. The Stuart Foundation has been helpful since 1998 when Jane Henderson came to an early planning meeting. Since then, Ted Lobman has been a strong supporter. Lisa Carlos and Joan Herman also attended early planning efforts. Robert Shireman at the james Irvine Foundation provided funding for the newly structured volume. The work reported in Chapter 7 was also supported in part under the Educational Research and Development Centers Program, PR/Award Number R305B60002, as administered by the Office of Educational Research and Improvement, US. Department of Education. The study described in Chapter 9 was commis— sioned by Stanford University’s Bridge Project: Strengthening K-16 'li‘ansition Policies, a national study funded by the Pew Charitable Trusts and the US. Department of Education’s Office of Educational Research and Improvement. Warm thanks are extended to Sarah Baughn for early help in research and organization, to Tor Ormseth and Peter Scott for providing research assistance, and to Marsha Ing, who assisted with the analyses. Kay Cooperman provided editorial assistance and Rachel Montgomery super— vised final production and tirelessly checked references. At the project’s end, judith Kafka provided speedy and meticulous copyediting. Many people gave their time for careful review of chapters: Richard Duran, Eugene Garcia, Dave Jolly, Barbara Merino and Allan Odden. Lisa Carlos, Diane Ilirshberg, Vicki Lavatos, Robert Manwaring and Rich Shavelson also offered help and guidance on several chapters. We thankjennifer Garner for her vision and talents in design, and for handling produc— tion so quickly and carefully. David Ruenzel brought patience and humor to the project, offering substantive and careful critiques, as well as original writing. He edited the entire manuscript. Finally, we would like to show appreciation for our PACE team, especially 'l‘erry Alter, Regina Burley, Robert Dillman, and Diana Smith. Elizabeth Burr Gerald C. Hayward Michael W. Kirst Bruce Fuller Berkeley Sacramento Stanford Ber/(clay I PACE Crucial Issues in California Education 2000: Are the Reform Pieces Fitting Together? Table of Contents Chapter 1 Chapter 2 Chapter 3 Chapter 4 Chapter 5 Chapter 6 Chapter 7 Chapter 8 Chapter 9 California’s Ambitious Education Reform Agenda: Will It Energize Schools and Teachers? David Ruenzel Early Education and Family Poverty Elizabeth Burr and Bruce Fuller The Schooling of English Learners Russell Rumberger and Patricia Gandara School Finance Neal Finkelstein, William Furry and Luis Huerta Governance and Accountability Michael W Kirst, Gerald C. Hayward and Bruce Fuller Teacher Quality The Center for the Future of Teaching and Learning Student Assessment and Student Achievement in the California Public School System Joan L. Herman, Richard S. Brown, and Eva L. Baker Connecting California’s K-12 and Higher Education Systems: Challenges and Opportunities Andrea Venezia Alignment Among Secondary and Post— Secondary Assessments in California Vi-Nhuan Le, Laura Hamilton and Abby Robyn 23 45 79 95 113 153 177 PA C E I Chapter 1 California’s Ambitious Education Reform Agenda: Will It Energize Schools and Teachers? David Ruenzel PACE An Unprecedented Surge in Reform Ideas California’s schools may face scarcities of many key ingredients, from qualified teachers to modern classrooms. But there is no shortage of ideas when it comes to how policy— makers are eagerly searching for ways to fix the state‘s troubled public schools. The team that crafted this volume, Crucial loner 1'11 (Kali/bruit] Education 2000: :17‘6 f/Jt’ Ix’ij/iu‘m I’icrer lv'z'm'ng Wigrf/Jerfl faced a massive challenge simply keeping up with the reforms being legislated in Sacramento and the ways in which local districts have struggled to imple— ment them. \\'hile the new mandates forged by policymakers have been well—intentioned, they threaten in number and complexity to over— whelm educators. ( Ialifbrnias educators have gone through several generations of school reform. The 1060s brought early categorical programs aimed at ser\ ing previously undersen‘ed groups, from children with weak reading skills to non—lCnglish speaking youngsters. In 1983, more than 40 separate reforms were approved by the legislature. The 1990s brought new ini— tiatives in the school-choice arena, such as charter schools, a new state testing program, a mandated attempt to end social promotion of children, and radical reductions in class sizes. Bilingual teaching methods were outlawed unless a critical mass of local parents demanded that they continue. But the new Sacramento—led accountability system, successfully pushed through the legisla- ture by Gray Davis during his initial months as governor, is unprecedented in a number of ways. Aiming “to restore the greatness of California education” in the governor’s words, Sacramento for the first time is tracking which schools effectively raise children’s learning curves over time, and which schools fail to do so. Carrots or sticks are allocated by Sacramento. Curriculum guidelines and a new statewide exam, only partially aligned with what teachers are expected to teach, are also crafted in the state capital. The legislature has now told all school districts to implement a peer evaluation process for all teachers. Districts continue to struggle with reducing class sizes, finding enough qualified teachers, as well as ending social promotion and creating new summer school programs for those who flunk a grade level. PACE I [\J Yet only in selected cases do schools and teachers receive additional resources to push hard on all these reform fronts. From a policy perspective, the governance of public education continues to steadily move to Sacramento and away from local school boards. The expecta- tions and mandates placed on these local boards, district staff, principals, and classroom teachers are rising dramatically. But Sacramento’s political will—as well as the vot— ers’—to provide additional resources to get the job done remains constrained. In March 2000 a majority of voters said they did not want to make it easier for local educators to sell bonds to renovate dilapidated school facilities. In this volume, the PACE team offers mixed observations about this flurry of reform activity. On one hand, we feel good about the civic debate that has invigorated California for the past decade and a half. The expressed concerns of parents, civic leaders, employers, and editor— ial boards has moved policymakers—at both state and local levels—to enact a breathtaking array of policy initiatives. On the other hand, the PACE team wonders if these myriad reforms will add up to a coher— ent set of institutional changes. That is, are we weaving together a patchwork quilt that, while colorful, fails to hang together over time? Several of the chapters that follow detail pieces of the reform puzzle, then ask whether the pieces are fitting together. A second set of questions must be put on the table: How do these reforms deliver more high- ly qualified and skillful teachers, and how do they motivate the State’s teachers to innovate and implement more effective teaching prac— tices? In other words, do these puzzle pieces of CRUCIAL ISSUES IN CALIFORNIA EDUCATION reform fit together into a coherent “ theory of action,” whereby policies emanating from Sacramento will energize teachers in the school down the street? From the outset we focus on the question of policy coherence. The chapters that follow push forward on the issues of whether teachers will be moved to improve and whether man- dates without additional resources will really be able to bring forth more stimulating classrooms and pedagogy. Do the Puzzle Pieces Comprise Coherent Reform? If the current California school reform move- ment is in some ways a departure from earlier ones—particularly in terms of scale—it does appear eerily similar in one less than desirable way. While many of the reforms are sensible enough when considered as isolated compo- nents, there is the threat that they will never cohere into the program of systemic reform that is truly needed to improve student achievement for {111 California Students. Fragmentation has long hampered the state’s education system, and it may do so for a long time to come. Of course, policymakers have long been aware of the need for systemic reform and have made serious efforts to push it forward. Responding to the limitations of Single compo— nents of reform such as standards and site— based management to improve Student achieve- ment, policymakers and educators in many states, including California, began to create during the 1990s a reform agenda that takes into account the need to move on several fronts ZOOO California’s Ambitious Education Reform Agenda: Will It Energize Schools and Teachers? at once. In California, many local districts, sometimes acting on their own initiative, moved toward systemic, coherent reforms. They realized that improving curriculum, establishing new roles for teachers, and devel- oping school—level structures to support teach- ing and learning were each pieces of a solution that had to be addressed concurrently, not iso— lated topics to be sequentially cycled through policy mechanisms. ()n the state level, proponents for what has become known as standards—based reform pro- posed four key interrelated reforms aimed at fostering student mastery of more rigorous, challenging academic content: ' establishing challenging academic standards for what all students should know and be able to do; ° aligning policies to these standards, such as testing, teacher certification, textbook selec- tion, and professional development; ° restructuring the school governance system so that schools and districts are delegated the responsibility for developing specific instruc— tional approaches that meet state academic standards; ° developing accountability mechanisms so that districts, schools, teachers, and students will all be held responsible for improved aca— demic achievement. Although many California business executives and educators have espoused this model of sys— temic reform, it has so far been more success— fully implemented in other states such as Connecticut and Kentucky In fact, in some ways it can be said that California has jumped ahead on implementing—somewhat impetu— ously, in l’ACE’s view—the accountability com— ponent before the alignment and governance issues have been adequately dealt with. Consequently, California is currently saddled with a high stakes accountability system based on a single measure, the Stanford 9—3 stan- dardized test that has little correlation with the state’s academic standards. Some of the state’s difficulty in fostering sys- temic reform lies, as the following chapters make clear, at the margins of, and even outside of the direct purview of, K-12 policy. The pupil population, for instance, continues to grow at over 80,000 a year, making it difficult for edu— cators to focus on quality systemic reform while trying to accommodate such large numbers. Heightening the challenge for educators is the fact that many of these children live and attend school under very difficult circumstances. Statewide, the share of children living within impoverished families has climbed 24 percent since the late 19705, now standing at one mil— lion youngsters in the state. Over the same period reading scores have dropped to the same dismal levels observed in Alabama and Mississippi, demonstrating that poverty played a major role in this drop. Clearly, as Chapter Two argues, the state must work not only at improving education policy, but at improving the living conditions of California’s poorest families and children. The surge in enrollment, combined with class—size reduction, has also resulted in a seri— ous shortage of high quality teachers; in some California school districts—especially those serving the neediest students—over 30 percent of the faculty are serving on emergency creden— tials. Such inexperienced, unprepared teachers often have a difficult time surviving from one day to the next, much less trying to implement reform policies they scarcely understand. PACE I Still, many of the obstacles to systemic K—12 reform are as internal as they are external; PACE believes that policymakers simply have not done all they need to do in creating a truly coherent approach to school reform. Systemic school reform, for instance, is supposed to be based upon assessments aligned to rigorous academic standards, something California is far from accomplishing. As noted above, the stan— dardized test that California students are required to take — the Stanford 9 — is not at all aligned to the state’s standards, although aug— mented test items from the standards are being added each year. Still, some educators wonder out loud if the assessments will ever be fully aligned to the standards. Indeed, the new accountability system puts educators in the paradoxical and scarcely tenable position of being judged on Stanford 9 scores that don’t reflect the curriculum students are supposed to be learning. Politic-ally, things began to look promising in 1999 in terms of improving the state’s frac— tured state education governance pattern and aligning the system. Governor Davis took charge of the executive branch machinery, while his Democratic party was firmly in con— trol of both legislative branches. This would have seemed to be a great opportunity to cir— cumvent the incoherence and implementation failures that confronted the \Vilson administra- tion; after all, \Vilson faced a hostile Democratic legislature and had a fractious rela— tionship with the Democratic State Superintendent of Schools, Delaine Eastin. However, PACE does not feel that a true plan of policy alignment and coherence has yet emerged from the state despite the many initia— tives it has launched. California state policy, as CRUCIAL ISSL‘ES 1.\' CALIFORNIA EDL'cATIoN this edition of Crucial law/ex makes clear, still has many obstacles to overcome in developing an education policy that sets clear objectives for schools and supports those schools with suffi- cient resources and autonomy. A Summary of the Chapters Chapter Two, “Early Education and Family Poverty,” argues that California K—12 education reform, even if it does achieve coherence, will be of limited effectiveness unless issues pertain— ing to family poverty and inadequate early edu- cation are more fully addressed. \Vhen poor children enter school they are two to three years behind their more affluent peers in almost ever measure, which does not bode well for a state in which 26 percent of all chil— dren live in poverty. Preschool programs, the authors demonstrate, can make a significant difference in closing the gap, but in California they are hampered by uneven quality. varying affordability, and a weak coordination system that has different state agencies administering different programs. Even when good preschools and child care are available, parents find it difficult to get the necessary information about them. The authors offer recommenda— tions for improved early education, including the establishment of links to K—13 reform. Chapter Three, “The Schooling of English Learners," ponders the question of how we can better educate the 35 percent of California stu— dents who are English language learners. Complicating the challenge, the authors demonstrate, are the high poverty rates among the families of English language learners, the (the 7‘)" __/ still uncertain effects of Proposition ZOOO California’s Ambitious Education Reform Agenda: Will It Energize Schools and Teachers? 1998 English-only initiative), the pressures of high—stakes testing, and—most important of all—the shortage of high—quality teachers. Only one—third of English language learners had cer- tified teachers in 1998, partially on account of K—S class—size reduction that siphoned the most qualified teachers from schools serving poor students to those serving the most affluent. Until an adequate number of well—trained teachers can be secured, the education of English language learners will be in jeopardy. Chapter Four, “School Finance,” argues for the reconsideration of a state education funding system that does not, at the present time, have a strong connection to California’s educational objectives. Categoricals now consume 39 per— cent of state education funding, which means that tnany of the dollars going to schools are already accounted for before they even reach the schoolhouse door. While the authors don’t suggest that categorical funding should be eliminated— that is neither politically feasible nor desirable on account of equity considera— tionsfitl‘tey worty about the constraints an ever—eVpanding number of categoricals place on schools that need flexibility in order to improve student learning. The authors also discuss the need for policymakers and legislators to define what “adequate” education funding means in a state that increasingly demands it but yet can— not say how additional resources would be aligned with educational goals. Chapter Five, “Governance and ;\ccountability,” demonstrates how local school districts and boards have lost a significant amount of power over the last thirty years, as much of the decision—making regarding school accountability, curriculum, and finance now rests with the state. Yet despite this centraliza— tion and the increasing power of the governor, the California education system sometimes appears headless, as “no single entity or indi— vidual has the authority to set the course for education reform.” The California public edu- cation governance system is deeply splintered with the governor, legislature, state board, California Department of Education, and other entities having influence over different pieces of education policy. The authors argue that such governmental fragmentation tends to undermine efforts to put forth a coherent pro— gram of reform. Chapter Six, “Teacher Quality,” analyzes the paradox California finds itself in. On one hand, the state has made important strides in impor— tant areas such as setting professional standards for teaching and expanding mentoring pro- grams for beginning teachers; on the other hand, California continues to be plagued by an escalating shortage that has placed thousands of emergency—permit teachers in the schools serv- ing our poorest, neediest students. Qualified math and science teachers are particularly diffi— cult to find, as public education cannot com— pete with the salaries in a booming high tech economy. VVhile the authors see no “quick fix,” they do offer a number of long—range strategies to improve teacher recruiting, professional development, and overall quality. Chapter Seven, “Student Assessment and Student Achievement in the California Public School System,” portrays an assessment system that is still evolving, albeit tentatively, to a stan— dards—based system. The current high-stakes assessment, the normative Stanford 9, is not only not aligned to the state’s academic content standards, but provides a very limited “snap— shot” of student achievement in California. PACE I While the Stanford 9 scores generally show California students achieving satisfactorily, scores from other measures, such as the highly regarded National Assessment of Educational Progress (NAEP) are still substantially below average. The authors argue for an assessment system that weighs more than a single measure, cautioning against an overvaluation of stan- dardized test scores that generally do not reflect what we most want students to know and be able to do. Chapter Eight, “Connecting California’s K— 12 and Higher Education Systems,” explores how deep disjunctures between the two systems send confusing signals to students preparing for post—secondary education. As matters currently stand, California students have to take many hours of standardized tests that count for little when they apply to college. And, once accepted to college, they have to take placement exams that are not aligned from one institution to the next, much less to the California high school curriculum. The author recommends the estab— lishment of a K—lo policymaking body that can build bridges between the two systems. Chapter Nine, “Alignment Among Secondary and Post—Secondary Assessments in California,” examines the alignments and mis— alignments in six different types of commonly used tests. Some math tests, for instance, emphasize contextual problem—solving, whereas others emphasize abstract procedures. Some reading tests emphasize the ability to draw inferences, whereas others ask for deeper analy— sis. Some of the misaligmnents between tests are inevitable, the authors argue, as one cannot expect a basic-skills test to emphasize the same skills as a college entrance examination. CRt'clAL lSSL‘ES 1.\' CALIFORNIA Euro-t'riox Nevertheless, the authors draw on research to suggest that many of the misaligmnents are confusing and harmful to students who receive mixed signals regarding what kinds of skills and knowledge are of primary importance. The Need for Coherence and Capacity Building While this edition of Crucial [XX/It’s demon- strates that many of the recent reform efforts are fragmented and incomplete, we at PACE don’t want to sound unduly pessimistic. After all, considerable progress has been made over the last three years. Standards are complete and there is at least some movement toward the alignment of other policies. Furthermore, there is some evidence that the culture of teaching and learning in California is beginning to change. Teachers and schools are focusing more intensely on student achievement, and increasing numbers of students are beginning to understand and believe that how they per- form in school will have consequences for their lives beyond school. Still, we believe that more steps must be taken in the next few years—steps that will fos— ter improved student achievement without overburdening school and school districts with more state policy directives. The shift must be made from creating new reforms to helping schools and school districts effectively imple— ment the ones already in place. Most important of all, in our view, is the need for more capacity building: There is a profound mismatch between the demands that are being placed on teachers and students and the resources they ZOOO California’s Ambitious Education Reform Agenda: Will It Energize Schools and Teachers? have to meet these demands. California policy— reforms and building the capacity of schools makers are well aware of the need for such to implement them rather than on adding new things as more counselors, better professional ones to an already very full slate. After the development, increased teacher pay, and high— reform frenzy of the last few years educators quality teachers for our poorest schools, but need breathing space, not an onslaught of new will find it difficult to make up quickly for the initiatives. California’s schools must now be many years of declining educational resources. given the time, opportunity, and resources We urge, then, that policymakers concen— they need to succeed. trate on bringing coherence to existing PACE W Chapter 2 Early Education and Family Poverty Elizabeth Burr and Bruce Fuller PACE Why Should Family Poverty Concern All Californians? ecent debates around accountability in the K—12 system too often ignore the fact that learning begins long before children enter kindergarten. Research demonstrates that chil— dren with insufficient early learning opportuni— ties are likely to start school with serious deficits in knowledge and basic skills that are difficult to overcome in later years. Consider these facts: ° Black six year olds (who are disproportion— ately poor) read at the level of white five year—olds.‘ 0 Half of the variability in high school seniors’ test scores is due to differences that were already apparent in first grade. 0 About one—third of young children do not attend a formal preschool program, largely because of a shortage of servicesf “Ihile poverty is the strongest predictor of school achievement, and the central reason children fall behind at an extremely early age, quality child care and preschool can mitigate some of its effects and prepare youngsters for kindergarten. In Texas, where an early reading initiative is in place, the persistent gap between minorities and whites has closed remarkably. Students’ test scores in Georgia also have risen since an early literacy program was launched. As effective as such programs can be, the ability to find and pay for quality preschool poses a serious problem for many families. This is particularly true in California, where one in four children live in poverty—a higher rate than in all but seven states." This leads to a couple of critical questions this chapter will strive to answer: In a time of unparalleled pros- perity, how can California leave so many chil- dren behind? W’hat policies could help reduce poverty and raise school performance? A I/Videnz'ng Income Gap Poverty is commonly perceived as a welfare problem, but increasing numbers of working families are struggling to pay for housing, health care and child care. Indeed, with the high cost of living in many California commu— nities, even some middle—class families have trouble making ends meet. Therefore, efforts to end the welfare system will do nothing unless they include strategies to end child poverty. Instead of benefiting all families, the boom— ing economy has exacerbated an already trou- PACE H 9 bling income gap. Between 1969 and 1994, income inequality rose more sharply in California than in the nation as a whole a trend that continues.4 The gap between the top and bottom fifth of California families has grown by 77 percent since the 705. From the mid—19805 to the mid—19905, that gap widened faster than in all but four states, while the gap between the rich and the middle class increased faster than in all but three states.‘ The middle class has been squeezed, shouldering a heavy tax burden and benefiting from few of the pub- lic programs aimed at lower-income families. Unequal incomes lead to unequal opportu— nity for families who are struggling harder than ever to provide for their children. Increasingly, low—income working families are forced to choose between buying food or going to the doctor. Due partly to rising income inequality, child poverty in California has been climbing at an alarming rate, from 20.8 percent in 1987 to 26 percent in 1999." This means that nearly 2.5 million children are poor, living in households 35% 30% 25% 20% ' 15% 10% , -------- 5% 0% Poverty for All Ages Poverty for Poverty for Children 5-17 Children 0-4 earning $16,450 or less for a family of four. Many of those 2.5 million live with adults who lack adult self-sufficiency; live with adults who are self-sufficient but lack the additional income to also support their children; or live in poverty because of the demographic structure of their household. Together with New York and Texas, California has a higher rate of child poverty than the nation as a whole. The grow— ing income disparity stems in part from California’s large immigrant population and its high numbers of children with single mothers who have little education. Family structure and race are predictors of poverty. Blacks and Latinos are more likely than whites to be poor. Nationwide, children who live with only one parent are more likely to be poor than those who live with two parents. The long-term effects of poverty are grave. Poor children are also more likely than kids from affluent or middle class families to experi— $140,000 $120,000 $100,000 $80,000 $60,000 Average Incomes $44,270 $40,000 40 3‘0 $20,000 $11280 $90330 $0 Middle Poorest Fifth Fifth Fifth Population Highest I Mid 1980s Mid 19905 Figure 1. Poverty Among Age Groups (:RL'CIAL Issues IN CALIFORNIA ICI)LI<:A'I‘IoN Figure 2. Worsening Income Inequality in California 2000 ence poor health, to die during childhood, to be retained or drop out of school, to give birth out—of-wedlock, to experience violent crime, to grow up to be poor adults, and to suffer other undesirable outcomes. Research on early learning and child devel— opment illustrates the connection between ris— ing child poverty and low test scores. Children who have not had access to stimulating envi- romnents or education opportunities lag behind children who have. Again, race and family status predict these circumstances. For example, participation in preschools is substantially lower for Latino families than for other ethnic groupsf And 49 percent of children with one parent live below the poverty level in California, compared to 19 percent of children in two-parent families.“ Furthermore, poverty exerts a cumulative dis— advantage as children continue in school. As more children qualify for free or reduced—price meals in California (48 percent in 1998), their ability to succeed in school goes down. So without programs that assure school readiness, achievement will remain low for an unaccept— ably large segment of the student population. “Vithout continuous, targeted government intervention to strengthen the financial stability of all families, including providing equal oppor— tunities for early learning, neither the academic achievement of today’s students nor the state’s productivity will reach its fullest potential. A Brief History of Child Care and Early Education Investments California has long been a leader in recogniz— ing the value of high-quality child-care and Early Education and Family Poverty development programs for infants, toddlers, pre—kjndergarten children, and school-age chil- dren from low—income families who are work— ing or receiving public assistance. The state has prioritized child-care subsidies for low—income families, behind only programs for abused or neglected children. In addition to higher fund- ing levels, California offers an impressive vari- ety of programs, including publicly funded pro- grams, nonprofit agencies and private providers. However, too many state and federal efforts have been developed in a haphazard, fragmented, and decentralized way. Partly as a consequence, the number of available child- care spaces has been insufficient, resources inadequate, and the quality at facilities uneven. The state preschool program was first fund— ed in 1965 through school districts to prepare low-income children aged 3 to 5 for school. In the 19705, Schools Chief Wilson Riles expanded federal efforts to strengthen early education for about two—thirds of the state’s children in kindergarten through third grade with the Early Childhood Education plan. The program aimed to ensure mastery of reading, stronger instruc— tion in writing and math, teacher training, and greater parental involvement. Current efforts to bolster reading mastery by the end of third grade may have stemmed from Riles’ vision. More recently, grants to public school dis— tricts and preschool programs, along with a national child—care block program in 1990, ush— ered in a shift toward parental vouchers. In 1992, California ranked sixth in the nation in per—child expenditures on child care and early childhood development, and in 1994 it ranked fifth in the percentage of state tax revenues spent on these services." Showing its commit— ment to early education, the state increased its PACE I 11 child—care funding from the Department of Education (CDE) and Department of Social Services (DSS) from $324.4 million in 1987 to $2.4 billion in 1997; this figure excludes federal Head Start Centers, on which about $400 mil— lion is spent per year in California.” The child— care and development system in California con- tinues to be the largest and most comprehen- sive in the nation with funding well over $1.2 billion dollars for 1998—99. Need for Greater Coordination Even so, significant coordination problems arise when different agencies administer child- care programs for welfare recipients and work— ing poor families. DSS has been responsible for most federally funded TANF—related child-care programs, which are state—supervised and coun- ty-administered.11 CDE is in charge of all state— funded child care and early childhood develop— ment programs as well as the federal Child Care and Development Block Grant (CCDBG). Yet the two agencies have different goals: DSS views child care as a support service to help welfare families become self—sufficient, while CDE considers child care a critical com— ponent of education. The programs also vary in terms of eligibility requirements, maximum payment amounts, eligible providers, priority groups for subsidies, and time limits. These dif— ferences obstruct families’ efforts to maintain a stable child—care arrangement when moving on and off welfare or from program to program. They also create institutional and political impediments to a streamlined universal preschool program. Added Demands of I/Velfare Reform The state is now in its third year of implementa— tion, starting in September 1999, of the CRUCIAL ISSUES IN CALIFORNIA EDUCATION California Work Opportunity and Responsibility to Kids (CalWORKs) program, which aims to move families from welfare dependency to work and self—sufficiency. Under CalWORKs, partici- pants must engage in work and/or work prepara- tion activities and are provided with an array of welfare—to—work services, including child care. Putting 500,000 welfare recipients—single mothers—to work will continue to strain an already strapped child—care system. Even though California augmented the federal block—grant funding to boost child-care investments, the state is not meeting increased demand. As more women, especially single mothers, enter and stay in the workforce after having children, increasing numbers of families depend on non—maternal care for their infants and children. In 1975, 39 percent of mothers with children under six worked outside the home; today, the number is 62 percent.” Additionally, growing numbers of parents— across social classes and ethnic groups—have become more determined to advance the early development and learning of their children. Welfare recipients face difficulties navigating the complicated child—care system. Often recip— ients are not sufficiently informed by their caseworkers of the options available to them, in part because of changing requirements. But low—income working families in California also face major barriers to finding child care. These working families—who pay a sliding-scale, income—based fee for child care——have difficulty accessing services because in most counties there is no centralized place to apply. Once they apply, families often face long and uncoor— dinated waiting lists.“ With an inadequate sup- ply of child-care providers, parents often wait years for child care, paying out—of—pocket for care and putting their jobs in jeopardy by 2000 sometimes having to stay home to care for their children. It is estimated that only 20 percent of eligible families receive subsidies to help pay for child care in some counties.H In fact, some families may not even be aware that they are eligible for subsidies. Early findings from a PACE longitudinal study show that a number of factors, such as a mother’s race or ethnicity, the age of her child, and whether other adults live in the home, influence whether the mother will use a child—care subsidy.“ Fortunately, political leaders are aware of the importance of educational opportunities for the youngest Californians. Former Governor \Vilson greatly expanded child care and preschool opportunities for low—income chil— dren. And Gray Davis has supported the effort with increased funding for materials and read— ing instruction. State Superintendent Delaine Eastin has been the boldest of all, asserting that only universal preschool will ensure that all children start school ready to learn. Another bright spot in terms of investment in young children was the first round of funding from the California Children and Families Commission, which was heavily targeted on early education, child care and literacy. The State Commission recognizes that in addition to equalizing access to quality programs, more providers must be trained and incentives must be provided to sta— bilize the child care workforce. Poverty and the Link to Education Reform The effects of poverty on school achievement, school completion, and adult earnings are well documented, and are even stronger than the Early Education and Family Poverty effects of single parenthood on these factors.“i The gap in achievement between poor and better—off children is apparent as early as kindergarten, and nationwide, six out of ten low—income students in fourth grade cannot read.“ Furthermore, the consequences of poverty are long lasting. One study found that the connections between poverty, parental stress, family conflict and harsh parenting account for most of the differences in school grades among tenth graders. Since 1964, studies have shown that early experiences in the home shape learning ability.” And recent brain research shows that children’s use of language originates long before they speak their first word; by their sixth month, infants are already cracking the language code. Since most brain development occurs between birth and five years, early and sustained educa— tional activities are critical to school readiness. When poor children enter school they are already two to three years behind their more affluent peers in almost every measure, since early academic performance is dependent on non—school experiences. Better—off parents can afford to move into neighborhoods with higher quality schools. Having more discretionary time, they also are better able to read to their children, to assist their children with home- work, and to engage them in a variety of intel— lectually stimulating activities from the preschool years forward. Since learning begins five years before chil— dren enter kindergarten, early literacy experi— ences provide a major boost. But poor and minority families are less likely to engage in lit— eracy activities than their wealthier and white counterparts. Nationally, white parents were more likely (90 percent) to have read to their PACE I 13 children three or more times per week than Hispanic parents (65 percent). The education level of parents also influences the frequency of literacy experiences at home. And more parents with children enrolled in preschool visited a library in the past month (43 percent), as com- pared to parents whose children were not enrolled (32 percent). While it is tempting for politicians these days to blame schools for low test scores, socioeconomic background is the biggest pre- dictor of school performance. Since the lowest performing schools are the ones with the great— est share of poor students, offering these chil- dren the same early enrichment as their affluent peers would help to even out the achievement disparity. And since better educational opportu— nities are one way to reduce the growing income gap, there is a clear imperative to ensure equal access to quality preschool programs. The Effects ofPoverly on Learning 0 Fewer resources. Children who attend schools with high numbers of poor students are more likely to have under—qualified teachers and less access to preschool, early reading programs, counseling, and small class sizes. Often a culture of low standards and expectations prevails. 0 Lower test scores. Recent achievement results (STAR) show that the state’s low— income children—those who qualify for free or reduced-price lunches—score on average half as high as the test-takers who are not classified as “economically disadvantaged.” In reading, 22 percent of low—income fourth— graders scored at or above the national aver— age, compared with 56 percent of fourth— graders who are not economically disadvan— CRL‘CIAL Isst'Es IN CALIFORNIA EDUCATION taged. In math, 33 percent of low-income sixth-graders met or topped the national average, compared to 56 percent of better- off sixth-graders.” Poverty is also related to SAT achievement. Studies have shown that for every $10,000 increase in income (through $100,000), there is a corresponding rise in students’ verbal and math SAT scores. Low-income students were five times less likely to go to college than their better—off counterparts. As noted earlier, educa- tional attainment is Inore important than ever in earning a livable income. So, if increasing num— bers of low-income students are not college- bound, they are not likely to escape poverty. Less school engagement. Children in lower— income families are also more likely than those who live in wealthier households to have behavioral and emotional problems, and they are less likely to be “highly engaged” in school. For example, 41 percent of children represent— ing all income levels cared about doing well and willingly completed homework assign- ments. But only 34 percent of lower-income students were engaged in school. But it is important to note that lower—income parents still read to their children and help them par— ticipate in activities outside the classroom, such as clubs and sports. These rates of parent involvement in school—related activities vary by race/ethnicity, parent education level, and fami— ly structure?" High—Quality, Affordable Child Care in Short Supply While finding a child—care space is difficult in California, its cost may be even more prohibi— ZOOO tive. Middle—class families must spend dispro— portionate percentages of their discretionary income on child care, as compared to high- income families. And working—class families who earn 75 percent of the state median income (approximately $28,000 per year) do not qualify for subsidies. Therefore, while high—income families pay for private care, and low—income families are eligible for public assistance, the working and middle classes are hit hardest in the search for quality and afford— able child care. The average annual cost of child care for a 4 year—old in an urban area is much higher than the average annual cost of public college tuition. For example, the average annual cost of center—based child care for a 4 year—old in Alameda County is $6,032, whereas the average public college tuition is $2,731.xxi Parents of infants pay even more. In California, the average annual cost for infant care in a cen— ter is $7,812.33 In any case, the cost is high whether parents choose center or family child care, whether they live in an urban or rural area, or whether they have infants, preschool— ers, or school—age children. As mentioned earli— er, state funds are not able to cover the child— care costs of all the families who are eligible to Early Education and Family Poverty receive it: Only about a fifth of qualifying Californians receive child-care subsidies. Unequal Access to Child Care In France, 99 percent of all 3 and 4 year—olds attend preschool. But in California, only about half attend some form of partial or full—day child care, and as many as 75 percent of these pro- grams are not developmentally or educationally appropriate.“ This is partly parental choice; some families prefer informal child—care settings. And some parents stay home with their children. But the state’s lower child-care enrollment is also a result of insufficient child-care spaces. Several factors drive the local preschool and child—care supply: the size of the child popula— tion; the maternal employment rate and family income; the average family social characteris- tics (including incidence of female—headed households, maternal education level, share of non-English speaking parents, and family propensity to enroll in public assistance); and the number and strength of community/reli— gious organizations.24 Not surprisingly, child—care supply is uneven across neighborhoods and counties in California, depending on income. W hen wel— . For a Family at Minimum Wage ($1 1 ,960/year) I For a Low-Income Family ($30,000 or less) I For a Family at Median Income ($38,979) Figure 3. Portion of Income Needed to Pay for an Infant in a Licensed Child Care Center Source: The California Child Care Portfolio 1999, California Child Care Resource & Referral Network PACE I 15 16 fare caseloads in California dropped from about 902,000 in January 1994 to about 639,000 January 1999, more mothers with young chil- dren entered the workforce, and more child— care spaces were needed in neighborhoods with the scarcest supply. Families on public assis— tance and low—income working families have less access to child care than wealthier families. While access is limited in the poorest neigh— borhoods, blue—collar families are actually hit the hardest.” They can’t afford not to work, yet even average-priced child care is unaffordable. These parents must choose lower-cost, lower- quality care. This means that the children most in need of child care to provide a strong start for school are the least likely to find it. Uneven Quality in Child Care There is a clear need to augment child—care supply, especially in low—income neighbor- hoods. But special attention must also be paid to program quality. High-quality programs have been shown to increase a child’s chance for school success. They are characterized by well—prepared and well—compensated providers; a low provider/child ratio; developmentally and age—appropriate practices; parent involvement; links to community services, such as health care or parent education; strong management and administration; and a safe, healthy, comfortable environment. Research has shown that quality programs enable preschoolers to enter kinder— garten with the skills needed for coping with 2O 18 03 O) A i\) O Alameda Kern I Number of Slots per 100 Children ill Los Angeles Sacramento San Diego San Francisco Statewide The number of available spaces in center and preschools per 100 children under 5 varies widely from county to county. Data are county averages for 1998. Figure 4. Total Supply of Licensed Child Care Per 100 Children Source: California Child Care Resource and Referral Network, 1998 and State of California, Department of Finance. December 1998. (:RL'CIALISSL'ES1N CALIFORNIA EDL'CsYI‘ION lOOO school tasks. These children show greater knowledge of verbal and numerical concepts, receive higher ratings on social competency, show greater task orientation, are more likely to make normal progress throughout the pri— mary grades, and are less likely to be held back or placed in special education classes.“ Similarly, low-quality programs can actually do harm; programs with insufficient funding, high staff turnover, and poor management can impede children’s potential to learns“ It is also important to situate welfare—poor and working-poor families within their quite variable neighborhoods. Neighborhood and metropolitan contexts vary in terms of employ— ment opportunities, ethnic composition and child—rearing norms, and organizational infra— structure. VVhile there has been much long—term research on early—childhood education pro— grams serving poor and at—risk children, a new study from the National Institute of Child Health and Human Development (NICHD) is described as the first to show a link between the quality of child care and children’s school performance. The four—year study, called The Children qffbe C0512 Quality, and Outcomes 51‘1le (In 7?) Sr/Jool, followed children through second grade and demonstrates that higher quality child care is related to fewer reports of problem behaviors, higher cognitive performance, high— er language ability, and higher levels of school readiness”. The study also found that children who attended centers with higher~quality classroom practices had better language and math skills in their preschool years through second grade than children in the study who attended centers that provided lower—quality care. This supports Early Education and Family Poverty the belief that high-quality child care not only increases school readiness but also helps chil— dren continue to succeed. Furthermore, chil- dren in the study who had closer relationships with their child-care teachers had fewer prob- lem behaviors and better thinking skills. According to the researchers, warm teacher- child relationships also had some influence on children’s language and math skills through grade two, but those effects were not as strong. The study found an even stronger relationship between high-quality care and the cognitive and emotional development of children whose mothers had a high school education or less, an important finding for California. As mentioned earlier, mothers who did not complete high school are much more likely to be poor than those who have a college degree. High—quality child care can mitigate the effects of poor home environments lacking in educational materials or other pro—development activities. Factors such as family income, mothers’ psy- chological well—being, and maternal behavior have more of an influence on children’s social competence at two and three years of age than does the children’s child—care arrangement. However, quality child care was related to greater social competence and cooperation in children and less problem behavior at two and three years of age. Also, more group experi— ences predicted more cooperation with other children and fewer problem behaviors at both two and three years of age. Finally, the consis— tency of the child—care setting also played a role in the development of social competence. At age 2, children who had been in several differ— ent child—care arrangements showed more problem behaviors than did children who had been in fewer day—care arrangements.” If PACE I 17 18 California wants all its children to be ready for school, it Inust improve both the quantity and quality of child—care experiences. A Fragmented Child Care Governance System California’s child—care system is highly decen- tralized, with programs that are state-super— vised and county—administered. At the state level, agencies set overall policies, determine eligibility criteria and benefit levels, monitor local practices, and provide technical assistance to counties to ensure that state policies are fol- lowed. Because counties have varying amounts of administrative flexibility within these para— meters, problems arise around decision—mak— ing, authority, and fiscal responsibility. As a result, thin data exists on child—care capacity, the number of organizations operating, the number of children served, and the quality of teachers and staff. No single agency knows how many providers benefit from child—care vouch— ers. California taxpayers have been supporting a $1.2 billion preschool and child—care industry with almost no information about the supply of organizations, the individuals serviced, or the quality of services. The child—care program is also highly frag- mented, as it is delivered in three stages. Stage 1 is administered by the Department of Social Services, while Stage 2 and Stage 3 (Set—Aside) are administered by CDE through its certifi— cate-based Alternative Payment Programs. Eligibility for federal and state subsidized ser— vices will continue to be based primarily on income and need, with additional criteria depending on program type and fund source. CRUCIAL ISSUES IN CALIFORNIA EDUCATION In order for CDE and D53 to help transition families off welfare, and to help low-income families remain self—sufficient, they must work collaboratively to develop a streamlined and consolidated state plan for child care and devel- opment services that meets the needs of California’s families and children. Hopeful Policy Directions California families under 200 percent of the poverty line fall below the national average on indicators such as employment rates among parents, health insurance, the frequency of reading and storytelling to young children, and the ability to afford food.m With such variation in opportunities for young children, an undeni— able need exists for integrated policies that reduce the harmful and far—reaching effects of economic instability. Essential to mediating the state’s bifurcated labor structure would be an equal opportunity agenda that includes plans for family support in addition to accountability and tough standards for education. Following are some recommendations to this end. 0 Equalize access to child care and preschool for all children in California. The state has 1.13 million children aged 3 and 4, only a portion of whom attend pro— grams that boost their academic and social skills. This is partly because child—care supply is highly uneven across and within California counties. Parents in Los Angeles are half as likely to fInd a preschool or child—care center slot for their youngsters as those living in San Francisco?l Even in counties where the sup— ply is higher, preschools are inequitably dis— ZOOO tributed between affluent and blue-collar communities. Furthermore, the number of young children statewide is rising more rapidly than the preschool system’s capacity to keep up. Preschool spaces grew just 2.2 percent in the average community between 1996 and 1998, while the child population grew over 10.8 percent. Efforts should be made to expand child care capacity in the neighborhoods and language communities where supply is most scarce. Offer aid for working class and lower- middle—class families. Affluent parents enjoy access to three times as many child— care spaces as blue—collar and middle-class families. Less well—off parents lack purchas— ing power for private centers but are not eli— gible for aid to pay for preschooling. The fact that so many Californian children live in or near poverty despite the presence of working parents demonstrates that poverty is a mainstream problem, affecting children from all racial and ethnic backgrounds, from all regions of the state. Providing low—cost health insurance, wage supplements to low— income working families, job opportunities, work—related services, and child—care assis— tance to working poor families would enable them to secure steady employment and achieve financial stability. Only with addi— tional supports can these parents keep a job and ensure that their children are healthy and cared for. Develop a Nlaster Plan that includes exploring the feasibility of universal preschool. State Superintendent Delaine Eastin’s plan for universal preschool, which would slowly phase in free early education Early Education and Family Poverty programs starting with low-income commu— nities, is consistent with the state’s commit— ment to higher expectations and standards for all students. Necessary to its success would be stabilizing and diversifying the preschoool workforce. In addition, efforts to improve the ratio of child-care providers to children, lower group sizes, increase caregivers’ levels of education, and increase the safety and intellectual stimulation of child-care settings are needed. A recent RAND study found that for every dollar spent on early childhood pro— grams, society later saved several dollars on welfare, special education, and criminal jus— tice.” In a time of economic plenty, it makes sense to expand early education programs that have lasting effects. We cannot raise the reading proficiency of California’s 9 year—olds until the opportunity to learn the basic build— ing blocks of language becomes equally avail- able. While child—care expansion highlights the need to simplify the governance struc- ture, the State and County Proposition 10 Commissions are creating another layer of governance. Nevertheless, they are moving ahead in innovative directions. Allocate dollars to improve child-care quality and increase capacity for CalWORKs and low-income working families. Offering safe and stimulating child- care opportunities to mothers moving off public assistance is critical to the success of welfare reform. Early findings from the PACE Growing Up in Poverty study show that, despite increased state funding, many moth- ers are not taking advantage of their subsi— dies.“ More resources should be targeted to improving access to information and case PACE I 19 management, so that mothers are locating and securing the child care they need while they move into the workforce. Support the economic stability of fami— lies. Increasing tax relief for working poor families, such as the earned income tax credit (EITC) for married couples, would give a hand to families still struggling to make ends meet. The EITC appears to be the most effective federal policy for providing low— income families with crucial annual savings. If every eligible family in Silicon Valley filed for the federal earned income tax credit, over $70 million more dollars would flow into their households, at no cost to Sacramento. Similarly, under a fourth of all eligible par— ents in the county receive adequate informa— tion about child—care subsidies, vouchers that now equal a third of a working poor parent’s take-home pay. Expanding health coverage for uninsured children would also boost the well—being of many low—income families, especially those who lose their Medicaid cov— erage when they leave welfare. Other forms of continued and targeted aid, such as cash assistance, child—care subsidies, and food stamps would ensure that children grow up to be healthy, cared for, and ready to learn. Link early education to school reform. The evidence is clear that school reforms beginning at kindergarten are too late. The early years are the critical period in child development where youngsters can reach or fail to attain a healthy start. CRUCIAL ISSUES IN CALIFORNIA EDucATION Committing to funding early childhood pro- grams and full-day kindergarten as part of education reforms would help close the achievement gap between poor children and their better—off counterparts. No matter how many education reforms descend from Sacramento, if children’s basic needs are not being Inet, improved learning cannot take place. Reading scores will not climb in the early grades as long as access to preschooling remains so unequal across and within coun- ties in California. Furthermore, new incen- tives to retain K—IZ teachers will continue to drain the pool of preschool teachers unless we build parallel efforts in early education. By all accounts, the country’s economy is booming, with nine years of unimpeded growth, plentiful jobs, lowered interest rates and a surplus federal budget. But not all Americans have watched their income rise along with the stock market. Despite the prosperous economy, the Governor’s empha- sis on education reform, and recent increases in child—care funding, most low—income chil- dren come to kindergarten less prepared than their more affluent peers. Until parents have the economic security and resources to aid their children’s path through school, test scores will not budge. Isolated and small— scale programs will do little to equalize opportunity and raise student performance levels unless the state is willing to focus on improving home environments and econom— ic stability for all our families. ZOOO Early Education and Family Poverty Notes 1. Christopherjencks and Meredith Phillips, eds., The Black-\Vhite Test Score Gap (\Ar’ashington, DC: Brookings Institution Press, 1998). 2. The Baby Boom Echo: No End in Sight (\Vashington, DC: US. Department of Education, 1999). 3. The State of Our Children (Oakland, CA: Children NOW, 1998). 4. Deborah Reed, California’s Rising Income Inequality: Causes and Concerns (San Francisco: Public Policy Institute of California, 1999). 5- Pulling Apart: A State—by-State Analysis of Income Trends (“Hishingtom DC: Center on Budget and Policy Priorities, 1999). 6. California Report Card (Oakland, CA: Children NOW, 1999). 7. Bruce Fuller, Fran Kipnis and Yvonne Choong, An Unfair Head Start: California Families Face Gaps in Preschool and Child Care Availability (Berkeley, CA: Policy Analysis for California Education, 1997). 8. Megan Gallagher and Sheila Zedlewski, Snapshots of American Families: Income and Hardship, Poverty Among Children (“hshingtom DC: Urban Institute 1997). 9. Rob Geen, \Vendy Zimmermann, Toby Douglas, Sheila Zedlewski, Shelley Waters Boots, Income Support and Social Services for Low—Income People in California (\Vashington, DC: Urban Institute 1999). 10. Fuller, 1997. 11. Temporary Aid to Needy Families (TANF) replaced Aid to Families with Dependent Children (AFDC) under welfare reform. 13. Gallagher and Zedlewski, 1997. 13. Casey Coonerty and Tamsin Levy, \Vaiting for Child Care: How do Parents Adjust to Scarce Options in Santa Clara County? (Berkeley, CA: Policy Analysis for California Education, 1998). H. Geen, 1999. 15. Bruce Fuller, Sharon Lynn Kagan, Gretchen Caspary et al., Remember the Children, \Vave 1 Technical Report (Berkeley, CA: Policy Analysis for California Education, 2000). lo. Greg]. Duncan and Jeanne Brooks—Gunn, eds., Consequences of Growing Up Poor (New York: Russell Sage Foundation, 1997). 17. .\like Clary, “Florida Becomes Lab for School Voucher Debate,” Los Angeles Times, 9 August 1999. IH. Benjamin Bloom, Stability and Change in Human Characteristics. (New York: John \Viley and Sons, 196—1). 19. .\lelissa llealy, “Less Poverty in 1997, Census Bureau Finds.” Los Angeles Times, 25 September, 1998. 10. Gallagher and Zedlewski, 1997. 31. Karen Schulman and Gina Adams, The High Cost of Child Care Puts Quality Care Out of Reach for iVIany Families. (“asliingtom DC: Children’s Defense Fund, 1998). 33. California Report Card, 1999. PACE I 21 1'? 23. Superintendent's Universal Preschool Task Force, Ready to Learn: Quality Preschools in the 21st Century (Sacramento, CA: California De )artment of Education 1998 . W 24. Fuller, 1997. 25. Bruce Fuller, Fran Kipnis and Patty Siegel, Child Care Indicators, Volumes I and II (Berkeley and San Francisco CA: Policy Analysis for California Education and the California Child Care Resource and Referral Network, 1999) 9 26. Carolee IIowes and (LE. Ilamilton, “Child Care for Young Children”, in Handbook of Research on the Education of Young Children, ed. Bernard Spodek. (New York: Macmillan, 1993). 37. Carnegie 'l‘ask Force on Learning in the Primary Grades, Years of Promise: A Comprehensive Learning Strategy for Children (New York: Carnegie Corporation ofNew York, 1996). 38. The NICIII) Study of Early Child Care (\Yashington, DC: National Institute ofChild Health and Human Development 1999). 29. NICIII), 1999. 30. Gallagher and Zedlewski, 1997. l. Fuller, 1999. .JJ ‘7 'J . Lynn :\. Karoly, Peter \Y. (ireenwood, Susan S. liveringham,Jill Hoube, M. Rehecca Killiurn, C. Peter Rydell, Matthew Sanders. and James Chiesa. Investing in Our Children: \Yhat \Ye Know and Don’t Know About the Costs and Benefits of Early Childhood Inten'entions ( Santa Monica, C:\: The RAND Corporation, 1998). 33. Fuller et al.. 3000. (:Rl'tilAl. lssL'i-ts 1.\' CALIFORXH IiDL‘ox’rloN 2000 Chapter 3 The Schooling of English Learners! Russell W. Rumberger and Patricia Gandara University of California Linguistic Minority Research Institute n increasing number of students entering California’s schools come from non- English speaking backgrounds. Although some of these language minority students enter school already proficient in English, the major— ity do not. These students are now referred to as English learners: There are several reasons why Californians need to pay careful attention to the schooling of language minority students in their public schools. First, language minority students now constitute more than one—third of all students in California’s schools—a proportion that will grow even higher in the future. Clearly, the success of California’s students and schools will increasingly depend on the state’s ability to suc— cessfully educate language minority students. Second, English learners require a special— ized curriculum and properly trained teachers to support their development of English litera— cy. Complicating matters is the fact that these students, even as they learn English, must also have access to the rest of the required academic curriculum if they are to keep pace with their English—speaking cohorts. Third, the education of English learners has been highly politicized. Controversy centers around the use of native language instruction— whether it is better to first develop the native language literacy of English learners and pro— vide initial academic content through bilingual education or, on the other hand, to simply immerse them in English and provide initial academic content through simplified English instruction. While existing evidence generally supports the bilingual approach, the research is hotly debated and far from conclusive regard— ing which general approach makes more sense for which students and under what conditions“ At the same time, there is a growing political movement in many states to mandate, through voter initiatives, English—only instruction. In June 1998, California voters approved Proposition 227, an initiative that greatly restricted the use of bilingual education. This chapter provides an overview of the schooling of English learners in California. First, we review the nature and growth of the language minority population. Second, we review the political context surrounding the instruction of English learners, focusing on the adoption and impact of Proposition 227. Third, we examine the nature of the teaching force for English learners. Fourth, we analyze the achievement of English learners. Finally, we conclude with several pending or emerging PACE A 23 issues that will continue to affect the education of English learners into the foreseeable future. The Growing Language Minority Population Many California students come from non— English speaking backgrounds. This is due, in large part, to the large number of immigrants in California. In 1997, 25 percent of California’s residents were born outside the United States, more than any other state.4 It is also due to differences in the rates that immi- grant families become proficient in English, which depends upon the opportunities for learning and using English in their daily lives.‘ Both federal and state laws require that pub- lic schools identify students who are not yet proficient in English in order to provide them with supplemental services. This is done as a two—step process. First, schools identify stu- dents who come from non-English speaking backgrounds through a home language survey that asks parents a number of questions about the language background of their child." If the answers to any of these questions indicate that the child comes from a non-English speaking background, the child is identified as a lan- guage minority student. The second step of the process is to assess the English language proficiency of the stu- dent. This is typically done with one of several language proficiency tests available from com- English Learner Fluent English Language Minority Total Proficient Total Enrollment Number Percent of Number Percent of Number Percent of Number Total Total Total Enrollment Enrollment Enrollment All Students Grade K-5 907,379 32% 257,409 9% 1,164,788 41% 2,836,042 Grade 6—12 515,529 18% 498,363 17% 1,013,892 35% 2,916,775 Ungraded 19,784 22% 2,591 3% 22,375 25% 91,294 Total 1,442,692 25% 758,363 13% 2,201,055 38% 5,844,111 Spanish-Speaking and Latino Students Grade K—5 759,845 60% 160,115 13% 919,960 73% 1,262,243 Grade 6-12 403,531 36% 316,883 29% 720,414 65% 1,107,629 Ungraded 18,177 43% 2,104 5% 20,281 48% 42,187 Total 1,181,553 49% 479,102 20% 1,660,655 69% 2,412,059 (% of all students) (82%) (63%) (75%) (41%) Table 1. California Public School Enrollment by Language Minority Status and Grade Level, 1999 Source: California State Department of Education, Language Census Summary Statistic, 1998-99 (Sacramento, California: author). Retrieved November 8, 1999 from the World Wide Web: http://www.cde.ca.gov/demographlcs/reports/statewide/ Icsum99.htm; California State Department of Education, Enro/lment by Ethnic Group, 1987—82 through 1998—99 (Sacramento, California: author). Retrieved November 8, 1999 from the World Wide Web: http://www.cde.ca.gov/demographics/reports/ statewide/ethstud.htm. CRUCIAL ISSUES IN CALIFORNIA EDUCATION 2000 mercial test publishers.‘ In kindergarten, when most students enter school, the tests only assess a student’s oral English proficiency. Beginning in second grade, the language proficiency tests evaluate both oral and written English profi— ciency. The tests usually rate students’ English proficiency at five or six levels, ranging from non—English speaking to fluent English speak- ing.“ If students can understand English as it is used in school for instruction, they are classi~ fied as Fluent English Proficient (FEP) and not provided any special services. If students are not sufficiently proficient in English to under- stand classroom instruction, they are identified as English learners (ELs). The California Department of Education conducts an annual language census each spring to count the number of language minority stu— The Schooling of English Learners and the teaching force that serves them. The 1999 Language Census identified 2.2 million language minority students in California, which represented 38 percent of the total student population in 1998—99 (see Table 1). About two—thirds of language minority students were identified as English learners and one—third as Fluent English Proficient (FEP), but these pro— portions vary widely by grade level. Among younger students, the vast majority of language minority students are English learners, while in the upper grades the proportions of ELs to F EPs are more nearly equal. This pattern reflects the fact that, over time, an increasing number of English learners become proficient in English and are reclassified as Fluent English Proficient. But as we point out below, the process of reclassification is far from straight— dents and to identify the instructional programs forward. 3,500,000 Actual Projected Latino 3,000,000 / // 'Total Language Minority 1,500,000 / , ’ ” T T ' h 1,000,000 / A / , Englis Learners / / / Fluent English ASIan/Flllpino 500,000 _, Black 0 1988-89 1998-99 2008-09 Figure 1. California Public K-12 Enrollment by Major Ethnic and Language Minority Groups, 1988-89 to 2008-09 Source: California Department of Finance, California Public K—12 Enrollment Proejections by Ethnicity: 1999 Series, Retrieved from the World Wide Web November 21, 1999: http://www.dof.ca.gov/html/Demograp/K12ethtb.htm; California Department of Education, Language Census Report for California Public Schools, various years. PACE I [\J l In California, three—quarters of language minority students——Inore than 1.6 million—are Latino and come from Spanish—speaking back- grounds (see bottom panel in Table l). The remaining language minority population comes from a wide variety of language backgrounds, with Asian languages (Cantonese, Vietnamese, and Hmong) being the next most common groups." The population of English learners in California’s schools has grown dramatically over the last fifteen years. Between 1983-84 and 1998-99, the number of English learners increased almost five times faster than the over- all student population (196 percent versus 43 percent).10 In 1983—84, one out of eight California students was an English learner— today it is one out of four. This proportion will likely increase in the future. According to pro— jections from the California Department of Finance, Latino enrollment in California’s pub— lic schools will increase more than three times as fast as overall enrollment (see Figure 1). And since the majority of Latino students come from non—English speaking backgrounds, this increase will likely result in a growing number of English learners in California’s schools. While the procedures for identifying lan- guage minority students and assessing their ini— tial level of English (usually oral) proficiency are relatively straightforward, the procedures for re—classifying students as fluent English proficient and instructing students to achieve English fluency are not. Until recently, district procedures for reclassifying English learners had to follow quite prescriptive state guidelines. But the California State Board of Education recently abolished many of those guidelines. ” CRUCIAL IsSL'Es IN CALll-‘ORNIA EDUCATION In the past, reclassification was based on multiple measures of both English proficiency and student achievement. These assessments were based on either commercial English profi- ciency tests or district-developed assessments. In addition, students had to perform above a certain percentile level (usually 35 to 36) on a norm-referenced test in reading in order to be reclassified as Fluent English Proficient. The achievement-level requirement was not only to ensure that English learners were proficient in English, but to ensure that they were minimally successful in school before losing all supple- mental language support. Critics have argued, however, that using even a relatively low cut-off on a norm-referenced achievement test sets too high a standard since the use of percentile mea- surement virtually ensures that a significant percentage of English learners can never meet the criterion. In effect, they would have to out- perform about one-third of native—English speakers in order to do so.” Even with the pre- vious standard of using the 36 percentile as a cut—off, however, six to eight percent of all English learners are reclassified as Fluent English Proficient each year. And over the last ten years, the number of English learners reclassified as English proficient has increased at almost the same rate as the overall popula— tion of English learners—100 percent (also see Figure 1). One subject of considerable debate concerns how long it takes for students to become profi— cient in English. The answer to this question depends on how English proficiency is defined and measured. Even based on the more com— mon approaches described above, the length of time is considerable. A good illustration comes ZOOO from a recent study of a school district in the San Francisco Bay area with a mix of Spanish and Vietnamese background students.“ The study examined the English proficiency and reclassification of a sample of 1,872 students in grades 1-6 who had entered the district as English learners in kindergarten. As Figure 2 shows, it takes longer for students to become proficient in written English than in oral English. By the end of fourth grade, after being in the district for five years, 90 percent of the students were classified as proficient in oral English. But it took seven years in the district for 90 percent of the students to be classified as proficient in English reading and writing. These findings probably understate the amount of time it takes to become proficient in English because the sample only included students who had been in the same district since kinder— garten. Research has shown that student mobil— ity increases the amount of time it takes to become proficient in English.” Other studies have found that the amount of time it takes to become proficient in English reading and writ— ing varies from six to ten years.“ Some scholars believe that existing indica— tors of English proficiency are insufficient to ensure the continued school success of English learners. They argue that to succeed in school, especially in secondaiy and postsecondaiy school, English learners need to acquire (mulc— Im't' ling/id), “the specific type of English entailed in reading and writing academic papers and in discussing academic issues.”“’ Academic English involves using specific linguistic func— tions of the language—such as persuading, arguing, and hypothesizing—that are not well represented in general measures of English proficiency. 'l‘herefore, English learners who The Schooling of English Learners may be classified as English proficient based on standardized English proficiency tests may not have acquired proficiency in academic English. Some of these students may even do well enough in secondary school to get into college, but they often encounter difficulty doing col- lege work. For example, at the University of California, Irvine, which enrolls the largest population of language minority students in the UC system, 60 percent of incoming freshmen failed the freshman writing exam in 1998.” Over 90 per- cent of these students were language minority 0,9 //I/:— 0.8 f/ / 0.7 / 0.6 f / 0.5 / 0.4 / 0.3 Criterion 0.2 Mean Sc0ie, 1 K 0.1 0.0 ‘1' " 1 2 3 4 5 6 Grade I Redesignation I Writing I Reading Oral English Figure 2. English Oral Proficiency, Reading and Writing Development and Redesignation Probability as a Function of Grade Level: One California School District Source: Kenji Hakuta, Yuko Goto Butler, and Daria Witt, How Long Does /t Take English Language Learners to Attain Proficiency? (Santa Barbara: UC Linguistic Minority Research Institute, forthcoming), Figure 8. PACE fl [\J students who had attended American schools for over eight years; furthermore, 65 percent of them had taken Honors and Advanced Placement English courses in high school. A similar pattern exists at the twenty—two cam— puses of the California State University System, where 65 percent of all entering Mexican American and Asian American students required remedial English in 1998.” These data suggest that even the most suc— cessful English learners—those who enroll in four-year colleges——-may not master the levels of English required in advanced academic set— tings. However, most English learners never advance that far. The reason is simply that learning English is difficult and learning acade— mic English is even more so. While ordinary or everyday English is learned both inside and outside of school, academic English is generally learned in school from teachers and textbooks, and only with proper instructional support.” Unfortunately, as we point out below, many English learners are not given the instructional support they need because of a lack of properly trained teachers who can provide support over a sufficient period of time. Proposition 227 and the Instruction of English Learners The rate at which English learners are reclassi— fied as English proficient and no longer in need of special services has become an important political issue in the larger debate about the schooling of English learners. Ever since the L111! 1‘. [Vié‘l7()/.\‘ (1974) decision, states and local school districts have been required to provide appropriate services to English learners. But CRUCIAL lSSL'liS IN CALIFORNIA EDUCATION the nature of those services has generated con- siderable controversy in many states, including California. The debate has focused on whether English learners should be instructed in their native language while learning English, or sim- ply instructed in English. California was one of the first states in the nation to enact a comprehensive bilingual edu- cation bill—the Chacon-Moscone Bilingual- Bicultural Education Act of 1976, which pro- vided detailed instructions to schools about the type of language support that should be provid— ed for English learners. By 1986, however, the existing California bilingual education legisla- tion had “sunsettet ” by not being reauthorized, so bilingual education programs continued under the authority of department of education regulations, which were shaped largely by fed— eral requirements. Numerous attempts were made between 1985 and 1998 to hammer out compromise legislation to restore the statutory basis for the provision of language support ser— vices for English learners, but none of these attempts was ultimately successful. In 1998, California became a battleground for a national movement to abolish all native language instruction by mandating English— only instruction. in California, this movement took the form of a voter initiative——Proposition 227—that severely restricted the use of primary language for instructional purposes, and instead provided for a transitional program of “struc— tured English immersion” that was not normal— ly to last more than one year.” The initiative was approved by the voters in June 1998 and schools were required to implement it in the opening days of the 1998—99 school year. For many districts, this meant that only about sixty days were available to prepare for this policy 2000 implementation. The state board of education rushed to provide guidelines for schools,21 although most decisions about how to imple— ment the mandate were left to the local educa— tion agencies. At the same time, districts were dealing with a plethora of other state initiatives that were having an impact on the schooling of English learners, which we discuss below. What has been the impact of Proposition 227? Proponents of 227 have argued that the shift toward more English instruction is already improving the test scores of English learners.” Yet at this early stage there is little research evi— dence to scientifically assess the impact of 227 on student achievement?‘However, preliminary research does indicate that Proposition 227 has had a considerable impact on the instruction of English learners in California. A team of University of California researchers2+ looked at the effects of Proposition 227 in sixteen districts and twenty- five schools during the initial months of imple- mentation.“ Urban, rural, and suburban K-8 and unified districts were included in the study, as were very large and very small districts. Most of the ten largest districts in the state were also included. Some of the districts had a history of strong support for primary language instruction and had extensive primary language programs before 227; others had relied heavily on English—only programs. Beginning in the fall of 1998, the teams interviewed administrators charged with the policy implementation at each district, and then followed up with interviews of principals, teachers, and bilingual coordina— tors in key schools within these districts. Researchers selected schools that had relatively large populations of English learners and would therefore be most affected by the policy. The Schooling of English Learners Classroom observations were also conducted in most of these schools. This study has yielded several important insights into the implementa- tion and impact of Proposition 227. Diversity of District Responses Across the sixteen districts and twenty-five schools, there was wide diversity of responses to the mandate, although this diversity was not without a pattern. Districts with a history of extensive primary language programs and sig— nificant numbers of certified bilingual staff were more likely to consult with their commu— nities and to attempt the continuation these programs than were districts and schools with weaker primary language programs and inade- quate numbers of certified bilingual staff. Researchers also found that where strong lead— ership was exercised at the top of the district, either in providing parents with information about alternative options to structured English immersion classes, or in urging principals to discontinue primary language instruction, schools followed suit. However, where district leadership was less prescriptive, the decisions fell to principals, creating a diversity of responses wit/yin the district?6 In both situa- tions, some teachers exercised considerable autonomy in interpreting district and school directives, resulting in a diversity of instruc- tional strategies within the same school. Variation in Procedures Regarding Provision of leiver Options In the initial months of implementation, there was considerable confusion across the state about the role of the district and the schools in informing parents of their rights to seek waivers from the structured English immersion PACE I 29 program provided under the provisions of Proposition 227. Although the state board of education had issued an advisory stating that parents were to be notified of the right to seek a waiver from SEI instruction, a fall 1998 sur- vey conducted by the California Department of Education showed that only ()7 percent of dis— tricts had formally notified parents of this option? Some districts interpreted the initia— tive as barring any proactive dissemination of waiver information while others considered it their duty under the law to provide parents with information about their program options. Thus, some of the districts moved quickly to provide waiver information to the schools and parents, while others provided such informa— tion only as requested from parents, or only after a lengthy period of debate and reflection. Schools and districts that facilitated access to information about the waiver option were more likely to continue to provide primary language instruction for significant numbers of students. Impact on Classroom Instruction \Vhat teachers chose to do in their own class— rooms in the post—227 period depended to a great extent on what they had done prior to 227, and on their own skills, experience, and beliefs about students’ learning. However, it was rare to encounter a teacher who contended that his or her instruction and class organiza— tion had not been affected. Not surprisingly, teachers who were certified and experienced in bilingual instruction, although no longer assigned to bilingual classrooms, were more likely to continue to provide some level of pri— mary language support for their students. However, this varied greatly depending on the Climate in their schools. These teachers were CRL‘CIAI. lSSL'I-ZS IN CALIFORNIA EDL‘CA'rlox careful to keep primary language support with- in the strict confines of providing instruction “overwhelmingly in English,” as defined by their district. Although many teachers who taught in waivered classrooms, using bilingual methods, contended that their teaching had not changed significantly, they were quick to note that they worried about the future and the pos- sibility that they would he required to change their practice over time. There was a real sense among many teachers that official policy was continuing to evolve. Many experienced bilin— gual teachers who were no longer in bilingual classrooms reported feeling frustrated by not being able to use the full range of skills they possessed to instruct their English learners. In the schools that were studied intensively, a much more reductionist notion of literacy was observed, compared to what went want on in these same classrooms prior to the implementa— tion of Proposition 227,-“ Language and litera- cy were rarely used as tools for learning other subjects; instead English itself—in terms of developing oral fluency and reading decoding skills—was becoming the focal point of instruc— tion. Teachers attributed this focus to their concerns about the English language testing to which students would be subjected and by which students” educational progress would be gauged by policymakers and the public. Issues in Implementation The implementation of 227 created a number of problems for schools and teachers: 0 Lost instructional time. The thirty day English—only period at the beginning of the year mandated by Proposition 337 resulted in a loss of instructional time for almost all English learners because of the temporary ZOOO and transient nature of the classes and unavailability of materials to meet all chil- dren’s needs. Inadequate materials and training for implementing the structured English immersion program. Several schools reported that while Spanish language texts were discarded or stored away, no compara— ble texts were available for students in the new English—only program and that teachers were uncertain about how to approach the instruction of their students. 0 Inadequate professional development in the teaching of English reading to English learners. Professional development time available within the regular school cal— endar has been reduced as a result of new legislation, and none of the districts or schools studied during the first half of the 1998—99 school year had provided training for teachers that focused specifically on the The Schooling of English Learners teaching of reading to English learners under the new Proposition 227 conditions. ' Teachers’ fear of legal reprisals. A hyper- interpretation of the new law was noted in some schools where teachers created instruc- tional practices and restrictions that were not mandated by the law in an attempt to protect themselves from the possibility of reprimand or prosecution. For example, a teacher might focus on specific features of English, such as word recognition, while ignoring other aspects of general literacy development, such as story-telling. Overall Effects of Proposition 227 All but four of the sixteen districts studied reduced the percentages of students receiving primary language instruction (reductions ranged from 12 to 100 percentage points); three districts maintained a similar percentage; and one contended that it increased the percent I ELD/SDAIE 33% I ELD/SDAIE + primary language support 22% as: Bilingual 29% Other/No Services 16% I ELD/SDAIE 39% I ELD/SDAIE + primary language support 33% fl Bilingual 12% Other/No Services 16% Figure 3. Types of Instructional Services for English Learners Before and After Proposition 227 Note: ELD/SDAIE is English Language Development (ELD) or Specially Designed Academic Instruction in English (SDAIE). Source: California State Department of Education, Language Census Summary Statistic, 1997-98 and (Sacramento, California: author). Retrieved November 8, 1999 from the World Wide Web: http://www.cde.ca.gov/demographics/reports/statewide/ |csum98.htm;Calif0rnia State Department of Education, Language Census Summary Statistic, 1998—99 and (Sacramento, California: author). Retrieved November 8, 1999 from the World Wide Web: http://www.cde.ca.gov/demographics/reports/ statewide/Icsum99htm. PACE I 31 of students who were assigned to primary lan— guage programs. Across the state, 29 percent of English learners were in a primary language program prior to 227, and only 12 percent were assigned to one after the implementation of 227 (See Figure 3). While there was a tendency for schools and districts with extensive primary language pro- grams to continue to provide these programs at some level, some schools with well—developed primary language programs completely aban— doned them in the wake of Proposition 227. Moreover, considerable change was found in the actual classroom practices of teachers, with much more emphasis on the use of English, even in schools that purported not to have changed or reduced their primary language instructional programs. Concerns about the requirement that students be tested in English drove these new practices as much as teachers’ concerns about avoiding reprimand or worse. The extent to which schools and districts were changing their perceptions about the need to recruit bilingual teachers was investigated in seven of the sixteen districts. Five of the seven districts continued to seek bilingual teachers; two decided to curtail these hires. This was in spite of the fact that both the department of education and the California Commission on Teacher Credentialing have continued to underscore the importance of BCLAD teachers for a number of instructional purposes.” Thus, while it was tempting to conclude on the basis of principal interviews that not a great deal had changed in some of the schools, a look inside the classrooms yielded a different per— spective. Even in bilingual classrooms, teachers were changing their practice to accommodate both practical concerns—such as the impact required English testing would have on their students— as well as concerns for their own professional well-being. Moreover, there was a pervasive sense that policies were still unfolding in many districts; consequently, teachers were unsure of what the future held. Teacher Recruitment and the Adequacy of the Teacher Pool for English Learners Perhaps the greatest challenge for the educa- tion of English learners is the recruitment and preparation of sufficient numbers of teachers who are qualified and skilled in meeting their specific learning needs. Two primary creden- tials are offered in California today that are supposed to address the needs of English learn- ers. One is the Cross-cultural, Language and Academic Development (CLAD) credential, which can be earned by examination or through coursework on cultural and linguistic diversity, which includes techniques for Specially Designed Academic Instruction in English (SDAIE) and English Language Development instruction. The other is the Bilingual Cross— cultural, Language and Academic Development (BCLAD) credential, which requires, in addi- tion to the basic CLAD requirements, profi— ciency in a second language.m If obtained through coursework requirements, the BCLAD usually includes more extensive preparation related to second language acquisition. As pointed out in Chapter 6, currently 28,5 00—or a little more than one in ten—of the state’s approximately 280,000 teachers are CRUCIAL ISSUES IN CALIFORNIA EDUCATION 2000 uncredentialed. This situation is not likely to improve any time soon. Furthermore, uncre— dentialed teachers are not evenly distributed across the state, nor are they evenly distributed among schools and classrooms containing dif— ferent types of students. English language learners, for instance, are extremely likely to have a less than fully qualified teacher. In 1998, prior to the passage of Proposition 227, California had a shortfall of 11,000 certified bilingual teachers and 34,000 teachers certified to provide appropriate English language train- ing (see Figure 4, left panel). This meant that only about one—third of all English learners had a fully certified teacher.“ Today, because of the growth of English immersion programs and the reduction of bilingual education programs brought about by The Schooling of English Learners Proposition 227, the number of teachers who hold credentials to work with English learners has grown dramatically. For example, the num- ber of teachers with English Language Development (ELD) and SDAIE training has increased to 50,122 (see Figure 4, right panel). Another 11,995 teachers have been “grandfa— thered” into the category of CLAD teachers through provisions of SB1969, which allows experienced teachers to receive certification through staff development training or college course work. Added to this number are 10,690 teachers with BCLAD and other bilingual cre- dentials. On paper, it appears that among those teachers in California who instruct English learners, a significant number (52 percent) have received some kind of preparation in instruct— ing English learners. Unfortunately, this prepa— 1997-98 I In-training ELD/SDAIE 34,439 I CTC Bilingual 15,783 I ln—training Bilingual 10,894 $3 crc ELD/SDAIE 35,166 Other ELD/SDAIE 7,488 I ln-training ELD/SDAIE 38,527 I CTC Bilingual 10,690 I ln-training Bilingual 5,670 CTC ELD/SDAIE 50,122 Other ELD/SDAIE 1,995 Figure 4. Number of Teachers Providing Instructional Services for English Learners by Certification, Before and After Proposition 227 Note: CTC teachers are teachers who hold valid certificates for the designated type of instructional service from California Commission for Teacher Credentialing (CTC). ELD/SDAIE teachers hold certificates to teacher English Language Development (ELD) or Specially Designed Academic Instruction in English (SDAIE). Other teachers hold an 831969 certificate or CDE approved district certificate. Source: California State Department of Education, Language Census Summary Statistic, 1997—98 and (Sacramento, California: author). Retrieved November 8, 1999 from the World Wide Web: http://www.cde.ca.gov/demographics/reports/statewide/ lcsum98.htm;CaIifornia State Department of Education, Language Census Summary Statistic, 1998-99 and (Sacramento, California: author). Retrieved November 8, 1999 from the World Wide Web: http://www.cde.ca.gov/demographlcs/reports/statewide/ lcsum99.htm. PACE I 33 34 ration is often cursory and only sufficient to make a teacher aware of what he or she does not know. Under SB1969, CLAD certification can often be acquired with only forty—five hours of relevant training.“ Moreover, students often are not assigned in their student teaching to the teachers who have been prepared to teach them. This is due, in large part, to the unequal distribution of qualified teachers across schools and districts. Given the teacher shortage in the state, the best prepared teachers can choose to take positions in the suburbs and in districts with less challenging populations, forcing the less well—prepared teachers into the inner cities and the schools with high proportions of poor students and English learners. A recent review of middle school and sec— ondary programs for English learners points out the acute problem of under—preparation of CLAD—credentialed teachers to meet the needs of English learners, especially in math and sci- ence.“ Because these classes are “gatekeepers” for college preparatory coursework, students who do not do well in them are typically assigned to a general course of study that does not provide them with the option of entering a four—year college. This can have life—altering consequences for these students. A similar problem exists in that CLAD—credentialed English teachers often do not utilize their class— es to prepare English learners for the academic requirements of subject areas. The present crisis in providing English learners with fully qualified teachers has been exacerbated by recent reforms, particularly class—size reduction. A recent early evaluation of class—size reduction in California found that it increased the disparities in the numbers of qualified teachers between schools with large CRUCIAL lsSL'Es IN CALIFORNIA EDUCATION concentrations of English learners and schools with small concentrations of English learners.” For example, the percentage of teachers not fully credentialed in schools with the least number of English learners (less than 8 per— cent) only increased from .2 percent in 1995-96 to 4.2 percent in 1997-98 (see Figure 5). However, the percentage in schools with the greatest proportion ()f English learners (40 per- cent or more) increased from 1.8 percent to 22.3 percent over the same two—year period. As a result, schools with the most English learners benefited the least from class-size reduction, at 25 'U 2 /' N g 20 "U 9 O 2 -:—, 15 LL *5 2 g 10 5 8 l— “5 g 5 .59 c 95 Q) . ,. Q 0 1995-96 1997—98 Quartile 1: Schools with 7.49% or fewer EL students I Quartile 2: Schools with 7.5% to 19.99% EL students I Quartile 3: Schools with 20% to 39.9% EL students - Quartile 4: Schools with 40% or more EL students Figure 5. Percentage of Publick K-3 Teachers Not Fully Credentialed by School Quartiles of English Learners: 1995-96 and 1997-98 Source: George W. Bohrnstedt and Brian M. Stecher (Eds). Class Size Reduction in California: Ear/y Evaluation Findings, 1996-1998 (Palo Alto: American Institutes for Research, 1999), Table D31. 2000 least in terms of access to fully credentialed teachers.“ Although Proposition 227 had no statutory effect on the credentialing of bilingual teachers (BCLAD), the issue has been raised whether certified bilingual teachers are still needed or desirable in a state that has placed rigid restric— tions on the provision of bilingual education. In a somewhat ironic twist, it may be that bilin— gual teachers are more necessary now than under the conditions that existed prior to the passage of the initiative. Structured English immersion, the instructional approach recom- mended by 227 language, is an approach that actually relies on a teacher’s knowledge of the students’ primary language. While the children are usually not instructed in their primary lan- guage, past studies have nevertheless demon— strated that this approach is most effective when it incorporates a significant amount of primary language support to ease the students into the English—only curriculum. Nioreover, bilingual teachers credentialed in California possess a body of knowledge about second lan— guage acquisition and the pedagogical tools that can enhance it that most non—bilingual teachers do not have.“ At a time when there is so little direction being provided to teachers to help them with English learners and their pri— mary language resources, this skill base of bilin— gual teachers may be especially important. There is also a growing awareness of the con— nection between parent involvement in school— ing and children’s academic achievement, pointing up the critical importance of home— school c<)mmunication.v \Vithout a teacher who speaks the language of the home, direct communication is not likely to occur. Thus, while the instructional methods of teachers The Schooling of English Learners may change under the new law, the importance of understanding children’s educational needs in conjunction with their linguistic develop— ment and communicating with their families does not. However, the perception that bilin- gual teachers may no longer be needed in California is likely to negatively affect both the supply and demand of such teachers for the state’s English learners. The Educational Achievement of English Learners Learning English is only one of the challenges facing English learners. The other is for them to succeed in all the other academic arenas of school. How are English learners in California doing in school? For many people inside and outside of the educational system, both in California and in the nation, achievement in school is best~ represented by one thing—scores on standardized tests. But as pointed out in the chapter on student performance, scores on standardized tests are not necessarily the best way nor should they be the only way to gauge the educational achievement of students. This is especially true in the case of English learners because most existing national and state assessments are conducted in English. Because English learners are not yet proficient in English, such assessments may not accurately reveal the subject matter knowledge of English learners. A recent report by the National Research Council on the use of testing for track— ing, promotion, and graduation posed the issue this way: “The central dilemma regarding partic— ipation of English—language learners in large— scale assessment programs is that, when students PACE I 35 36 are not proficient in the language of assessment (English), their scores on a test given in English will not accurately reflect their knowledge of the subject being assessed (except for a test that measures only English proficiency)”38 California has responded to this dilemma by exempting English learners from taking the two English—based state proficiency tests that are part of California’s Standardized Testing and Reporting (STAR) program—the Stanford 9 achievement test and the STAR augmentation tests—if they have been enrolled in a California school for less than twelve months.” Instead, Spanish-speaking students enrolled less than twelve months are required to take the Spanish Assessment of Basic Education (SABE). 4" Based on existing research evidence, this policy means that English-based test scores for English learn— ers in California will tend to understate their knowledge of subjects other than English.“ Another problem is that scores on standard— ized tests are typically reported as national per— centile ranks, which only rates the performance of students to a relative standard—the perfor— mance of other students—rather than to a fixed standard that actually reveals what students know. This method of ranking ensures that some portion of students, regardless of what they know, will be ranked low. Nonetheless, this is the most common way of reporting test score results, which almost inevitably places English learners toward the bottom. For example, in the 1999 statewide test pro— gram, English learners scored considerably lower than English—only students in the Stanford 9 reading and math tests. Among fourth grade students, for example, only 11 percent of English learners scored at the national average (50th percentile) in reading and 21 percent at the national average in math, compared to 5 3 in reading and 51 percent, respectively, for English-only students (Figure 6, top panel). Similar disparities existed for stu- dents in grades seven and eleven. In 1999, California students were also test- ed, albeit in a very limited way, on their knowl- edge of the newly adopted California academic content standards. Unlike national exams like the Stanford 9, these tests were specifically designed to test students’ knowledge in the subject areas that the state feels are most important for students to know. In addition, the only comparisons possible are between differ- ent groups of California test takers, not a national population. Results on the math por- tion of the 1999 tests are shown in the bottom panel of Figure 6. This time, the comparisons suggest that English language learners are much closer in terms of performance to English proficient students, although they still score below them at all grade levels.*3 But even these comparisons do not provide a complete picture of the achievement of lan— guage minority students. These comparisons only focus on the achievement of language minority students who are not yet proficient in English (English learners) while ignoring the achievement of language minority students who are proficient in English. In other words, the comparisons do not account for those who enter the schools as fluent English speakers or for those who become fluent while in schools and hence are redesignated as fluent English proficient (FEP). To fully judge the perfor— mance of language minority students and the programs that are designed to teach them, it is important to examine both the performance of students who are still learning English and the CRUCIAL ISSUES IN CALIFORNIA EDUCATION 2000 The Schooling of English Learners 7O Reading Math 60 50 4O 30 20 1O % Scoring At or Above National Percentile Rank Grade 4 Grade 7 Grade 11 Grade 4 Grade 7 Grade 11 - English Only I Redesignated Fluent English Proficient I Fluent English Proficient English Learners 60 50 40 30 0/0 Correct 20 . f j.- . 10 ff __ Grade 4 Grade 7 Grade 11 I English Only/Proficient I English Learners Figure 6. 1999 State SAT-9 Reading and Math and STAR Augmentation (Math) Test Scores by Language Background Source: California State Department of Education, California Standardized Testing and Reporting (STAR),1999: State Reports (Sacramento, California: author). Retrieved November 30, 1999 from the World Wide Web: http://starcde.ca.gov/star99/reports/ Englishhtml and http://207.87.22.181/STAR/. PACE 37 performance of students who have become fully proficient in the language. California has just released Stanford 9 test scores of fluent English proficient students sep- arately from English—only (native English speakers) students. These results show that, in general, both fluent English proficient and redesignated English proficient students score as high or higher on standardized achievement tests in math as native English speakers (Figure 6, top panel). In reading, fluent English speak— ers score similarly to native English speakers in the lower grades, but score lower in the upper grades, which again supports the earlier claim that English learners have a considerably hard— er time acquiring more advanced, academic English in the upper grades.“ These results suggest that if all English learners could become proficient in English, then their achievement would be at least com- parable to that of other students. This conclu- sion may be simplistic, however, because it is based on two related beliefs. One is that the reason English learners have lower levels of educational achievement is primarily because of their lack of English skills. The other is that non-English speakers will demonstrate much higher achievement once they learn English. But existing research questions both beliefs. A recent study provides a good illustration. The study examined the influence of language background and other factors on the 1998 Stanford 9 test performance for 26,126 second, third, and fourth grade students in eight 60 50 40 30 20 Mean NCE Scores 10 Poor Not Poor I English I Spanish Other White Hispanic CRL'CIAL Issr'Es IN CALIFORNIA EDL'cA’riox Figure 7. 1998 SAT-9 Reading Scores by Language Background and Poverty, Hispanics and Whites: Eight Southern California School Districts Source: Douglas E. Mitchell and Ross Mitchell, The Impact of California's C/ass Size Reduction Initiative on Student Achievement: Detailed Findings from Eight Schoo/ Districts (Riverside: California Educational Research Cooperative. 1999). Available on the World Wide Web: http://cerc.ucr.edu/pub|ications. ZOOO Southern California school districts.“ First, the study examined the independent effects of two factors language background and poverty—on student achievement (see Figure 7, left panel). These results show that poverty affects the achievement of all students regardless of their language background. Because the majority of English learners are poor, it means that they are at a particular disadvantage in school. Next, the study examined the impact of language background and ethnicity on student achieve— ment (see Figure 7, right panel). These results show that even Hispanic students from English—speaking backgrounds had significantly lower test scores than Whites from English— speaking backgrounds. This suggests that something other than English proficiency must be accounting for those differences.xlv It fur- ther suggests that while improving the English proficiency of English learners will improve their academic achievement, English proficien— cy alone is unlikely to raise their achievement to the levels of White, native—English speakers. Prospects for the Future California has passed a number of major reforms in recent years that are only now beginning to be implemented. Many of these reforms are likely to have an impact on the future schooling of English learners. Furthermore, these reforms raise many issues that will need to be resolved. For example, the full impact of Proposition 227 on California’s English learners is still not known. However, early indications are that most children will not transition successfully from structured English immersion to main— The Schooling of English Learners stream English classes within one year. With the repeal of the reclassification guidelines, what constitutes readiness for transition to English-only is an uncertain and highly contro— versial issue. The state department of education is currently drafting guidelines under the authority of Title 5 of the California Education Code to help districts and schools make these decisions. New legislation on pupil promotion and retention (AB1626, Chapter 742, 1998) requires that students who are at risk of being retained because of failure to meet grade-level standards be provided additional educational services, including supplemental instruction and manda— tory summer school. It is also noted in the legis— lation that students who are not proficient in English should not be retained solely on the basis of language handicap. It appears, however, that since English learners are likely to be at high risk of failing to meet educational stan— dards, additional services will need to be provid— ed for them. The costs and logistics involved in providing these services for large numbers of English learners have not been fully considered either by the state or by school districts. Yet research has found that retention alone is an ineffective and costly means by which to 4" so provision of improve student performance, supplementary services will ultimately be a less expensive response to the dilemma. High school exit examinations represent another area of educational reform with poten- tially large consequences for English learners. The numbers and proportions of EL students are greatest in the early grades, and for these younger students there may be sufficient time to intervene so that the possibility of failing to attain a high school diploma in spite of having PACE I 39 40 completed all other high school requirements can be averted. However, approximately one— third of EL students are found in the secondary schools (see Table 1), where there is little time to gain the full English fluency and sufficient command of the secondary curriculum that are needed to pass the high school exit exam in English. This examination is only now being developed for students who plan to graduate in 2004, but the repercussions could be severe for English learners if an appropriate solution is not found to accommodating the testmg needs of these students.“ Legislative efforts to extend the school year by reducing teachers’ out—of—class time for pro— fessional development has “face validity,” but may in fact be shortsighted during this time when teachers, many of them under-qualified, are grappling with so many increased perfor— mance demands. In a recent survey conducted by the department of education on the impact of Proposition 227, schools cited professional development to help teachers teach English learners as one of the most highly unmet needs.“ The University of California study cited earlier found no instance in which teach— ers had been provided with professional devel— opment geared specifically to the instruction of reading for English learners. Assessment of English learners will also remain a difficult and controversial issue for some time to come. Currently, all English learners who have been in school for at least twelve months must be tested in English on the STAR test annually. Alany districts and parents have expressed strong concerns about the ways in which this testing may affect the students and their records. However, we have seen that the most immediate impact of this testing CRUCIAL lSSL‘ES IN CALIFORNIA EDUCATION appears to be on instruction. Teachers, whether in bilingual or SE1 classrooms, expressed con- cern about their students being tested prema- turely in English and therefore were anxious to focus on oral fluency in English rather than broader literacy skills. As one researcher put it, “language and literacy are rarely tools for learning but rather English language learning (oral fluency) is becoming the target of instruc- tion.” The impact of this shift in instructional emphasis and student outcomes should be monitored to assess its short-term and long- term effects on the development of literacy skills for English learners. A related issue is the assessment of English Language Development (ELD) for English learners. While ELD standards have recently been adopted by the state board of education, and legislation passed in 1997 (AB748) required that a test be developed that allowed for the assessment of ELD standards, the development of an appropriate and relevant test has just begun. Many scholars believe that it is critical to monitor this early acquisition of English skills in order to prevent failure later when children are expected to meet main— stream English curricular demands; however, there is by no means widespread agreement on this issue.‘0 Finally, we find no issue more compelling or more urgent than the need to recruit, retain, and strengthen the skills of teachers who serve English learners. Particular atten— tion also needs to be paid to the competencies of middle and high school teachers who are often overlooked in the discussions on teacher preparation. However, given the current teacher shortages, the increasing numbers of English learners, and the numerous reform ini— lOOO The Schooling of English Learners tiatives with which schools and districts are challenge. K-12 schools alone cannot meet dealing, it is not clear where the will or the these enormous challenges." resources will come from to seriously address In summary, California faces a number of this problem. Certainly, it appears that both challenges in trying to improve the schooling the state, through various incentive funding conditions and learning outcomes of English schemes like sign—up bonuses and scholarships, learners. As their numbers increase, the future and its postsecondary institutions, through success of all California’s current reform efforts expanded teacher education and professional will be impacted by the state’s ability to suc- development programs, will need to rise to the cessfully meet these challenges. Notes 1. We would like to thank Richard Duran, Eugene Garcia, and Barbara Merino for comments on an earlier version of this paper. 2. Prior to 1998, the State of California identified such students as Limited English Proficient (LEP). But a recent report on the schooling of English learners by the National Research Council suggested using new terminology— English—language learners. The State of California adopted the term English learner (EL) student in 1998. See Diane August and Kenji Halmta, Improving Schooling for Language-Minority Children (\Vashington, DC: National Academy Press, 1997), 1. 3. See August and Hakuta, Improving Schooling; Patricia Gandara, Review of Research on the Instruction of Limited English Proficient Students (Santa Barbara: University of California Linguistic iVIinority Research Institute, 1997); Catherine Snow, M. Susan Burns, Peg Griffin (Eds), Preventing Reading Difficulties in Young Children (Washington, DC: National Academy Press, 1998). 4. US. Bureau of the Census, Nativity and Parentage of the Population for Regions, Divisions, and States: 1997 (\Vashington, DC: US. Bureau of the Census, 1999). Retrieved from the World Wide Web: http://www.census.gov/pop— ulation/wwwv/socdemo/foreign/foreign98.html. 5. Most immigrants want to learn English and do become proficient in English over time. But their opportunities for learning and using English vary widely due to such factors as social class and place of residence. See Ruben G. Rumbaut, “The new Californians: Comparative research findings on the educational progress of immigrant children.” in Ruben G. Rumbaut and Wayne A. Cornelius, eds, California’s Immigrant Children: Theory, Research, and Implications for Educational Policy (San Diego, CA: Center for U.S.—i\'1exican Studies, University of California, 1995), 17—69. 6. These questions include the child’s native language and the language used by the parents and child at home. 7. The most common tests are the Language Assessment Scales (LAS), the Idea Proficiency Test (IPT), and the Bilingual Syntax Measure (BSM). 8. The levels of proficiency that must be demonstrated vary by age or grade level—the older the student, the more demanding the proficiency tasks that must be demonstrated to be classified as fluent English proficient. Proficiency levels are not directed related to the proficiency of native—English speakers, but rather to levels of proficiency that would be comparable to an average native—English speaker of a similar grade. 9. See California Department of Education, Language Census, 1999: Summary Report. Retrieved October 15, 1999 from the \Vorld \Vide Web: http://wwwcdeca.gov/demographics/reports/statewide/lcsum99.htm. PACE I 41 44 42. This further buttresses the argument that assessments of subject matter knowledge are highly language depen- dent. 43. These findings are consistent with other studies that find fluent English speakers often do as well or better than native English speakers in school. See, for example, Russell W Rumberger and Katherine A. Larson, “Toward explaining differences in educational achievement among Mexican~American language minority students.” Sociology of Education 71 (1998): 69-93. 44. Douglas E. Mitchell and Ross Mitchell, The Impact of California’s Class-size reduction Initiative on Student Achievement: Detailed Findings from Eight School Districts (Riverside: California Educational Research Cooperative, 1999). Available on the World Wide Web: http://cerc.ucr.edu/publications. 45. For a discussion of other factors that affect the achievement of English learners, see: August and Hakuta, Improving Schooling; Rumberger and Larson, “Toward explaining..”; National Research Council, Panel on High—Risk Youth, Losing generations: Adolescents in high-risk settings (Washington, D.C.: National Academy Press, 1993). 46. Heubert and Hauser, 1999. 47. The National Research Council committee on high stakes testing cautions against an over-reliance on test scores for graduation especially in the case where students may not have been given an adequate opportunity to learn the requi— site material. See Heubert and Hauser, High Stakes. 48. California Department of Education, Proposition 227 Survey, Interim Report, May 12, 1999. 49. Gandara and Maxwell—Jolly, forthcoming. 50. For a discussion of this issue by the profession organization, ESL Standards for Pre—K—lZ Students (Washington, D.C., 1999). 51. The implementation of recent State legislation regarding improvements in the assessment of teachers and in the certification of teacher education programs will contribute to these challenges. CRUCIAL ISSUES IN CALIFORNIA EDUCATION 2000 Chapter 4 School Finance in California: Does History Provide a Sufficient Policy Standard? Neal Finkelstein, William Furry and Luis Huerta Introduction Chapter One describes education policies in California that are in some ways beginning to converge. Standards anchor the system, which is buttressed in a number of ways by explicit accountability mechanisms for districts, schools, teachers, and the students themselves. As shown in Chapter 7, measurement and assessment systems, still among the most diffi- cult educational policy levers to manipulate, have gained some stability in recent years as yet rough but increasingly consistent tools that provide information about student performance and the system’s progress. Yet several glaring discontinuities in California education policy remain, especially in the area of financing. Policies concerning the financing of education must be far better arranged to support the educational plans that have evolved over the past fifteen years. We argue two central points in this chapter: ° The idiosyncratic history of school finance in California has over time left the discussion of “what does the money buy” disconnected from discussions of educational goals and practice. As obvious as it may seem, the issues of how money is raised, how money is spent, and who decides how to spend it are essen— tially connected to the quality of education. ' It is unlikely that even substantial adjust— ments to the existing school finance system can result in financing structures whereby the educational goals of the state are consis- tent with how resources are raised, allocated, and spent. Incremental reforms in the ways in which schools are financed will not yield a coherent set of policies. In the chapter that follows, the context of K—12 education finance is discussed with a sharp focus on the current lack of alignment between state standards and other educational initiatives, and the resources needed for their effective imple— mentation. For example, class size reduction, as has been evident over the past two years, has profound financing implications. The goal of hiring and retaining qualified teachers, as dis— cussed in Chapter 6, has substantial financing implications. Specific types of remedial educa— tional programming in reading and mathematics, too, have substantial cost implications. How can the education finance system in California better accommodate these needs, and at the same time be governed deliberately and rationally? For some time, PACE has provided readers of Conditions with some basic descriptive charac— PACE I 45 46 teristics about the financing of schools. Here again, a brief introductory section provides rele- vant background and an update on the financing of schools. Next, a policy discussion is provided that links school finance to the current momen— tum of educational change in California. This section asks whether we are financing a stan— dards—based reform effort, or some other histor— ical conglomeration of educational principles. To think through the disconnection between finance and stamiards—based reform, we then explore the real constraints that are part of the current financing system: equity considerations, statutory considerations, governance, and the precedents that have been imposed by the courts. The middle of this chapter takes a look at the development of categorical funding over a thirty year history; this section tells an impor— tant story about how the education finance sys— tem has responded to educational needs over time, for better or worse. Finally, the chapter ends with a set of cautions related to new direc— tions in school finance policy. The Context of School Finance in California As many policy analysts have commented over the past 20 years, California is a state where the convergence of court rulings, voter—initiated propositions, legislative activities, and econom— ic cycles has created a complex system unlike any other in the United States. Alany good descriptions of the histmy of school finance exist, and so in this chapter we will not pay extensive attention to describing the events that transpired over the past 30 years. PACE, in its CRUCIAL Iss‘L'ias IN CALIFORNIA EDL'CA’I‘ION 1994—95 edition of Calida/011$, provides a com- prehensive history; EdSource, also, has made available an excellent synopsis of the policy his- tory of school finance in California. Finally, Manwaring and Sheffrin, and Downes and Shah, among others, have provided extensive academic analyses and interpretations of the events that have led to the current day’s discus- sion.1 For our purposes in this chapter, we reit— erate a few central policy constraints and data about the current finance S'VSCCIH. A Limited Selection of California’s School Finance Policy Constraints The Serrano Case. The policy constraints around school finance began with the 1971 California Supreme Court opinion in Serra/Io 2'. Priest (Serrano I). By 1998, legal cases had been brought in forty-three other states; in nineteen of them, state supreme courts found the school financing system to be unconstitutionalf In the Serrano case, the plaintiffs argued that educa— tion spending linked to property wealth gener— ated unacceptable expenditure differentials between school districts. The California Supreme Court found that the state’s school finance system, if the facts were as alleged, failed to meet the equal protection clauses of the California and United States Constitutions. The court opinion was based in part on the concept of fiscal neutrality, arguing that a school finance system based on property wealth was unconstitutional if an equal tax rate did not produce the same revenues per-pupil across all districts.‘ The case was remanded back to the state superior court level for trial, where it had been initially dismissed. This second segment of the litigation occurred between 1973 and 2000 School Finance in California: Does History Provide a Sufficient Policy Standard? 1974 and has been generally referred to as Serrano II. The superior court judge in Serrano II spec— ified that wealth—related differences in school funding must fall within a “band” of equality above and below the state average per-pupil funding, and that the amount should be con- siderably less than $100 per-pupil. Since 1974, the date of the Serrano II decision, the band with inflation adjustments has grown to approximately $335 on either side of the school district average per ADA of $3,785 (1997—98 figures reported by LAO, 1999).4 This figure includes only state and local gener— al purpose revenues; it does not include cate— gorical funding or support from other miscel— laneous sources. The Legislative Analyst reported that in 1997-98, approximately 98 percent of the state’s school districts met the constraint set by the Sewano band.S Proposition 13. Proposition 13, passed by the voters in June 1978, called for a new tax rate for residential, commercial and personal property based on acquisition value rather than market value. Initially, tax assessments were rolled back to 1975 property values, which marked the “base year” for all existing proper— ties in the state. Recalculated property taxes were limited to one percent of the 1975 assessed market value, with increases limited to the lesser of two percent of assessed value per year or the current rate of inflation. Property can be assessed at market value only when sold, or when new construction has been completed. The effect of Proposition 13 on education finance has been two—fold. First, by setting a maximum property tax rate of 1 percent, it eliminated the apparent inequities in local tax rates that had been the focus of the Sew/mo decision. At the same time, however, it broke the connection between local property taxation and support for schools by reducing the capaci- ty of the property tax base to support schools in most districts. Since Proposition 13, the state has been required to substantially fund schools through the General Fund—just one-third of revenues for schools are raised locally. In effect, then, California’s school finance system is cen— trally controlled. Requests for marginal increas- es in spending are made through the state leg- islative process and are not the decisions of local property taxpayers.” This change in the locus, of decision—making has had substantial implications for increased state attention to education issues. Revenue Limits. Following the Serrano decision, the legislature enacted a long-term equalization financing plan for schools in 1972 by inventing the “revenue limit.” This was designed to provide foundation funding for schools. The first year of its implementation was in 1973—74. Each California district has a base revenue limit per—pupil, measured by ADA, from general state funds and local prop— erty taxes. Over time, inflation adjustments for revenue limits have been a significant policy issue as they have guided the rate at which the state has equalized spending per-pupil between districts. As a result, the history of revenue limit adjustments is complex and important as districts have sought the resolution of perceived inequities stemming from the administration of the policy." Revenue limit worksheets, encom— passing adjustments for cost of living, unem— ployment insurance, meals for needy pupils, summer school, longer school day and year, and other details, are difficult to follow and understand. PACE I 47 48 Equalization between districts has been rela— tively successful over the past twenty-five years and has relied in part on the adjustment to the revenue limit. Goldfinger describes several of the discontinuities in the equalization process that have been the result of legislated adjust— ments to the revenue limits. In each case, the 861771710 band has acted as a default standard, placing enormous importance on the average per—pupil spending level in the state. As a conse- quence, substantial import is given to the rela— tive funding level of districts between one anoth— er, and less to the absolute level of funding. It is also important to remember that the calculation of the revenue limit is independent of federal and state categorical support which can account for more than 40 percent of fund- ing for some districts. This point is further dis- cussed below. Proposition 98. In November 1988, the voters approved Proposition 98, which provides a funding guarantee for primary, secondary and community college education. Amendments to the proposition, including Proposition 111 (in 1990), have resulted in a funding floor for edu- cation with specific rules about how the funding guarantee should be upheld in high and low state revenue growth circumstances. Proposition 98 has resulted in some stability for education funding in California, although the funding floor was consistently interpreted as a funding ceiling in the economic downturn of the late 19805 and early 19905. What seems clear about Proposition 98 is that it sets aside a fixed per— centage (about 40 percent) of the state general fund for K-14 education, therefore pitting the interests of the state’s schools and community colleges against those of higher education, CRUCIAL ISSUES IN CALIFORNIA EDUCATION health care, welfare, corrections, transportation, and other obligations of the general fund. School Facilities Funding. California’s stu- dent population continually exceeded the national average in annual growth throughout the last decade, and is expected to increase by an additional 15 percent (nearly 1 million additional students) over the next 10 years.8 Combined with statewide efforts to reduce class sizes in ele- mentary grades, local school districts have become burdened with demands for new facili- ties construction, maintenance and renovation. In California funding public school facilities is the responsibility of local school districts and county offices of education. The state does pro- vide a portion of revenues through several state facilities programs, however school districts are expected to provide local funds in order to receive a proportion of state matching funds." In 1999, the California Department of Education estimated new construction needs for a five year period from 1998—2003 to cost nearly $17 billion.” Hard—pressed to fund nec— essary facilities projects, school districts have turned to issuing local general obligation bonds as a source of revenue to fund capital improve— ments. Passing local school bond measures for facilities expenditures has been a growing trend over the last decade. Since 1986, school dis— tricts have passed 420 school bond measures amounting to nearly $17 billion in revenues}1 In 1998, the state sponsored Proposition 1A, a bond measure which amounted to $9.2 billion, with $6.7 billion reserved for K-12 and $2.5 billion for higher education. The growth in the use of school bonds to fund facilities came in the wake of limitations set by Proposition 13 which imposed a proper— ZOOO School Finance in California: Does History Provide a Sufficient Policy Standard? ty tax rate cap of one percent, required a two— thirds voter approval rate for any local or state tax rate increase, and suspended the ability of municipalities to issue general obligation bonds. As school funding responsibilities shift— ed from the local to the state level, and as local budgets were constrained by an inability to raise revenue for local needs, school districts were unable to respond to the facilities needs of a growing student population—leaving aging buildings to deteriorate in disrepair. In 1986 California voters approved Proposition 46, which amended the state Constitution to permit municipalities to increase property taxes beyond the one percent limit set by Proposition 13, and issue general obligation bonds to finance capital improve— ments for public services. Faced with a backlog of deferred maintenance costs and fewer discre— tionary funds to use for facilities and other gen- eral local needs, school districts began to rely on school bonds as a source to fund capital improvements.” However, a two—thirds majori— ty of voters is still required to approve a prop— erty tax rate increase necessary for repayment ofbonds” The effect of devolving the responsibility for funding new school construction and facilities improvements to the local level in conjunction with a constant reduction in local discretionary funds, contrasted with a school finance system controlled at the state level, has resulted in a two—fold uneven playing field. First, school dis— tricts which are successful in garnering the two- thirds vote necessary for passing a school bond measure will receive state matching funds for construction and likely meet local needs. However, school districts who are unable to pass a school bond measure or are unable to afford the indebtedness associated with repay- ment of a school bond measure, will not be able to receive matching capital improvement funds from the state, and are less likely to meet local needs.H Second and most concerning in light of the Serrano decision which advanced the concept of fiscal neutrality—a low property wealth district will need to levy a higher tax rate in order to repay a bond of equal magni— tude issued by a high property wealth district. In effect, the same equalization efforts that were successfully applied to district revenue limits under court order, have not been applied to capital improvements funding. Thus, placing a substantially uneven fiscal burden on low property wealth districts in their efforts to provide adequate facilities for students. The Current State of Education Fimmce The picture of California’s school funding sys- tem is made complete by a series of tables and charts that track California’s funding history. As has been demonstrated by many analysts, California lags substantially behind other indus- trialized states in America and has had periods in recent years where school funding has not kept pace with inflation. For many years in the 19805 and 19905 it was factually accurate to crit- icize the level of financial support in California as nearly last in the nation. This is no longer true. Estimates vary, but California currently ranks somewhere between 30th and 37th in the United States on per—pupil expenditure, depend— ing on the analysis used. A 1997—98 projection of per—pupil expenditure of $5584 placed California thirty—seventh in the United States." For compar— ison, PACE reported in Conditions of Education in 1994—95 that California ranked 40th in the PACE I 49 50 nation. The contribution of lottery revenues to school funding has also been greatly misunder— stood and accounts for a notably small additional revenue boost; in 1997—98, the lottery provided just over one percent of the budget for education in the state. Figure 1, below, shows the per-pupil funding for students in California from 1971 to the present. A 50—state comparison on several indicators follows, showing the relative ranking of California compared to other states in the nation. As described in the following section, categor- ical funding has grown over the years to a level that approaches 39 percent of total funding in California’s schools, about $9 billion. The distri— bution of these funds, purposely, is not uniform across districts or students and that distribution is complex from both the perspective of state and district policy structures. What is clear, and demonstrated below, is that categorical funding has always been intended to be supplementaiy— even as made clear by the 867771770 court. As a result, the growth in categorical funding has evolved into a web of supplements—whatever the justification—that create instability in the central education funding stmcture. As we will see in the following pages, nearly every conceivable issue in education has a categorical program attached. In total, the proliferation has no coherence, is diffi- cult to understand, lacks consistent accountability structures, and is not linked to the evolving edu— cation reform strategies in the state. $7000 $6000 $5000 $4000 $3000 $2000 $1000 _ / / / / $0 1969—70 1979—80 I United States - California 1989-90 1995—96 Figure 1. Expenditures per Pupil (in current dollars) in Average Daily Attendance in Public Elementary and Secondary Schools. California and the United States: 1969-70 to 1995-96 CRUCIAL ISSUES IN CALIFORNIA EDUCATION 2000 School Finance in California: Does History Provide a Sufficient Policy Standard? Unadjusted education spending per student (1 998) Education spending per student, adjusted for regional cost differences every $1,000 in state wealth (gross state product) (1997) (1 998) Education spending for Alabama Alaska Arizona Arkansas California Colorado Connecticut Delaware Florida Georgia Hawaii Idaho Illinois Indiana Iowa Kansas Kentucky Louisiana Maine Maryland Massachusetts Michigan Minnesota Mississippi Missouri Montana Nebraska Nevada New Hampshire New Jersey New Mexico New York North Carolina North Dakota Ohio Oklahoma Oregon Pennsylvania Rhode Island South Carolina South Dakota Tennessee Texas Utah Vermont Virginia Washington West Virgina Wisconsin Wyoming US. $4,780 $8,337 $4,593 $4,590 $5,514 $5,519 $8,790 $7,425 $5,579 $5,587 $5,415 $4,628 $6,182 $6,226 $6,019 $5,645 $5,539 $5,352 $6,614 $6,678 $7,642 $6,979 $6,636 $4,291 $5,523 $5,769 $6,021 $5,219 $6,493 $10,140 $4,961 $8,808 $5,288 $5,098 $6,178 $4,794 $6,205 $7,382 $7,642 $5,276 $4,842 $4,767 $5,482 $3,632 $6,672 $6,023 $6,225 $6,189 $7,123 $6,218 $6,168 $5,356 $6,581 $4,629 $5,268 $4,939 $5,599 $7,635 $7,255 $5,829 $5,998 $5,430 $5,029 $5,991 $6,661 $6,823 $6,311 $6,196 $5,989 $6,739 $6,544 $6,518 $6,873 $6,767 $4,924 $5,817 $6,349 $6,799 $5,478 $6,195 $8,801 $5,339 $7,853 $5,763 $5,979 $6,251 $5,317 $6,422 $7,202 $6,930 $5,827 $5,667 $5,223 $5,815 $3,804 $6,746 $6,215 $5,995 $6,908 $7,448 $6,790 $6,168 $36.32 $44.80 $35.67 $38.96 $32.75 $33.00 $37.97 $27.78 $35.64 $35.32 $30.22 $41.95 $33.30 $47.46 $39.32 $42.58 $36.44 $31.19 $49.47 $39.56 $33.21 $48.43 $41 .48 $34.86 $36.49 $49.19 $39.13 $30.58 $35.27 $42.77 $37.53 $40.44 $29.70 $36.22 $38.93 $41.04 $35.82 $42.29 $43.58 $40.32 $34.61 $29.08 $37.23 $40.24 $53.09 $34.48 $39.04 $51 .34 $45.91 $36.13 $37.35 Table 1. Comparative Indicators on Measures of School Finance: California Compared to the Fifity States (1998) Source: Selected Tables, Quality Counts 2000, Education Week, Volume XIX, Number 18, January 13, 2000 PACE a California’s Description Level Rank Nationally Unadjusted education spending per student (1998) $5,514 33rd Education spending per student, adjusted for regional cost differences (1998) $4,939 47th Education spending for every $1,000 in state wealth (gross state product) (1997) $32.75 44th Relative inequity in spending per student among districts 11.4% 22nd Percent of annual education expenditure spent on instruction (1997) 60.6% 36th Percent of total taxable resources spent on education (1997) 3.3% 41st Table 2. California’s National Ranking on Selected Measures Related to School Finance Source: Selected Tables, Quality Counts 2000, Education Week, Volume XlX, Number 18, January 13, 2000 52 The combination of categorical support and revenue limit support ought to be coherent and be tied to what we best know about educational effec— tiveness. But as the pattern of categorical funding continues, it is essential, from the state level, to assess whether there is internal coherence to the linkage between finance and educational goals, as exemplified by California’s evolving standards— based education reform agenda. The Growth of Categorical Funding In the past forty years the number of state cate— gorical programs in public elementary and sec— ondary education has grown from five to more than eighty. Why has there been such prolifera— tion? Why are new programs being created every year in Sacramento, even before last year’s new programs have been explained to the schools, much less implemented and evaluated? In an effort to inform and stimulate debate, we begin by outlining the history of categorical programs and making some observations about the process that produces them year after year.“ 1959-1960 through 1965-1966: The Years of Stability In the early 1960s, there were five categorical programs that accounted for only 8 percent of total State General Fund (SGF) allocations to school districts and county offices of education. The graybeards of categorical programs, all of which are alive and well after forty years (though Driver Training has had its ups and downs), are: ' Special Education ' Pupil Transportation ' Free Textbooks/Instructional Materials ' Children’s Centers/Child Development ' Driver Training During the seven year period from 1959—60 through 1965—66 only two significant addi— tions were made to the original five. A pro— gram of funding for the Mentally Gifted (later called GATE—Gifted and Talented Education) was established in the early 19605, and the first efforts were made in the field of compensatory education with the creation of a teacher training program funded at $900,000. CRUCIAL lssuus IN CALIFoRNiA EDUCATION 2000 School Finance in California: Does History Provide a Sufficient Policy Standard? 50% 40% 30% /\ 20% [\V 10% ,_/ 0% 1959—60 1969—70 1979-80 1989-90 1995—96 I Percent of General Funds Allocated as Categorical Funds Figure 2. California’s Growth in Categorical Funding, 1959-60 to 1998-99. Percent of State General Funds Allocated as Categorical Funds $10 $6 $4 $2 / 1959—60 1969-70 1979-80 1989-90 1998—99 $0 I Total Categorical Allocations Figure 3. Expenditures in Billions (in current dollars) on Californian Categorical Funding 1959-60 to 1998-99 PA C F, I 1966-67 through 1968-69: Potential for Growth In just three years at the end of the sixties, cat- egorical program funding doubled as a per— centage of total SGF allocated to schools (increasing from 8.2 percent to 16.8 percent). While general funds appropriated to the schools increased 23 percent during this three- year period, funding for Special Education and for Children’s Centers nearly doubled. Allocations for compensatory education were flat at about $10 million over the three years, but, interestingly, compensatory education included one of the first class-size reduction efforts, with more than $12 million allocated for this purpose. Two of the most durable categorical programs ever established were given birth at this time: 0 Miller—Unruh Reading (known originally as the Special Elementary School Reading Instructional Program) 0 Demonstration Programs in Reading and Math (later renamed Demonstration Programs in Intensive Instruction) Reflecting the power of the programs’ authors, funding for Miller—Unruh leaped from $2 mil— lion in 1966—67 to more than $15 million two years later. The Demonstration Program in Reading and Math, which was started as a com— ponent of the compensatory education pro— gram, survived numerous assaults by the Legislative Analyst Office in the mid—1980’s and continues to operate in the new millenni— um, although inflation has taken a toll on the amount of funding it receives. CRUCIAL ISSUES IN CALIFORNIA EDUCATION 1969- 70 through 1973-74: Stability Ending With A Flourish Total SGF allocations to schools grew less than 10 percent between 1969-70 and 1972-73. Categorical funding was fairly stable during these years, increasing slowly from 16.8 per- cent of SGF allocations to K-12 education in 1969-70 to about 18 percent in 1973-74. But 1973-74 saw a tremendous increase in state revenues, enabling a 42 percent increase in SGF dollars for public school education. Wilson Riles, first elected Superintendent of Public Instruction in 1970, found his opportu— nity, sponsoring two important new categorical programs in 1973: ' Early Childhood Education (initial funding of $24 million) ' Educationally Disadvantaged Youth ($81 million) These programs were essentially offshoots and expansions of the old compensatory education program (established in 1965-66) which had limped along for nearly a decade with $10 mil- lion in annual funding. 1973-74 also witnessed the creation of Bilingual—Bicultural Education which grew from $4 million to $11 million and was eventually folded into the Economic Impact Aid program in 1978—79. Meanwhile, the old stalwarts continued their steady growth: GATE reached $12.6 mil— lion in 1973-74; Driver Training achieved $16.4 million; Instructional Materials climbed to $24 million; Special Education (the largest of all categoricals) grew to $180 million; Transportation continued its steady ascent to 2000 School Finance in California: Does History Provide a Sufficient Policy Standard? $35 million; and Miller—Unruh leveled off at $18 million. 1 974- 75' through 1 978 - 79: Restructuring School Finance From 1974-75 through 1978-79 funding for categorical programs more than doubled to nearly a billion dollars ($976 million). As a per— centage of the SGF going to the schools, the categorical share peaked in 1977—78 at 29 per— cent. It then precipitously tumbled to only 17 percent in 1978-79. A revolution had struck school finance that would change the game entirely: As noted earlier, Proposition 13 was passed by the voters, severely limiting local property taxes, and thereby shifting control over the funding of K—12 education from school districts to the state. Armed with a huge surplus of funds stem— ming from the strong recovery from the reces— sion of 1973, in 1978-79 the state increased its allocation to local schools by $2.4 billion, an increase of greater than 80 percent. Significantly, revenue limits did not grow—this money was provided to backfill a shift of property taxes from schools to local government. Notably, though, relatively little of the huge expansion of state funding for schools went into categoricals, which only increased from $862 million in 1977- 78 to $976 million in 1978—79. Thus, most of the new money went into unrestricted general aid because it replaced local property taxes. The emphasis continued in the late—19705 to be on the provision of services for children of low-income families. The following programs were added: ' A state child nutrition program was started in 1974-75 at $13 million, which grew to $33 million by 1978-79. ° A major initiative in child care was begun in 1977—78, funded at $73 million to start. ' In 1978—79, Riles’ Early Childhood Education and Educationally Disadvantaged Youth programs (along with the much small- er Bilingual—Bicultural program) were recon- figured and folded into two new programs: the School Improvement Program (SIP) and the Economic Impact Aid (EIA) program. From 1973—74 through 1978—79, funding for compensatory education increased from $120 million to nearly $250 million. SIP and EIA survive today, both being funded at about $400 million each in 1998-99. ° State funding for desegregation was initiated in 1978—79. This program, which benefits only a small percentage of all the school dis— tricts in the state, has grown to $633 million in 1998—99. 0 Urban Impact/Meade Aid began in 1976—77 with $8 million and reached nearly $100 mil— lion in the late—1980s when it was rolled into the revenue limit. As its name suggests, most of this money benefited large urban districts such as Los Angeles and San Francisco. The Urban Impact/Meade Aid soon became one of the most notorious categorical programs as it targeted money to specific districts. 1979-80 through 1985-86: A New Emfor Categoricals A major change in the nature of categorical pro— grams in California occurred between 1979—80 PACE I and 1985—86. This change would escalate in the second half of the 80s and continue throughout the 1990s. The change was the creation of the “mini—categorical.” Mini—categoricals were new state programs of local assistance with very nar— row purposes, usually fiinded at very low levels, as the legislative author was typically more inter- ested in establishing the program than in the funding level. Because of their narrow focus, these mini—categoricals never developed the political support for large-scale expansion. Whereas the earlier categoricals aimed to pro— vide categorical services to all eligible pupils, schools, or districts, the new mini-categoricals, on account of their narrowness, rarely increased their funding beyond cost-of-living-adjustments. 1982-85 1998-99 Foster Youth Services $0.7 $4.5 High School Counseling $6.5 $14.6 Small School District Transportation $18.6 (in revenue limit) lntergenerational Education $0.17 $0.17 Youth Suicide Prevention $0.3 $0.0 Curriculum on Birth Defects $0.5 $0.0 Agricultural Education $3.1 $3.9 Specialized Secondary Schools $2.1 $4.4 Drug & Alcohol Abuse Prevention $0.43 $0.0 School Business Personnel Staff Development $0.25 $0.0 Educational Technology $1.9 $55.4 Institute of Computer Technology $0.1 $0.49 Education Improvement Incentive Program $14.4 $0.0 Classroom Teacher Instruction Improvement $17.1 $0.0 Peninsula (Partnership) Academies $0.6 $14.0 Vocational Education Student Organizations $0.43 $0.66 Teacher Education and Computer Centers $6.3 $0.0 School/Law Enforcement Partnership $0.15 $26.4 Pupil Dropout Prevention & Recovery $2.7 $18.9 Minimum Teacher Salary $2.9 (in revenue limit) Administrator Training & Evaluation $1.5 $6.1 Year-Round Schools Incentives $4.2 $71.7 Mentor Teacher Program $10.8 $80.6 Table 3. Funding of Major Categorical Programs (in millions): 1982-85 and 1998-99 56 CRUCIAL lSSL'ES IN CALIFORNIA EDUCATION 2000 School Finance in California: Does History Provide a Sufficient Policy Standard? The following categorical programs (in mil— lions) were created between 1982—83 and 1985- 86. Many were included in SE 813, a legendary package which emerged out of spectacular bud- get negotiations in the spring and summer of 1983. Prior to SE 813, it was assumed that reg— ular education programs were okay, and that categorical support should be targeted for stu— dents with special needs. But SB 813 chal— lenged this assumption, inferring that regular education needed support as well. One of the most significant developments during the late 70s and early 80s was the emphasis on staff development. Expansion of categorical programs for teacher training began in 1977-78 and 1978—79 with the cre— ation of the Bilingual Teacher Corps, Professional Development Centers, and School Personnel Staff Development 8: Resource Centers. However, these programs were each funded at less than $2 million annually. The big push came in 1983 with the creation of the iVIentor Teacher Program, the Teacher Education and Computer Centers, and the Classroom Teacher Instructional Improvement Program—all at much higher levels of funding. The early 805 also saw the establishment of training programs for administrators and school business personnel. In 1978—79 and in 1982—83 there were 23 categorical programs. By 1985—86, there were 44. Growth in the number of programs was accompanied by major growth in the percent— age of the SGF allocated to schools that was devoted to categoricals and in the total dollar amount (up from $976 million to $2.9 billion.) From the reduced categorical funding base of 18 percent established in 1978—79 when the state bailed out the school districts after the passage of Proposition 13, the percentage climbed back to 27 percent in 1985—86, approaching its all time high of 29.3 percent in 1977—78. (Actually, it would not be until 1990— 91 that the percentage allocated to categoricals would exceed the 1977—78 benchmark.) Growth in the major categoricals was substan— 1978-79 1985-86 Pupil Transportation $60 $286 Child Care $80 $265 Instructional Materials $43 $95 School Improvement Program $123 $214 Economic Impact Aid $123 $195 Urban Impact Aid $44 $86 Special Education $237 $879 Desegregation $60 ’ $289 Table 4. Funding of Major Categorical Programs (in millions): 1978-79 and 1985-86 PACE tial during the first half of the 80s as shown below in Table 4. 1986-87 through 1990-91: Prop 98 Creates an Environment fbr More Categoricals As noted earlier in this chapter, the passage in 1988 of Proposition 98, which attempted to define a minimum level of funding for K-12 education, had significant consequences for resource allocation to the schools. Prior to Proposition 98, decisions about funding for the schools were made, to a much greater extent than after, in the context of the overall budget for all public services. For example, during the Deukmeiian zulministration, battles over the K— 12 COLA were waged in the context of how much should go for welfare, health, and higher education. Proposition 98 resulted in the virtu- ally total isolation of school funding decisions from the rest of the state budget. K-13 and the community colleges were given a “number,” based on the complex Proposition 98 formula, and then were left on their own to split up the money (there were contentious disputes within and between the K—12 and the community col— lege segments, however). After Proposition 98, the education budget was in some years a “done deal” weeks before the final budget negotia— tions even occurred. The isolation of the K—12 education budget heightened conflict between contending educa— tion interests. Their focus now was on dividing a known pie, not on fighting to increase the total size of the pie. The first great negotiation of the Proposition 98 era was held in the spring of 1988. Legislators and their own staffs from both houses, representatives of the governor, and staff members from the Department of Finance met for several weeks in the Cavern. lSSL'l-ZS 1.\' CALIFORNIA Fan'cx’riox Governor’s Conference Room to negotiate the budget. Assembly Republicans threw the deal— making for a loop when they demonstrated that large urban districts with mighty political mus- cle, such as San Jose, Los Angeles, and San Francisco, were receiving from $600 to $1300 dollars more per-pupil for compensatm‘y educa- tion than many other districts which in fact had higher percentages of low-income, minority, and non-English speaking pupils. The upshot was the creation of the Supplemental Grants program, which targeted money for districts with relatively low amounts of funding per- pupil based on the combined total of their cate— gorical funds and revenue limit income. A final significant consequence of Proposition 98 was the creation of large pots of “one—time” money each year. ()ne-time money resulted when the Department of Finance underestimated the amount of the Proposition 98 guarantee (which it invariably did) so that at the end of the fiscal year the Proposition 98 guarantee was underfunded. The amount underfunded was considered “one—time” money by prudent budgeters because it would be allo— cated and spent in the succeeding fiscal year. It was not possible for districts to spend that money on ongoing obligations (such as salaries) because that, in effect, would be double—count— ing the money as part of the ongoing Proposition 98 base. While most of the one— time money was allocated in large block grants on a per—pupil basis or on a per school—site basis (and there usually were “categorical” strings on how the money could be spent), it became a wonderful source for “education pork." Governor Wilson, in the late 905, was the first to permit wholesale allocations of funds to indi— vidual districts for isolated projects. ZOOO School Finance in California: Does History Provide a Sufficient Policy Standard? Also under Proposition 98 the “hidden” cat— egorical programs were given increased scruti— ny because they took money out of the pie. The hidden categoricals are state mandates passed by the legislature directing schools to perform certain actions such as medical exami- nations and collective bargaining. These costs are funded out of Proposition 98 dollars. By the 905, the cost of state mandates had reached the neighborhood of $100 million per year. In general, it is difficult to determine whether Proposition 98, with its constraints, has improved or impaired the quality of educa— tion budget—making. It has not reduced the use of categoricals, that is for certain. Perhaps the more focused debate over the use of education funds has been a benefit. But Proposition 98 also sullied the decision—making process to a certain extent. NIany times a legislator (whose legislative specialty was not education) sitting on an education committee would ask if a pro— posed new program was to be funded out of Proposition 98 dollars. If the answer was yes, then the legislator did not care very much whether the program was passed or not because it was “Prop 98 money” that would be going to education one way or another. In addition to Supplemental Grants, the ini- tial years of Proposition 98 saw the continued emphasis on teacher training through new cate- goricals. Between 1988 and 1990, three major new training programs and one small one were established: ° Professional Development Program (Initial funding of $20 million in 1988-89) 0 New Teacher Project ($3 million in 1988-89, growing to more than $70 million today) ° Subject Matter Projects ($5.3 million in 1990—91) 0 Geography Education ($100,000 in 1989—90) The negotiations that produced Supplemental Grants also established two significant new programs: 0 High School Class Size Reduction ($31 mil- lion in 1990—91) ° School Restructuring Grants ($6.5 million in 1990—91) The huge windfall in the first years after Proposition 98’s passage also fueled the contin- ued growth of the traditional categoricals: Increases in these programs—particularly in Special Education—boosted the percent— age of SGF allocations to schools that was in categoricals to 31.6 percent in 1990—91, sur— 1985-86 1990-91 Special Education $879 $1,681 Year—round School incentives $4.5 $35.4 School Improvement Program $214 $315 Economic Impact Aid $195 $272 Desegregation $289 $503 Child Nutrition $31 $52 Table 5. Funding of Major Categorical Programs (in millions): 1985-86 and 1990-91 PACE Z ()0 1990-91 1994-95 Special Education $1,682 $1,617 GATE $29 $32 Pupil Transportation $328 $332 Miller—Unruh $22 $22 Desegregation $503 $502 School Improvement Program $315 $317 Child Care $325 $407 Instructional Materials $129 $164 Education Technology $14 $23 Economic Impact Aid $272 $324 Table 6. Funding of Major Categorical Programs (in millions): 1990-91 and 1994-95 passing for the first time the previous record of 29.3 percent set more than a decade earli— er in 1977—78. 1991-1992 through 1994-95: Recession Strikes Pete Wilson came into office in 1991with his education agenda. Though he was able to establish two new categoricals in 1991—92 that embodied his ideas, the deep recession of the early 90s thwarted his ambitions and also slammed the door shut on increases for most of the traditional categoricals through 1994—95. Rapid economic growth after that year led to an explosion of new programs and ultimately to the creation of the Class Size Reduction cate— gorical program for grades K—3. Of the programs that did not have flat growth between 1990—91 and 1994—95, Instructional AIaterials, Education Technology, and Economic Impact Aid picked up in 199—1— 95 as the economy started to recover. Child care was a Governor \Vilson priority through— out his years in office and he engineered CRUCIAL lssi'ias IN CALIFORNIA EDUCATION increases in this area during the early 90s. Wilson also pushed through two of his preven- tion programs in 1991 before the brunt of the recession was felt: Healthy Start (initial funding of$19 million in 1991—92) and Prenatal Substance Abuse ($4 million in 1991—92). 1995-96 through 1998-99: The PVilson Legacy California’s economy surged upward in the second half of the 90s. The traditional categor— icals resumed their growth at a rapid clip as shown in Table 7. Perhaps the most important event driving the budget process in the Spring of 1995 was the revelation the previous Fall that the reading skills of California students were near the bot— tom among all the states in the nation. The \Vilson administration responded with the California Reading Initiative, a combination of staff development and instructional materials to reinstate systematic phonics instruction in K—3. The dismal NAEP test scores, plus ten years of research on how children learn to read and 2000 School Finance in California: Does History Provide a Sufficient Policy Standard? bipartisan support, pushed the $167 million California Reading Initiative through the legis— lature with Virtually no opposition in 1995. This was followed up in the budget of 1997—98 with an additional $56 million for reading staff development. Also on the minds of people in 1994 and 1995 was school Violence. Never to lose an opportunity to solve a problem by creating a program, the legislature enacted, and Governor \Vilson signed into law, the following programs: 0 Metal Detectors (initial funding of$1 mil— lion in 1994—95) ° School Crime Report ($800,000 in 1994-95) ° Gang Risk Intervention ($3 million in 1995 - 96) 0 Targeted Truancy and Public Safety ($10.8 million in 1995—96) ' School Violence Reduction ($4.2 million in 1995—96) ' Conflict Resolution ($2.2 million in 1995 —96) In conjunction with these programs, Governor Wilson pushed for zero-tolerance for certain offenses on campus, such as carrying a gun. But the Democrats refused to throw students deemed guilty of such offenses “out on the street,” leading to the creation of another new categorical program—Community Day Schools for expelled pupils—that grew to $20 million by 1998-99. Additionally, Wilson proposed Single Gender Academies as a solution to the problems of restless young boys. It was passed and funded at $5 million for starters. The rapid growth of the Internet was the spark that ignited the Wilson administration’s major initiative in education technology—the Digital High School which was proposed in 1997 as a five—year, half-billion dollar program. Fuelled by a combination of one-time money and ongoing funds, the Digital High School incorporated a unique allocation strategy: high schools were grouped by size and within each 1994-95 1998-99 Special Education $1,617 $2,112 GATE $32 $56 Pupil Transportation $332 $521 Miller-Unruh $22 $32 Desegregation $502 $633 School Improvement Program $317 $395 Child Care $407 $794 instructional Materials $164 $172 Education Technology $23 $55 Economic Impact Aid $324 $401 Pupil Assessment $6 $67 Table 7. Funding of Major Categorical Programs (in millions): 1994-95 and 1998-99 PACE I group grant recipients were selected at random. Over the five—year period, all high schools would eventually receive a large grant for hard— ware, software, and staff training. A major program that began in 1997-98 was the Staff Development Days Buy—Out, which required districts to provide 180 days of instruction while prohibiting the practice of using up to eight of these days for staff training with the pupils not in school. This program cost $50 million in 1997-98 and $195 million in 1998—99. The great structural reform of the 905 was, of course, Class—Size Reduction in grades K-3. Funded at $771 million in 1996—97 for three grade levels, it grew to $1.58 billion annually by 1998-99 as participation increased. For many years legislation had been introduced to reduce class size, but only a half-hearted effort in grades nine and ten had ever passed. However, in the mid-19905, reports from other states, particularly Tennessee, indicated that class size reduction had improved student achievement. The availability of large quanti— ties of new money, Governor Wilson’s presi— dential ambitions, and his animosity towards the California Teachers Association, combined to make the time right to see whether reading and math deficiencies in the early grades could be remedied through class size reduction rather than through changes in instructional methods. Further, Governor Wilson was looking for Science Laboratory Equipment Mathematics Teacher Training School Site Grants College Preparation Grants College Admissions Test Preparation Advanced Placement Test Fees International Baccalaureate Program Grade Nine Class Size Reduction School Library Materials Remedial Summer School High-Risk First-Time Offenders Math Instructional Materials Aligned with New Content Standards After-School Learning & Safe Neighborhoods Partnership Program Community-Based English Tutoring (Prop 227) Selected One-time Allocations in 1998-99 (in millions) $71.5 $250.0 $28.5 $180.0 Selected New Programs Established in 1998-99 (in millions) $50.0 $5.0 $10.0 $1.5 $1.1 $50.0 $44.5 $158.5 $105.0 $20.0 Table 8. Selected One-time Allocations in 1998-99 and Selected New Programs Established in 1 998-99 CRUCIAL ISSUES IN CALIFORNIA EDUCATION 2000 School Finance in California: Does History Provide a Sufficient Policy Standard? assurances that these new monies would not find their way solely into augmented salaries for teachers. One—time monies were abundant in the late 90s, giving rise to large “block grants” with strings attached. Per ADA block grant funds were often restricted to purchases of books, computer equipment, and deferred mainte— nance. Furthermore, grants to school sites cir- cumvented the traditional authority of local school boards to allocate funds among schools within districts. One of the most troublesome practices to emerge during the \Vilson administration was the pork—barreling of one-time funds. \Vhile the Chino USD Drug Awareness Program, the Soledad USD Library, the Claremont USD Special Education Facility, and many others may be worthwhile projects, line-itemizing them in a wholesale manner in the State Budget in 1995—96 was a new practice that raised legislators’ expectations for future years. They were not disappointed in 1998—99 when nineteen additional individual district projects were funded. In the last budget (1998-99) produced under the aegis of Governor Wilson, the state allocat— ed more than $3 billion of new funds to K—12 education. These funds not only fueled the growth of the traditional categoricals, but were also used for substantial one—time categorical grants and the creation of new programs. h/lany of the new programs have a compen- satory education theme—remedial summer Students with Disabilities Special Education Early Intervention for Success Specific Sub-populations Mentally Gifted/GATE lndian Education Centers Indian Education Programs Foster Youth Services High School Counseling Single Gender Academies Community Day Schools Remedial instruction Summer School California/Japan Scholars Scholarships Transportation Transportation of Regular Pupils $2,111.9 $1.9 $55.6 $3.4 $0.47 $4.5 $14.6 $3.8 $20.4 $105.0 $0.05 $521.1 Table 9. California’s Categorical Programs (in millions) 1998-99 Source: PACE Analysis PAC F. A ()3 ()4 Specific Subjects Special Elementary School Reading Instructional Program Conservation Education (Environmental Ed.) Intergenerational Education Agricultural Education Specialized Secondary Schools Local Arts Ed Partnership Grants international Baccalaureate California Civil Liberties Public Education Latino Museum Community-Based English Tutoring (Prop 227) Science Lab Equipment (one—time) Administration Loans (WCC: 90-91; Coachella: 91-92; Compton 93-94, 97-98, 98-99) Deferred Maintenance Year-Round Schools Incentive Program Administrator Training and Evaluation Reader Services for Blind Teachers Teacher Dismissal School -based Management Standardized Account Code Structure County Office Oversight Pupil Residency Verification FCMAT—CSIS FCMAT—Comptton Recovery Plan Child Development Child Care Instructional Materials Free Textbooks/Instructional Materials Instr. Mat. for Standards Aligned Core Curriculum (math) Educational Technology Educational Technology Institute of Computer Technology Digital High School Single-School Districts—Laptops Technology Literacy Challenge Grants $31.8 $0.5 $0.17 $3.9 $4.4 $3.0 $1.1 $1.0 $0.75 $50.0 $71.5 $0.3 $135.0 $71.7 $6.1 $0.29 $0.03 $0.98 $5.5 $3.7 $0.16 $3.0 $0.5 $793.6 $172.1 $250.0 $55.4 $0.49 $136.0 $1.2 $45.2 Table 9 continued. California’s Categorical Programs (in millions) 1998-99 CRUCIAL ISSUES 1N CALIFORNIA EDUCATION 2000 School Finance in California: Does History Provide a Sufficient Policy Standard? Compensatory Education— Low-Income Populations/Bilingual School Improvement Program $394.5 Economic Impact Aid $400.9 Opportunity Classes and Programs $8.1 Healthy Start $49.0 Advancement Via Individual Determination $1.0 College Preparation Grants $5.0 Advanced Placement Exams $1.5 College Admission Tests Program $10.0 Desegregation/Compensatory Programs Court and Federal Mandates—Desegregation $632.7 Reform Demonstration Programs in Reading and Math $5.9 Class Size Reduction (high school) $35.4 Class Size Reduction (ninth grade) $44.5 Class Size Reduction (K—3) $15809 Staff Development Day Buyout $195.0 Vocational Education Regional Occupational Centers and Programs (ROC/Ps) $309.4 Peninsula (Partnership) Academies $14.0 Vocational Education Student Organizations $0.66 Food Child Nutrition $74.5 Staff Development Bilingual Teacher Corps—Bilingual Teacher Training $1.5 Mentor Teacher Program $80.6 Teaching Improvement Programs (Intersegmental) $1.7 Professional Development Program $19.1 New Teacher Project (Beginning Teacher Support) $67.7 Geography Education $0.11 Math Staff Development $28.5 Individual District/Area Projects Regional Science Resource Center $1.5 Table 9 continued. California’s Categorical Programs (in millions) 1998-99 PACE I Chino USD Drug Awareness $0.6 Angel Gate Academy $0.6 Apportionment to Oxnard USD $4.2 LAUS—At-Risk Youth $0.6 LAUSD—CA Arts Initiative $0.3 Huntington Beach High School $0.1 Napa Valley ROC/P Computer Equipment $0.35 Pasadena USD—Books for Tutoring $0.02 Santa Paula USD—Pool Renovation $0.08 Montebello USD—School Security Devices $0.05 LA County Office of Ed.—Middle School Civics Curr. $0.18 Lucia Mar USD—Performing Arts Center $0.5 Loa Alamitos USD—High School for the Arts $0.7 San Bernardino COE—Afterschool At-Risk Youth $0.03 Santa Clara COE—Develop Ed Networks $0.05 imperial COE—Tech Infrastructure Project $0.35 Anaheim City USD—Anaheim Archives $0.38 Merced COE—Pilot Job Opportunities Program $0.15 Bellflower USD—Bellflower Against Gangs $0.05 Glendale USD—Facilities Technology $1.0 Glendale & Burbank USDs—MTL School to Work $0.1 Grossmont HSD—Athletic Facility $0.2 Targeted General Aid School Site Grants (one—time) $180.0 State Mandates State Mandates $97.2 Long Beach Mandate Payment $4.1 At-Risk Youth Pupil Dropout Prevention and Recovery $18.9 Gang Risk Intervention , $3.0 High—Risk First—Time Offenders $20.0 School Safety School Law Enforcement Partnership $26.4 Conflict Resolution $0.3 School Community Policing $10.0 Library Protection $0.7 Table 9 continued. California’s Categorical Programs (in millions) 1998-99 66 CRUCIAL ISSUES IN CALIFORNIA EDUCATION 2000 School Finance in California: Does History Provide a Sufficient Policy Standard? After—School Learning & Safe Neighborhoods Program Statewide Pupil Assessment Pupil Assessment Golden State Merit Diploma School Libraries School Library Materials Total 1998-99 Categorical Programs Table 9 continued. California’s Categorical Programs (in millions) 1998-99 school, after—school learning, preparation for billion in 1998-99, reaching a record high of 39 college admissions tests, high—risk first-time percent of SGF allocated to schools. offenders, advanced placement test fees, and A complete listing of California’s categorical college preparation grants. The only new pro— programs is shown above for the 1998—99 year. grams directly linked to academic standards recently promulgated by the state board of edu— 1999-2000 Governor Gray Davis and The cation are the math instructional materials and New Millennium teacher training programs. The upshot of the Immediately upon his election in 1998, frenetic activity around categoricals in the late Governor Gray Davis called a special session of 905 was that total funding for these programs the Legislature to address education issues. increased from $4.7 billion in 1994—95 to $9.3 What came out of that session was a set of Teacher Peer Review (plus $83 million: Mentor Teacher Program) Performance Incentives for Teachers at Low—Performing Schools Immediate Intervention / Under Performing Schools Program Governor’s School Performance Awards Elementary School Intensive Reading Per-Pupil Block Grants for Instructional Materials $42 $50 $96 $96 $75 $134 Table 10. Major New Categorical Programs in 1999-2000 (in millions) PAC E W ()7 ()8 strict accountability policies, coupled with incentive structures for school improvement. New accountability measures have been linked to the STAR achievement tests, which are gradually being transformed from a nationwide standardized exam to one that reflects the acad— emic content standards that have been adopted by the California State Board of Education. The major new categorical programs con— tained in the 1999—2000 budget are shown in Table 11. According to the Department of Finance, with the $134 million for textbooks in addition to other instructional materials funding in the 1999—3000 budget, all schools will have suffi— cient funds to purchase for each pupil three textbooks that are aligned with the new state standards. Szmmzmjy: Not counting the individual dis- trict projects, a complete listing shows more than 150 categorical programs in operation in the public schools during the past forty years." \Ve should note that each year there are scores of categorical program proposals that do not become law, and over the past twenty years hundreds, if not thousands, of new programs have been stopped in the appropriations com— mittees of both houses. A retrospective on the patterns of categori— cal funding indicate a progression from one set of emphasis to the next over the years. Initially, beginning in the 19605, categorical funding was targeted for the educationally disadvantaged through compensatory programming. By the late—1970s and early—19805, programs were increasingly being developed to expand into the regular education program to provide addition— Carer“, Isst'its 1.\' CALIFORNIA EDL'CATION al capacity to schools. Ten years later, the emphasis changed to be more targeted to pro- fessional development, linked in part to the emerging California content frameworks. The proliferation of supplemental grants came next. And finally, the late—19905 introduced class size reduction as the largest program outside of spe- cial education. In recent years, additional cate- gorical emphasis has been placed in violence prevention and technology, but the funding is less substantial than the other areas of emphasis that have been noted. Over thirty years, the shift of control over the financing of schools from local districts to the state legislature has resulted in a new sense of responsibility in Sacramento for the perfor— mance of the public schools; this certainly seems to represent the spirit of today as no plausible category of programs is untouched by the categorical funding stream. More than sim— ply the number of programs, the sum total of categorical funding is not systematically linked to the state’s education reform agenda in any coherent fashion. Oddly enough, in a policy environment where accountability for outcomes is of greatest concern, the financing system is left out of the rationale. By establishing categorical programs, the state sets up default mandates for the ways in which spending should occur and schools should be organized. But if these very mecha— nisms that the state has mandated fail, can blame really be placed in the schools? The irony is that categorical funding, and all that it supports, is ultimately exempt from the rigor of the school and student accountability system that the state has implemented in recent years. 2000 School Finance in California: Does History Provide a Sufficient Policy Standard? Next Steps: Exercising Caution in School Finance Reform This chapter has provided a line of argumenta— tion that there is not a clear connection between the state’s educational objectives and the funding system that has evolved. As has been suggested by many analysts over the past several years, the time to restructure education finance has arrived. The solutions involve con— siderations of equity, adequacy, and governance, each of which bring substantial complexity to the reform conversation. Equity Concerns Categorical programming, once intended to add compensatory services for the few to increase opportunity, now adds substantial ser- vices of many kinds for most students. The implications of this change need to be well understood not only from the perspective of fiscal equity, but also inequities that avail them— selves by highly varied educational opportuni— ties across schools in California. A recent report of the US. General Accounting Office explains that when state and local revenues are summed and adjusted for the cost of education and student need, California is reported to have substantial intrastate fiscal disparities.” At the same time, analysts are increasingly able to demonstrate substantial inequities across the state, both within and across school districts: teacher quality, course offerings, physical conditions, among others. California, once seen as leading the nation in the extent to which equity drove the education system, now is seen as just about average.”And while the revenue limit carries a burden of equalizing the system, it has increasingly less leverage with the substantial growth in categor- ical funding. Raising this point is fundamental because conversations about the rationality of the school finance system must embrace, at the same time, overarching concerns about equity, and at the same time, educational objectives of the system. For example, we are at the same time bound by explicit considerations of equity (through the Sermno decisions), and pressures to provide additional support through categorical funding to districts, where needs are demonstrated and opportunities avail themselves. While on its face, a rational set of objectives, and completely consistent with the intent of the court, we sug- gest that the Serrano equity principle is harder to apply in policy discussions as categorical support grows. In part the difficulty is measur— ing the extent to which revenue limits and cate— gorical funds reach their intended goals togeth— er, and making a set of leaps in judgement that the ultimate allocations reflect equitable distri— butions in an educational sense. It should not be surprising that this difficulty is upon us. Of course the equity rationale of the original Serrano decisions is about fiscal equity—the distribution of revenues being inde— pendent of property wealth. We would argue that the distribution of categorical aid also ought to be about equity, but equity that is driven by educational need, with specific goals of effective— ness as a leading principle. Of course the central point is that the two conceptions of equity are difficult to blend, and the current financing sys- tem reflects this difficulty in spades. We argue that the policy constraints in California since the 19705 set an odd prece— dent—here and across the country. It became “policy lore” that if a state increased equaliza— PACE I 6‘) tion of funding following coru't-ordered reform, as did California, then the likely event to come would be a substantial reduction in funding for K—12 education. Several points are inaccurate. First, even while California’s relative funding level declined compared to the national average post—Serrano, and the decline was substantial, real funding continued to improve year-over- year for much of the past thirty years. Second, while the relative decline that California experi— enced is indisputable, the events in California are not those that other states have experienced when substantial school finance litigation cat- alyzed the school finance system to improve equity. The main point is that histon has demonstrated that greater equalization of spending in a state is not incompatible with higher levels of state spending for K-12 educa— tion.” Of course, California’s idiosyncratic school finance history is different from other states, as is the extent to which economic declines in past years impeded state general fund growth. Perhaps more than anything, the policy constraints that California faces, taken together, are unlike other states to be sure. The emphasis on the equity of funding as seen in state policy discussions through only the revenue limit structure, over many years, has contributed to a displacement of discus— sions about larger questions of equity in California. \Vithout ongoing careful analysis of the relative needs of students, schools, and dis— tricts, the increase in categorical funding struc— tures, and the increase in categorical support as a percent of education funds, run the risk of Violating principles of educational equity. CRL'CIAI, lSSL'ES IN CALIFORNIA EDUCATION Educational Adequacy The state—increasingly standards driven, with assessments in place—is in an unusual position to realign funding systems with educational pri— orities that are outcome based. The central point is to align long-term educational priori— ties with funding designs. If there is an approach consistent with this alignment that has caught the attention of state policymakers nationwide in recent years, it is that of “educa— tional adequacy.” We take some time here to review the arguments in favor and against this approach, and consider how “adequacy” discus- sions might play out in California in the com— ing years}l Standards-lmsed reform strategies put adequacy, with respect to school finance, in the center of the discussion. Resolving this dis- cussion will require a complete restructuring of the existing school finance polices, structures and programs. Concepts of “educational adequacy” have existed for decades and have always played a role in not only understanding school finance policy, but also indirectly in litigating school finance policy. In part because early school finance cases that came before State Supreme Courts argued that Violations of equal protec— tion were at issue, equity principles took center stage in the deliberation and decisions. Increasingly over the past 10 years, and begin— ning in Kentucky in 198‘), school finance litiga— tion has become tied to conceptions of an “ade— quate” education, consistent with state constitu— tional guarantees. Adequacy clauses, in some form, guarantee students an adequate level of educational opportunity that is consistent with a 2.000 School Finance in California: Does History Provide a Sufficient Policy Standard? 1998—present ing capital costs of school construction. Adequacy—based lawsuits pending Dates Description States Involved 1970 — 1980 Court decisions gave attention to the ade- New Jersey, Washington, quacy and wealth neutrality of school West Virginia finance systems. 1989 Entire state education program found to be Kentucky unconstitutional 1989—1998 Courts ruled that their state constitutions’ Alabama, Massachusetts, education clauses guarantee students an New Hampshire, New adequate level of educational opportunities Jersey, New York, North that should allow them to achieve certain Carolina, Ohio, Tennessee, desired educational outcomes. Wyoming 1994 Adequacy case won by claimants concern- Arizona Louisiana, Minnesota, Pennsylvania, South Carolina Table 11. Adequacy Related Litigation by State22 specified set of desirable educational outcomes. Courts have written considerably in their deci— sions about what these desirable outcomes ought to be, which in and of itself has made the translation from judicial to legislative intent quite difficult. The table below shows the list of states that have been involved in educational adequacy litigation over the past decade. The policy design challenge is to identify, and articulate, those components of educational activities that constitute an “adequate” educa— tion. In general, the specification by State Supreme Courts on this point has been about educational outputs, leaving it to the legislature to determine what inputs yield an adequate education. There are a number of court cases that have been argued 0n adequacy grounds, and an increasing number of legislative reme— dies that havc used the courts’ language as a catalyst for the policy change. \Vhether driven by a response to litigation or policy initiative, the goal is to understand the relationship between how much we spend and a specific set of outcomes that constitute an “adequate” education. Understanding this rela- tionship has been a focus of researchers’ work for more than thirty years and continues to be difficult to articulate with specificity; as we all know, the relationship between what education dollars buy and what students learn is compli— cated by many factors including schools, fami— lies, communities, poverty, and their interac— tion. The central point is that discussions of “adequacy” demand an explicit conversation about setting a level—the level at which educa— tional inputs “yield” a set of outcomes that are PACE A 71 acceptable by some standard. But the con— founding factors of determining students’ par— ticular needs and their relationship to these specified outcomes makes the explicit conversa— tion extremely difficult. Challenges aside, most states, are now grap- pling more explicitly than ever with the construct of adequacy—even without any catalyst in the form of school finance litigation. In California, as well as the majority of states which have embraced standards-based reform, the articula— tion of what students need to know is now being faced head on by the question of ‘how much does that cost.’ As each state refines its capability to measure progress against state standards, the stakes of answering the question increase. Examples of Educational Adequacy fiom Four States Examples from four states illustrate the variation with which states have used a model of educa— tional adequacy in the formation of school finance policy. It will be a matter of time before experts can assess whether these new concep— tions have served public policy well, and whether the analytic framework has sufficient integrity to withstand time and political debate. The states that have adopted some conception of educa— tional adequacy in their policy discussions are working to understand the practical implications of the approach. In each state, clearly, the funda— mental questions of what constitutes “adequate” remain extremely difficult. This is not to say that agreement has not been reached, nor that it can be reached, by educators and policymakers who have invested in this approach. CRUCIAL Issuns IN CALIFORNIA EDUCATION Kentucky In 1989 the Kentucky Supreme Court declared the state’s entire system of public schools to be unconstitutional on the grounds that the sys- tem failed to afford all students equal access to adequate educational opportunities. Following the ruling, the Kentucky Legislature immedi- ately created what has been recognized as the most comprehensive experiment in educational reform—the Kentucky Education Reform Act of 1990, known as KERA. The vast changes embodied within KERA completely overhauled the state’s education system—linking educa— tional programs with school finance formulas, shifting to an outcome-based education struc- ture based on state education standards and coupled to a performance-based accountability and assessment system, increasing local level discretion in school governance and financial decisions, requiring pre-school for all at-risk youth, and establishing a new foundation pro- gram which increased the minimum basic state aid for schools. In the years following the court’s ruling, schools have scrambled to meet the demands of implementing KERA. Schools welcomed an increase in general purpose funding, but were challenged with implementing school site councils, adopting new curriculum standards and assessments, and adjusting to an account— ability system with rewards and sanctions. However, the experiences of Kentucky schools serve as valuable examples of the challenges met during the adoption and coordination ofa systemic school reform founded on conceptions of educational adequacy. ZOOO School Finance in California: Does History Provide a Sufficient Policy Standard? Wyoming In 1995 the Wyoming Supreme Court ordered the legislature to design an educational system that would provide all students with a “proper education”:3 The legislature responded to the ruling by defining a basket of education goods and services, consisting of a list of core courses and expected educational outcomes. After deliberation and agreement upon which com— ponents were to be included in the basket, the legislature, with the help of education consul— tants and a panel of \Vyoming education experts, estimated the educational expenses that would constitute a “proper education”, and cre— ated a cost—based block grant funding model. The new funding formula was designed around two factors: 1) A measure of students average daily membership; and 2) A model of average funding levels for three prototypical school models—elementary, middle, and high schools. The prototypical school models were designed around 25 principal cost components which account for specific instructional and operating costs deemed necessary to insure a “proper education.” The components were divided into the following five categories: 1) Personnel; 2) Supplies, materials, and equip— ment; 3) Special services; 4) Special student characteristics; and 5) Special school, district, and regional conditions.H Specific to each school model, the components were “costed- out” and a market cost was assigned to each. Revenue is distributed to districts according to the type of school models and the organization— al characteristics of schools Within each district. Ohio Ohio began the implementation of an educa— tional adequacy model in 1995 in response to school finance litigation. The model has been developed by the Ohio Department of Education with substantial consultation from education finance consultants. The conception of adequacy in Ohio begins with an affirmation that some number of schools in the state are performing “adequately,” within the resources they have available to them. Notably, the Ohio legislature approved a new funding plan that was designed around the principle of “adequa- cy,” but voters refused to approve the tax increase that was needed. The model, in its most recent form, uses the average per—pupil spending level in Ohio dis— tricts where the spending level was correlated with acceptable educational outcomes. The outcome measure is a set of criterion-refer— enced results from state assessments—the per— centage of students passing minimum compe— tency levels in particular areas of the curricu— lum. The model identified 102 out of 607 Ohio school districts whose students met 17 of 18 criteria. The districts were carefully chosen so as not to be outliers in their proportion of high and low property wealth, or spending level. Attendance rates of 93 percent or higher, and dropout rates of 3 percent or lower were also required to keep school districts in the sample. Using school districts that met all of the crite— ria, analysts constructed a weighted mean per- pupil revenue from district expenditure pat— terns. This resulted in a per—pupil amount and became the definition of “adequacy” before adjustments for costs that were beyond the control of the district.35 Oregon The state of Oregon has developed an educa— tional adequacy model that ties financing to PACE I 73 74 students making educational achievements that are competency based. The Oregon Quality Education Model is based on the premise that the school should be the unit of analysis, although state funding is still to be distributed on a per—pupil basis from the state. The history of the design can be tied to interest by Oregon’s top policymakers to understand the best return on investment, and the implications for rationing public funding for education, as in health care. The implementation required the development of a sophisticated cost—based data collection and accounting system, implemented systematically across the state?“ The model is described as a work in progress, but will likely be used during budgeting discussions in the upcoming legislative session. Further, the model is not intended to be an allocation or distributional model, but rather a tool to allow policymakers to understand and deliberate the total education budget. The model assigns detailed cost estimates to the primary elements that make up the instruc- tional program of a school: teaching staff, administrative support, and supplies. The instructional program is determined to be of a sufficient quality when it contains all of the components that have been deemed necessary through extensive consultation within the state: specific academic content, performance stan— dards, specific developmental goals, class size constraints, professional development opportu— nities, the duration of instruction, and opera— tional support. The costs of “prototype” schools have been developed within these con— straints, and then adjusted for socioeconomic factors, location, age of building, and other fac- tors that establish the schools’ context. To the extent possible, each of these adjustments was CRUCIAL ISSUES IN CALIFORNIA EDUCATION made using the best available data from numer— ous sources throughout the state. The promises of the approach aside, the National Research Council’s Committee on Education Finance concluded in late-1999 the following regarding educational adequacy. These caveats seem particularly appropriate to the context of California: Deliberations in courthouses and Statehous- es suggest that a desire to implement ade- quacy may be outpacing current understand- ing of how to define and achieve it. In com- parison to wealth neutrality and equal spending, equity defined as adequacy requires difficult value choices, as well as policy decisions in areas in which the avail- able technical knowledge is weak. Implementing adequacy requires establish- ing anchors for identifying what is adequate; determining the costs of an adequate instructional delivery system; making adjust- ments for student, school, and geographic characteristics; adjusting for inflation from year to year; and developing an assessment system for measuring whether adequacy has been achieved.Z7 The cautions that are raised here are particular— ly applicable in California where measurement and technical questions, within the context of state coherence, have bedeviled policymakers; the implementation of a stable assessment sys- tem has taken decades, and time will tell whether the STAR system is in place for any length of time to make systematic statements about changes in student performance possible. With these caveats noted, California must begin the difficult data collection and delibera— tions that are needed to begin understanding the costs of the education system it has 2000 School Finance in California: Does History Provide a Sufficient Policy Standard? designed— developing new analytic methods to assess the relationships between funding and educational outcomes. As noted, discussions of “adequacy” demand an explicit conversation about setting a level—the level at which educational inputs “yield” a set of outcomes that are acceptable by some standard. We want to express caution that in a state that has seen such limited funding growth as California, “adequacy” discussions are not likely to occur without substantial pres— sure for marginal funding changes, and those marginal changes are not necessarily going to be “adequate” funding changes. These pres- sures should be resisted. Said another way, given the inherent complications with using educational “adequacy” as an analytic construct in the first place, the temptation of policymak— ers may be to simplify the process of determin— ing “adequacy” by simply addng revenues to the K—12 system. While this solution might add revenues, albeit on the margin, it would not have been consistent with the process that determining educational adequacy requires—careful thinking about the differential funding Obligations for students with special needs, for students in disadvantaged communi— ties, for students with exceptional skills, and for students who require a multitude of specialized services from schools. These are the difficult conversations that policymakers in California must have. California can learn from—and con— tribute to—thinking on educational adequacy, even though the field is developing and the barriers to the approach, being applied in a pure form, are still substantial. Governance Absent a rationale for the separation of deci- sion-making responsibilities around school spending, the school finance system itself can- not be rational. The education funding system needs to be explicitly connected to the gover— nance structures that are accountable for pro- viding services to districts, schools, and stu— dents. These include the state, but also include county, district, and school—site structures. As has been described earlier, the converging events of the 19705 and 19805 resulted in a school finance system that is indisputably cen— trally controlled. As has been noted by the Legislative Analyst’s Office on several occasions in recent years, the control of education funding at the state level has also been associated with the control of education more generally, at the state level. In recent years, the development of curriculum standards, the implementation of the STAR system, and the wide variety of account— ability mechanisms would suggest that the state’s priorities are more tightly aligned than they have been historically. Of course California is not alone in this design strategy, and it is increasingly hard to find a state that would not describe itself as driven by state—standards. The policy problem that remains is to main— tain central control over the collection and dis— tribution of funds, but allow for a rationale that systematically addresses equity concerns and the educational outcomes that have been identified at the state— and local—level. This does not nec— essarily mean a greater or lesser emphasis on categorical funding, but rather a coherence to the funding design that includes revenues PACE I 75 attached rationally to legitimately determined local needs. This point is given emphasis not to retrace the past thirty years, but to think criti- cally about the next thirty years. In the years ahead, a coherent education finance policy design will have to address both financing and equalization targets, and do so within a gover- nance structure that allows for the connection to be consistently maintained and adapted. Grubb and Huerta make a strong case that the education finance system has become ever—distant from discussions about effective— ness in teaching and learning, and suggests that decisions much closer to the school and classroom about the best use of revenues is well overdue.” This is but one of many rigor— ous arguments that have been made, in some way, about site—level decision-making, includ- ing but not limited to funding control. To be sure, an argument has been made over more than twenty years about the reasons for these site decisions, although volumes of writing on site—based management have been largely inconclusive on the connections between site—level decisions and student outcomes.” What we can say is that those who have stud- ied school sites carefully have argued that funding control is a necessary and essential step for devolution of control—an enabling condition that allows for educational changes to occur that are consistent with improved educational effectiveness. Conclusion California, as a state, has developed increasingly clear goals for K-12 education as exemplified by state standards. Absent from this progress has been the discussion of how financial resources CRUCIAL ISSUES IN CALIFORNIA EDUCATION should support these goals. The connection that must be made is to articulate the programs and strategies that can be used to meet the state standards, and to identify their costs. The school finance agenda for the begin— ning of a new century is to work through the many difficult issues that have been raised in this chapter: equitable distribution of fiscal and non-fiscal resources; measurement of the costs of providing an adequate education to students of highly varied needs; consideration of who is best positioned to make decisions about pro- grams and strategies that are consistent with reaching state educational standards; and design of policy coherence between the finance and accountability systems. To make progress on educational outcomes in California, policymak- ers need to provide leadership by opening dis- cussions on these main points. We have indicated that the history of cate- gorical funding is as much about displaced goals on the part of the state as it is about ten- sions in who ought to decide how funding is allocated at the school site. Complicating the history are policy constraints involving equity, tax burden, and the mechanisms that have been set up over time to protect education within a centralized funding structure. Nowhere in this analysis have we argued that categorical aid structures are necessarily bad policy; to the contrary, it is quite possible to imagine categor- ical funding systems that make sense in a clear- ly articulated rationale about which students need particular services, and how those services are linked and structured together into coher— ent educational visions—visions that are consis— tent with state objectives, and with the best expertise of educators. ZOOO School Finance in California: Does History Provide a Sufficient Policy Standard? Notes 1. R.L Manwaring and SM. Sheffrin 1997, Litigation, School Finance Reform, and Aggregate Educational Spending, International Tax and Public Finance 4(2):107—127; T.A. Downes and M.P. Shah, The Effect of School Finance Reform On the Level and Growth of Per—Pupil Expenditures (Unpublished paper, Department of Economics, Tufts University 1995). 2. Helen F. Ladd and Janet S. Hansen, eds. Making Money Matter: Financing America’s Schools, National Research Council, (Washington, DC: \lational Academy Press, 1999). 3. See J. Coons, Clune and Sugarman, Private \Vealth and Public Education (Cambridge, MA: Harvard University Press, 1970). 4. At the extreme, a differential within the accepted equity band is $670 per pupil. This difference, multiplied by an average class size of 30 pupils, results in a $20,100 difference in funding per classroom. 5. E. Hill, “Equalizing School District Funding: Option for a Sliding Scale COLA,” Legislative Analysts Office Report (Sacramento CA: Legislative Analysts Office, 27 April 1999). 6. Districts are allowed to raise additional taxes for school bonds. This requires a two—thirds majority of the voters. Parcel taxation, for school programming is allowed under the same rules. 7. See Paul Goldfinger, “A History of Revenue Limits: Or, Why is Your Base Revenue Limit Bigger than Mine,” Testimony to Assembly Education Committee on School Finance (Sacramento, CA: 23 November 1999). 8. See EdSource, “How California Compares: Indicators and Implications for Our Public Schools,” (Menlo Park, CA: EdSource, November 1998), http://w\nv.edsource,ors/htmfiles/lrn_facts.htm 9. School construction and maintenance is funded in accordance to School Facility Program (SBSO), which provides state matching funds to local efforts—50/50 for new construction, 80/20 for modernization of existing facilities, and 100% of costs for new construction or modernization in “hardship cases.” 10. See California Department of Education. “School Facilities Fingertip Facts.” School Facilities Planning Division, (Sacramento, CA: California Department of Education, Januaiy1999). http://xnnvcdeca.gov/dmsbranCh/sfsdiv/docs/facts.html 11. See Edsource, “Election Brief: Proposition 26, the Ailajority Rule Act,” (Menlo Park, CA: EdSource, January 2000). See also Legislative Analyst Office, “Proposition 26: School Facilities, Local Majority Vote, Bonds, Taxes,” (Sacramento, CA: Legislative Analyst Office, November 1999), http://www.lao.ca.gov/initiatives/2000/26_03_2000.html 12. The Legislative Analyst Office estimates that since 1988 local level discretionary funding has decreased from a high of nearly 78% of general budget funds, to a low of 68% in 1999. Many discretionary dollars have been replaced with restricted categorical program funds. See Legislative Analyst Office, “2000—01 Budget Analysis and Perspectives & Issues” (Sacramento, CA, Legislative Analyst Office, January, 2000). http://\nvw.lao.ca.gov/analysis_2000/analysisi2000_contents.html 13. The requirement of a two—thirds vote to pass general obligation bonds has existed since 1879, as written in the state’s original Constitution. 14. The state provides “hardship funds” to districts that display a financial need. However, a district must meet spe— cific criteria, including an outstanding bonded indebtedness rate of at least 30 0/o of bonding capacity or certification that the district has repeatedly failed to pass a school bond measure. Proposition 1A which passed in 1998, allocated only 15% ($1 billion) of the $6.7 billion reserved for K—12 education for hardship funds. By January 2000, 78% ($388 million) of the initial $500 million authorization of hardship funds, had already been disbursed to school districts. The remaining $500 million will be disbursed during the next authorization beginning Jilly 1, 2000. See California Department of Education, “School Facility Program (SB 50),” Office of Public School Construction. (Sacramento, CA, California Department of Education, 2000). PACE I 77 78 15. EdSource, “California’s K—12 School Finance System,” (Menlo Park, CA: EdSource,1995). 16. For the purposes of this paper, categorical programs include all state programs except school district revenue limit allocations. Categoricals are either targeted on a class of districts, schools, or pupils, or have restrictions on how the funds may he spent, or hoth. Funds for adult education and state bond funds have not heen included in this analysis. As explained later in the text, reconstructing the total picture of categorical allocations would be extremely costly, and it is recognized that a number ofsmall categoricals have been omitted from the table or their funding is included in other cat- egoricals. Best efforts have heen used to reconstruct the history of categorical funding, hut these data would likely not withstand a financial audit. 17. This is likely to he an underestimate for several reasons: (1) some small, ohscure programs cannot he located in hudget documents; (2) categorical programs supported by Funds other than the General Fund have not been included (for example, The Petroleum Violation Escrow Account was the funding source for school bus replacements in excess of $50 million); (3) school programs administered by agencies other than the Department of Education have not been included (such as the Department of Alcohol and Drug Abuse Prevention); and (4) a number of programs have numerous sub—programs (for example, Child Care is composed of more than ten identifiable programs, and Economic Impact Aid has bilingual and compensatory components). 18. US. General Accounting Office “State l‘ifforts to Equalize Funding Between Wealthy and Poor School Districts,” (“lashingtom DC. GAO/l1ElIS—98—92, 1998). 19. An equity index commonly used by school finance researchers— the coefficient of variation — was calculated for all states by researchers for l‘iducation \Veek in developing Quality Counts 2000. California got a grade of “C”, along with 17 other states. The relative inequity in spending per student among districts in California is 11.4"“. 20. See extensive analysis from the National Research Council on this point. Making Money Matter, 250—3. 21. A thorough discussion of educational adequacy, from hoth historical perspectives and recent implementation experiences can he ‘ound in a recent volume written by the National Research Council: See llelen Ladd, Rosemary Chalk, and janet S. llansen, eds, Equity and Adequacy in Education Finance (\Wshington, DC, National Academy Press, 1999). Ix.) Ix) . Adapted from Making Money Matter, p. 107. 23. See Guthrie and Rothstein in Equity and Adequacy in Education Finance, (National Academy Press 1999). 2-1. See A'1anagement Analysis and Planning Associates, +1. I 4 'J1 See Making Money Matter, 119. 26. A description of the Oregon Education Model, and the data collection infrastructure can he seen on the web at http://dl)i.ode.state.or.us/qualityed/ 27. Making Money Matter: Ii‘inancing America’s Schools, 1 12. 28. See Norton Gruhh, N. and Luis lluerta, “Straw into Gold, Resources, and Results: Spinning out the Implications of the “New” School Finance,” (Berkeley, CA: PACE, School of l‘iducation, University of California, Berkeley, forthcoming). 29. A. Summers and AAK'. Johnson, The effects of school—based management plans, in Improving America‘s Schools: The Role of Incentives, eds. E. llanushck and D jorgenson (\Vashington, DC: National Academy Press, 1996). CRUCIAL lSSUES IN CALIFoRNiA EDUCA'rIoN 2000 Chapter 5 Governance and Accountabil Michael W. Kirst, Gerald C. Hayward and Bruce Fuller PACE A Drift Toward State Control of Educafion Although many countries have established nationally regulated public school systems, the United States has always emphasized local control of education—leaving most questions of what and how children are taught to the dis— cretion of 15,000 local school districts. During the last two decades, however, policy—setting power in California public education has become increasingly centralized at the state— level, and this shift of authority over academic policy has become a major concern for local school officials. The perennial question of how we raise and teach our children continues to energize—and sometimes divide—our political leaders. \Vith over 300,000 employees of schools, K—IZ edu— cation consumes the largest percentage of the California state budget. Therefore, it is easy to see why state politicians don’t want to leave educational issues to local authorities. Seats on the state legislative education committees rank only below those of appropriations and rev— enue committees in terms of status and desir— ability. Indeed, California considers hundreds of education bills each year, so the discretion ity of local officials in school policymaking shrinks as state education codes and regulations grow incrementally. There has been an explosion in the number of regulations affecting schools that has result— ed in a 4,000—page California State Education Code, including court interpretations. For example, state and federal courts, which in the past had little to do with shaping school pro— grams, today set detailed priorities about issues ranging from student rights to special educa— tion. i‘Vlcanwhile, on the federal level, Congress provides funds for everything from schools in low-income areas to education for limited— English pupils—but, inevitably, with strings attached. At the same time, California limits the policy prerogatives of local governments through school finance reform, accountability, and regulations. Several initiatives including Propositions 13 (property taxes) and 227 (bilin— gual education) constrain local school board authority, and Chapter 4 provides a recent his— tory of categorical program controls. Still, as this chapter demonstrates, gover— nance of California’s schools is far more com— plex and problematic than any simple picture of the state gaining the upper hand over local school districts. If the state has gained more PACE fl 79 power so have, quite paradoxically, many indi— vidual schools which increasingly make many of their own hiring, budgetary, and pedagogical decisions. And local school districts, while they are comparative losers, still wield a lot of influ- ence they are not likely to surrender. This chapter begins by looking at the histor- ical and political context of school governance in California, followed by a close analysis of where we are now in terms of shifts in roles and power on the part of various education entities. We devote the concluding section to an analysis of how the Davis administration has further consolidated power in Sacramento, and at the various implications that this may have for California education over the long haul. The Historical and Political Context of School Governance in California Under the US. Constitution, education is a power reserved to the states. The basis for state control over education was well established as early as 1820 by constitutional and statutory provisions. The operation of most schools was delegated by states to local school boards. The specifics of K-12 public education governance systems vary by state. Notwithstanding such variances, though, it is possible to list the pri— mary components of California’s K-12 public education governance system. Historically, California has controlled local education through several means. The state establishes minimums below which different kinds of local school operations cannot fall. The rationale behind such regulations is that the state’s general welfare requires a basic educa- tional opportunity for all children, and thus pupils may be required to attend school a Inini— mum number of days each year, or their instruc- tion must include certain courses, taught in cer— tain ways, with teachers who have a particular kind of training. These early California require— ments were originally intended to raise stan— dards in rural schools. In order to conduct this minimum program, California requires school districts to levy a minimum property tax and guarantees a certain level of expenditure, known as the revenue limit. California has also speci— fied procedures for the reorganization of school districts. For example, around 1900, California abolished the decentralized ward—based city State Regional/County School District School Others Governor Regional/County Local Board Principals Mayors Board Legislature Regional/County Local Teachers Judges Superintendent Superintendent State Board Regional/County Local Department Parents Unions Department State Local School Business Superintendent Council Leaders State Community Department Leaders Table 1. Primary Components of the State K-12 Public Education Governance System CRUCIAL Issuas [N CALIFORNIA EDUCATION 2000 school boards. In the 1950s, California began to require consolidation of school districts and eliminated so many that the number shrank to the one thousand we have today. Only during the past 40 years has the state emerged as the primary policy and fiscal agent in the delivery of educational services to California’s school children. Court decisions and legislative prescriptions regarding the equalization of school funding, and the loss of local property tax discretion through Proposition 13, contributed heavily to this transformation. The state’s own capacity to act forcefully expanded as well. The number of legislative staff increased, enlarging the legisla— tive, oversight, and research capabilities. Similarly, federal educational programs required the state department of education to approve local applications for federal funding and provided federal dollars for expanding state administration purposes. During the same period, increasing turbu— lence locally (school violence, desegregation, taxpayer revolts, and the like), coupled with declining test scores, eroded the public’s confi— dence in local officials and professional educa— tors. State testing and minimum proficiencies for students and staff followed. Omnibus legis— lation such as Senate Bill B813 in 1985 includ— ed a required core curriculum, and categorical programs further chipped away at the discre— tion of local governing boards and superinten— dents to establish a local agenda. Alignment of state tests, texts, and curriculum guides increased state influence over what was taught. During the 19805, education came to be seen as centrally and crucially important to California’s ability to remain competitive eco— nomically, and to train a diversifying workforce Governance and Accountability to succeed in an increasingly technological labor market. The state’s interest in educational productivity and economic development became intertwined. From the state’s perspec- tive, the need to secure a competitive economic capability overshadowed its historical faith in local control. In short, local turbulence, public distrust of local officials, new state capacity to intervene, and a belief that higher and uniform educational standards served the state’s overall interests compelled state officials to assert con- trol it long ago ceded to local school districts. In many respects, California schools now con— stitute a state system that is operated locally. The state controls approximately 84 percent of school funding and uses a four-volume Education Code for regulation. The instru— ments of state educational governance include the governor, legislature, chief state school offi— cer, State Board of Education, State Department of Education, and several other state agencies. Secondly, the intensity and scope of state policy actions, like California’s omnibus reform legislation, Senate Bill 813, and subsequent administrative initiatives, have shifted the bal— ance of control away from local districts and toward the state capital. Local teacher bargain— ing contracts centralized decision authority within districts, but also dispersed authority to legislatures, courts, and public administrative agencies, like the California Public Employment Relations Board. In short, state policy and ballot initiatives from the top have encroached upon the authority of local decision makers, squeezing the “discretionary zone” of their activity into an ever smaller area. At the same time, increasing demands from special interest groups such as those representing PACE I 81 82 handicapped and gifted children have dimin— ished the ability of governing boards and super— intendents to set a district’s agenda. School board members and superintendents now more often react to other forces (changing coalitions, for example), and they do so with less public confidence. Increased state control is just one example of how local school boards are no longer the pre— eminent governance institution. Another exam— ple is the fact that teacher preparation and pre— school programs do not fall under local school board supervision. In addition, the impact of collective bargaining has reduced school boards’ discretionaiy powers in many jurisdictions. The legacy of changes over the past 40 years makes it hard to tell who, if anyone, is in charge of California public schools. One certainty, howev— er, is that local decision makers are less in con— trol. The focus of state education policy in the 19705 was primarily on programs earmarked for handicapped, disadvantaged, or other special- needs pupils. But in the 1980s, state education policy shifted to mandating educational prac- tices in those localities concerning what is taught, how it should be taught, who should teach it, and how results should be measured. California requires the provision of services for certain things, such as education for the handicapped. Indeed, a major argument for state control is that it can ensure equality and standardization of instruction and resources. Local control advocates, however, assert that local flexibility is desirable because the technol— ogy of teaching is so unclear and local needs are so diverse. In essence, the argument over local control focuses on the trade-off between two values—equal (and adequate) treatment and freedom of local choice. California has CRUCIAL ISSUES IN CALIFORNIA EDUCATION tried to bring about greater efficiency and accountability in local education through devices such as required state testing and pay- ments for increased test scores. The local con- trol advocates counter that educators in general and states in particular lack the knowledge to specify the most efficient educational methods. Consequently, they insist that local options are more desirable. It is certainly true that school boards pro- vide a critical local perspective to policymaking. In a country committed to representative democracy, school boards still provide citizen access that remote state and federal capitals cannot duplicate. At the same time, the public wants much more non-local intervention when it comes to educational standards and account- ability, programs for special-needs pupils, and many other areas. Consequently, Governor Davis featured a new high school graduation exam and a test—driven accountability system as his top priorities in 1999. The Impact of State Reform on Local School Policy PACE studies have illuminated the increasingly complex web of education governance, which has manifested itself in part in the growing influence of individuals and groups external to the local school board and central office. The pattern over the last decade has been for state authorities (legislators, governors, and chief state school officers) to increase their influence over funding and policy from the top. Simultaneously, employee unions, parents, interest groups, and private agencies (testing and accrediting) have squeezed the discre— ZOOO Governance and Accountability Year K-12 Proposition 98 Revenue Limits 1990—91 $18.6 1993-94 $21 .2 1996-97 $26.8 1999-00 $33.8 Percent Discretionary $15.5 83.4% $15.9 75.0% $19.6 73.1% $23.0 68.0% Table 2. K-12 Proposition 98 Discretionary Spending Share (Current Dollars in Billions) Source: Legislative Analyst’s Office Budget Analysis, 2000—01, E68 tionary zone of school board control from the bottom. Local central school authorities are sandwiched in by these contending forces. Clearly the zone of policy discretion at the school district level has grown smaller over the past 30 years. The expansion and shift in state education strategies in the 1990s did not include a rethinking of the local school boards role, nor did it envision the board as a leader for implementing change. However, a review of the impact of intensified state activity shows that although school boards were not initiators of many new state policies, they were often quick to implement state policies that could be easily accommodated such as increased gradua— tion courses. The changing governance picture is more complex than a zero—sum model where one level expands its influence (e.g., state) at the expense of the lower (local) level. iMany California state policies, for example, leave considerable room for flexibility and enhance local initiative. Education policy, then, involves mutual influence among education policy lev— 615, not zero—sum. $01116 state mandates—cg, requiring a semester of economics are strongly directive of local behavior. But man— dates and regulations have not been the main strategy through which the state guides and influences local curricular content. California curricular frameworks and content standards in science and social studies, for example, are not mandates; they provide a framework rather than a prescription for what must be taught. Consequently, many local districts use the state curricular framework as a springboard by which they develop their own local approaches to instruction. Furthermore, state policy is characterized by low enforcement and impre— cise policy directives, both of which increase local autonomy. Nevertheless, despite this autonomy, many local districts not only com— plied with California’s 1983 reform law (SB 813) but built on the state—based mandates to add new policies of their own . In their study of six states (including California), researchers from the Consortium for Policy Research in Education concluded: Local activism in refortn has been noted in several studies of the reform movement... This local activism takes a variety of forms: staying ahead of the state and of peers by enacting policies in anticipation of higher state policies to meet specific needs, and using state policies as a catalyst for achieving district objectives. The clearest current manifestation of local activism is the cur— riculum alignment and standardization movement underway in many districts.‘ These newer studies of the impact of state reforms indicate that both state and local control can increase as a result of state policymaking. PACE I 83 84 Finally, school boards and local adminis— tration must play a key role in making sense of a multitude of often confusing state initia- tives. Local educators need to ensure that various components of state reform are coherently linked. Centralization and Decentralization: Divergent Trends An offsetting trend to state policy centraliza— tion is the increasing operational authority of individual schools. The operational details of school—site devolution vary a great deal. In some models, the only ceding of central office discretion is over trivial matters such as the spending of budgeted funds for instructional supplies. This typically amounts to only a small proportion of overall school money, even when per pupil allocations are aggregated for all stu— dents in a large secondary school. At the oppo- site extreme are approaches where teachers, as a collective, make decisions regarding employ- ment and retention of new staff, allocate bud— gets, determine discipline policy, and control the daily school schedule. In between are varia- tions where the authority is allocated to princi— pals or heads but not to teachers, or hiring dis— cretion is allocated to a school but selection can take place only from a pool of recruits com— piled by central, district-wide authorities. School—site discretion appears at first glance to be paradoxical. In the face of growing state centralization of school decision-making, why would greater operating authority be ceded to school sites? The frequent justification is that, whereas it is necessary for central authorities to CRUCIAL ISSUES IN CALIFORNIA EDUCATION specify the what of schooling, it is not appropri- ate, or even sensible, for them to specify the bow. Another common rationale for decentraliza- tion is that teachers are “professionals.” State specification of their instructional behavior is therefore seen as demeaning. Presumably teachers know what their students need and are trained to meet those needs. Not enough is known about instruction to take the risk of specifying teaching behavior centrally. Under conditions of technical uncertainty, better to permit a “thousand flowers to bloom.” California’s School Choice Movement A review of education governance would be incomplete without consideration of California’s robust movement to broaden parental choice and diversify the kinds of schools available to children. This accelerating push to introduce market competition and directly empower parents represents a second form of accountability, apart from that involv- ing centralized mechanisms such as student testing and sanctions overseen by Sacramento. Under choice, the idea is that school staffs are directly accountable to parents and community values, not to Sacramento. Some alternative institutions—like publicly funded charter schools—are theoretically accountable to both. PACE’S study of school choice in California and nationwide, published in 1999, details what we know about the movement and its funda— mental effects on public and private schools. One new discovery: one—fourth of all students no longer attend their neighborhood school. 2000 Instead, they attend a magnet or charter school, participate in a voucher experiment or a much larger cross-town transfer program.l California remains a national leader in terms of creating new charter schools (289 now oper- ate statewide, not including three all—charter districts), authorizing open enrollment options, and hosting corporate—financed voucher experi— ments in Los Angeles, Oakland, and San Francisco. Table 3 reports on the scope of choice enrollments statewide and in selected urban school districts. Despite the steady growth and diversifica— tion of the choice movement, we know very lit— tle about whether this second form of (market) accountability yields significant effects. The recent PACE study did uncover consistent evi— dence of parental satisfaction with their chosen magnet 0r charter school. But hard evidence of achievement gains, relative to garden—variety public schools, remains scarce. Governance and Accountability Governance at the State Level: K-16 Gaps Setting aside local boards for a moment, California has numerous institutions that have a role in state governance and standards: ° Regents of the University of California ° Trustees of California State University ° Board of Governors of the California Community Colleges 0 State Board of Education ° California Post-secondary Education Commissions ° California Department of Social Services ° California Economic Development Department ° Governor’s Secretary of Education ' Superintendent of Public Instruction ° Assembly and State Legislative Committees ° State Job Training Coordinating Council California created an Education Roundtable in 1981 that focuses on issues that span lower and Program Student Enrollment Magnet 207,893 Charter Schools 37,436 Vouchers 4,433 Open Enrollment 238,598 Private Schools 615,011 Other 165,380 TOTAL 1,026,403* Table 3: California Students Annually Participating in School Choice (1997-99) Sources are detailed in Fuller at al. (1999) * Figure does not include open enrollment count because of overlap with other choice options. Total amount may be significantly larger. PACE I 85 86 higher education. Roundtable membership consists of the UC president, the CSU and California Community College Chancellors, the Superintendent of Public Instruction, the Director of the Post—secondary Education Commission, and the Chairman of the Association of Independent California Colleges and Universities. But the Roundtable has made only partial progress in aligning K—16 educa- tion assessment standards. As Chapter 8 demonstrates, these K—16 dis— junctures will be hard to fix unless there is an institutional center for California K—16 reform. There are few regular opportunities for K-12 educators to discuss standards issues with col— lege and university faculty or policymakers. The professional lives of K-12 and higher edu— cation proceed in separate orbits. K-12 State Governance Structure and Locus of Control California’s recent struggle in carrying out a continuous and coherent education reform agenda can be attributed partly to a fractured governance structure, and the partisan conflicts and alliances that arise with each election. The state superintendent of public instruction, who is elected, is also the executive officer of the state board, but must get approval for any cur— riculum-related policy from the state board. The governor appoints the state board. The governor, meanwhile, has authority over the budget, except when otherwise directed by the legislature. The state board has no day-to-day control over the actions of the CDE but approves policy drafted and carried out by the CDE. The legislature, meanwhile, can mandate CRL'CIAI. ISSL'ES l.\‘ CALIFORNIA EDUCATION policies, but has traditionally charged the state superintendent with carrying out and regulat- ing key education laws. In short, no single entity or individual at the state has the authority to set the course for edu- cation reform, carry it out, and alter its course when something goes wrong. The lack of a sin- gle entity or individual at the state level who is accountable for education, combined with the disruptive partisan maneuvering, has led some to suggest that the only solution is to eliminate an elected position for the state superintendent. Such a change would vest ownership for shap— ing and carrying out an education agenda firmly in the hands of the governor. Several state offi- cials, from both ends of the political spectrum, suggested that the governor appoint the super- intendent. As one legislator put it: “No one’s ever heard of electing the US. secretary of edu— cation.” A similar recommendation was made in 1996 by the legislatively created Constitutional Revision Commission, enacted to examine statewide governance and fiscal structures, but legislative proposals to make the change have not gone very far. Declining Role of the Department of Education The CDE, or more precisely—the state super- intendent of public instruction—is charged by the legislature with regulating and implement— ing the state—funded education programs. However, for the most part, the superinten— dent’s position is that of a bully pulpit—shaping a compelling message, rallying forces, and keep— ing critics at bay. Many agree that Bill Honig. Superintendent of Instruction from 1083 to 2000 1992, knew how to capitalize what little policy— making leverage his position offered because he understood all the pieces of the larger system in which he operated and how these pieces inter— acted. As one person explained: “CDE was not his legacy, so he exported his initiatives.” Accordingly, Honig recognized that genuine reform would have to be fostered from the bot— tom-up; therefore, he supported the expansion of an infrastructure of professional development networks, run by higher education institutions and led by teachers. To further his curriculum objectives, Honig made structural changes to the CDE, reorga- nizing staff units around subject matter areas. Under his predecessor, ‘Wilson Riles, the CDE was organized by special needs students and categorical programs aimed at serving such populations, (e.g., Title I, Migrant Education, Special Education). CDE involvement in cur— ricular matters during that time was purpose— fully minimal. One CDE employee from that time recalled Riles saying, “Our job is to get money to the schools and to leave curriculum to the locals.” In the 805, however, curriculum became the centerpiece of the CDE’s systemic reform strategies. To build further support and provide professional development opportunities around state curriculum policies, the CDE worked col— laboratively with cross—sections of state and local curriculum specialists and teacher—leaders in the development of the frameworks. During this period, the CDE was viewed as an expert institution with the professional capacity for leading such state—level curriculum efforts. Governance and Accountability With rare exception, the state board approved curriculum policies put forward to the board by the CDE. In fact, under Honig’s reign, the state board agenda and decision-making docket was largely determined by the CDE staff and the board often deferred to their policy recom- mendations. But by the early 90s, the CDE’s glory days were quickly coming to an end. Both the board and the governor became disenchanted with Honig and critical of the CDE’s policy—making role. Honig’s battles with State Board of Education President Joe Carrabino eventually led to a law suit filed by the board challenging the CDE’s policy authority and allegations about Honig’s fiscal improprieties. These issues distracted him, say observers, at a critical junc— ture for the CDE, that is, just at the time when the CDE was beginning to develop CLAS. The Eastin Em Delaine Eastin, the chair of the Assembly Education Committee, was elected state super— intendent of public instruction in 1994 (and defeated Maureen DiMarco, Governor VVilson’s secretary for education and child development). A diminishing operating budget has worked against Eastin in the last six years; the CDE has lost nearly 50 percent of its staff, leaving the CDE, until very recently, with just one mathematics and science specialist. Both Governor Wilson and Davis cut the CDE bud— get significantly, and “with those kinds of cuts, you don’t always do them strategically,” remarked one CDE official. Not only does Eastin have less resources with which to work, PACE 3 87 but she has less fiscal autonomy then Honig, and must now pass budget items through the State Board of Education. Rising Role of the State Board Throughout most of the 1980s, the board’s packed docket of decision items meant more time was spent making administrative decisions, such as focusing on the approval of waivers, than on engaging in serious, substantive discussions about complex policy issues. When such longer discussions did occur, they were usually crisis- driven (e.g., creation science in state— adopted textbooks). For the most part, the understaffed board also was dependent on the CDE for administrative support. The CDE had much more free reign and authority during this time. As one staff member recalled: “In the old days, Honig would just issue an advisory, but now you have to run everything through the board.” By 1990, however, the state board became more proactive as tensions between the gover— nor, the board, and Honig grew. Some of this tension, say observers, was due to personality conflicts, while some of it was due to a lack of clarity in the law about the delegation of pow— ers among agencies. In September of that year, Joe Carrabino, the president of the board, complained to the press that the superinten— dent treated the state board like an “advisory board” and escalated the feud by demanding greater oversight over the CDE’s budget and review of all policy directives, referred to as “underground regulations.” Honig refused to comply with the request on state constitutional and other legal grounds. In response, the board decided to sue. Since the (:RL'CIAL Nuts 15; CALIFORNIA liDL'CA'riox board was technically staffed by and considered a part of the CDE, the board was supposed to use the same general counsel as the defense, State Superintendent Honig. The state attorney general, however, agreed the board could seek outside counsel. As a result of that suit, the 3rd Appellate Court in 1993 ruled in favor of grant— ing the board more policy—making authority than the state superintendent wanted. In recent years, the board has assumed a more active role in influencing legislative pro— posals. State board members have testified on several pending legislative proposals. In the aftermath of the 1994 election, the board also became part of a powerful triumvirate that included Wilson, a Republican governor, the Republican-controlled Assembly, and the Republican state board, whose members were appointed by the governor. Further strengthening the board’s position was the approval of an additional $2 50,000 added to its budget to expand the board’s staff of one professional staff member to five, including an independent legal counsel. Increasing Legislative Intervention Throughout most of the 805 and early 905, the legislature was known for passing volumes of often disparate, piecemeal education laws. Term limits also have added to the political ten— sions and contributed to a loss of institutional memory about why prior reforms worked or did not work. The net effect has been highly variable reform efforts that have little chance of being comprehensive and cohesive. Nevertheless, starting in 1995, legislators harnessed existing state and federal categorical ZOOO resources around literacy, and improved the conditions for literacy by lowering class size in K-3 grades from an average of 30 to 20. Not only were these initiatives unprecedented in terms of the consensus they represented among an otherwise divisive body, but they also indicated an unusual level of intervention and top—down control by state—elected officials into the affairs of curriculum policy, particular— ly reading. Traditionally, reading policy has been left in the hands of the state board, with even more discretion given to local districts and schools on how to implement what stu— dents should know and be able to do on a day-to—day basis. Part of this new legislative intervention into state curriculum may stem from the ongoing tension between legislators and the civil ser- vants who are Charged with turning words on paper into action. As one legislator explained, he and his colleagues have no choice but to hand over a policy to the superintendent and his or her staff; they are, after all, considered the “technicians.” Yet, at the same time, he explained, there is always the risk that CDE civil servants may go too far and “extrapolate beyond the law.” The Governor’s Office Becomes Preeminent The governor has emerged as the most power— ful force in determining education policy. Prior to the 1994 election, most of Governor VVilson’s education initiatives, including a series of voucher proposals (called “ opportunity scholarships”) and calls to streamline the 11 volumes of the state’s education code, did not Governance and Accountability go very far. The governor was successful, how- ever, in promoting the passage of Proposition 184 in 1994, an initiative aimed at curtailing publicly funded services for illegal aliens, including educational services, although courts have overturned most of its provisions. In 1995, Wilson, under pressure from Assemblyman Steve Baldwin and other conservatives in the Assembly, threatened to return federal funds for Goals 2000, School—to—VVork, and other programs because they were considered too restrictive and impinged upon local control. But Wilson used class—size reduction as his key education policy in 1997 and hoped a student—to—teacher ratio of 20-1 would signifi- cantly improve the conditions that contribute to higher reading performance. State Governance Under Governor Davis In 1999, the first year of the Gray Davis administration, the power of the governor in education policy soared to unprecedented heights. California has always been recognized as a strong governor state, with the governor having substantial power, especially with the line—item veto. Coupling this traditional power with his party’s control of both houses of the legislature, a strong mandate from the voters, and a sharp focus on a single subject (in his State of the State address in 1999, the governor announced that education would be his “first, second and third priority”), Governor Davis was in a unique position to launch educational reform. He wasted little time in seizing the ini— tiative. Soon after his election, he announced the appointment of Gary Hart, former Chair of the Senate Education Committee as his new Secretary of Education and Child Development. Hart, a long—time prominent PACE I 89 90 advocate for school reform, arguably was the best possible appointee for that position, having earned the respect of legislators from both sides of the aisle after a long and distinguished career in both houses of the legislature. Davis also announced that he would con- vene the legislature in a Special Session on Education shortly after his inauguration. During this whirlwind of activity and in the budget deliberations which followed in the early days of 1999, the governor laid out an historically ambitious set of reform initiatives. This governor set a new standard for executive involvement by repeatedly signaling with great specificity what he deemed acceptable in the way of amendments. While legislative delibera- tions, for the most part, improved the original proposals, the legislature nevertheless played a secondary role, essentially accepting the gover— nor’s avalanche of initiatives. The governor’s initial reform proposals included the following: A New Accountability System The centerpiece of the governor’s education plan was accountability. The new accountability legislation requires that every school be given a ranking on the Academic Performance Index (API) by January of 2000. Schools are also given specific targets for expected improve— ment. Schools that meet or exceed these targets by a significant amount will be rewarded by additional funds, both on a per student basis (up to $150 per pupil) and for teacher bonuses (not to exceed $25,000 per teacher). In order to protect against simply rewarding schools with the highest scores—typically those with the most affluent students—schools are required to show comparable gains for students at differing performance levels. So far, although the legisla— tion called for multiple measures of academic performance, the sole determinant of the API, because of data limitations, is the cumulative school score on the SAT 9, the state mandated norm—referenced achievement tests. In addi— tion, 430 of the lowest performing schools (selected from schools performing in the bot- tom half of the distribution) are participating in the Immediate Intervention/ Underperformin g Schools Program (II/UUSP). These schools are eligible to receive funds to hire outside evalua— tors and work with their communities to devel— op plans to improve student achievement. Additional schools which fail to meet their tar— gets in 2000 and in subsequent years will be provided with financial assistance to improve their performance. If, over time schools fail to show improvement a series of increasingly seri— ous interventions may take place, possibly cul- minating in the ultimate sanction—the closure of the school. New Reading Initiatives The governor adopted a series ()f related initia— tives, all aimed at improving reading skills. First the governor initiated a “Call-to—Action.” read— ing campaign in an effort to involve a large cross—section of California’s population in a massive effort to demonstrate that everyone is responsible for making certain that youngsters read. The governor also will reward $5,000 to up to 400 schools whose students read the largest number of pages. Books must be approved by both the teacher and the parent. In addition, reading academies have been estab— lished to ensure that every youngster learns to read. These academies operate before and after school and during the summer to provide inten— sive reading instruction for students with read— CRUCIAL ISSUES IN (:ALIFORNIA EDUCATION 2000 ing difficulties. The governor also involved the University of California and the California State University system to provide Professional Development Institutes for Reading to develop skills for up to 6,000 teachers who would then participate in direct instruction or in supervising other teachers. This is a massive, unprecedented effort at professional development which is like— ly to grow even bigger: the governor’s most recent budget calls for an increase to 20,000 K— 3 teachers participating in the Institutes. High School Exit Examination Beginning with the class of 200-1, in order to graduate from high school, students must suc- cessfully pass this to—be-developed test. The test must be aligned with state academic con- tent standards. In spite of early developmental difficulties, Spring 2001 remains the target date for the first administration of the high school exit examination. Peer Assistance and Review The final major piece of the governor’s special session legislation was devoted to establishing a teacher Peer Assistance and Review (PAR) sys- tem as a feedback mechanism that allows exem— plary teachers to assist veteran teachers in need of development in subject matter knowledge and/or teaching strategies. PAR requires teach— ers, administrators, districts, and unions to work together to improve the quality of instruction. Districts must establish integrated professional development and performance assessment systems. PAR will replace the exist— ing A'Ientor Teacher Program byJuly 1, 2001, but districts may begin to transition to the new program in the current year. So far, over half of Governance and Accountability the districts have already applied to transition in the current year. Other Initiatives In addition to the above, the governor and the legislature have taken other legislative and bud- getary actions, including increased funding for standards—based instructional materials, English Language Development Institutes for grades 4- 12, and an English Language Development test to assess student acquisition of English lan— guage skills. Funding was also allocated to decrease class size in selected high school class— es. Efforts to boost the teacher workforce include incentives to increase the numbers of new teachers and to retain teachers already in the system, increased beginning teacher salaries, increased incentives for teacher aides to become teachers, and bonuses for teachers who obtain National Board of Professional Teaching Standards certification. The governor’s 2000—01 budget is primarily devoted to the teacher quality and supply issues, which we fully discuss in chapter 6. In summary form, the governor places significant emphasis in his new budget on professional development by adding new professional devel— opment institutes administered by the University of California in partnership with the California State University. The model for these institutes is the highly successful Subject Alatter Projects, a University of California— based network devoted to developing school teams consisting ofnew teachers, experienced teachers, and administrators who are expected to form a core of expertise to carry back to their schools. The budget proposes an unprece— dented expansion of this concept to provide PACE I ()1 ()2 professional development for teachers and administrators from schools with low SAT-9 scores for the following kinds of institutes: Grade 4—6 Mathematics, Grade 7—8 Algebra, High School Mathematics, High School Algebra, and High School English. Including the substantial increase in the dollars appropri— ated for the subject matter projects (from $15 million to $35 million), the new budget calls for approximately $120 million in new monies for professional development purposes. Summary: An Ambitious if Less than Coherent Reform Agenda Taken together, all of these initiatives represent a dizzying array of educational reform ideas designed to improve student achievement in California schools. More than any time in the state’s history, education policy is driven not by locally elected school boards, nor by the popu— larly elected state legislature, nor by the superin- tendent of public instruction, nor by the newly empowered state board of education. Even the politically powerful teacher unions and adminis— trators of school boards are now more limited in their influence. The governor, favored by a strong electoral mandate and driven by a deep and abiding public concern about the quality of education in California, is the progenitor of the ambitious reform plan. He plans to take respon— sibility for it, too, having publicly stated that he will not deserve re—election if test scores do not improve. The governor’s involvement in educa— tion is not only broad but deep, as he has often concerned himself with even the most minute details of the reform agenda. CRUCIAL ISSUES IN CALIFORNIA EDUCATION It is much too early to make judgements about the success or failure of these reform ini- tiatives, though there is no denying the many implementation problems which have occurred. First and foremost, most observers agree that the basing of a high stakes accountability sys- tem on a single measure, the SAT 9—a test normed on a national sample of students who bear little resemblance to California’s school population and based on a sample of questions only loosely reflective of California’s new stan- dards (although the test has been augmented with items that more clearly reflect the stan— dards)—is a severe problem. The SAT 9 has also come under increased scrutiny because of a series of scoring and reporting irregularities. Second, the Academic Performance Index is flawed. Although originally designed to reflect a number of indicators of school performance, because of severe data quality problems there are no other data currently collected by the CDE which can reliably be added to the school quality formula. Thus, relying only on a single measure of performance is highly problemati— cal. Third, data designed to allow schools to compare themselves with schools with similar student socioeconomic conditions is also seri— ously flawed—districts failed to report informa— tion in a consistent and coherent manner— requiring the Department of Education to throw out the comparative numbers and to go back to the districts for accurate data. The Immediate Intervention Under— performing Schools Program II/USP has been faced with similar implementation problems— getting planned assistance and a sufficient pool of quality evaluators in place in such a short time has proven difficult. ZOOO A recent study by Far West Laboratory and Management, Analysis and Planning on the state of accountability in California points to serious implementation problems. Most dis— tricts are barely underway, and few have imple— mented a satisfactory, standards—based account- ability mechanism. This highlights the real dilemma in California’s accountability efforts: the pressure to adopt accountability provisions and to do it quickly has so far outstripped dis— trict and state capacity to adequately respond to the many problems full-scale implementation brings. On one hand, pressure from the general public and the business community to address the problems facing schools is immediate. On the other hand, to do it right requires money, time, and careful planning. School districts making the case that more time and money are needed for reform are viewed as recalcitrant. The end result is a system which requires immediate action and discounts caution. It is important that the simplest solution—abandon— ing accountability as a reform strategy not prevail. But California’s long reliance on “quick fixes” should disabuse us of the effectiveness of any strategy that abandons careful deliberation. School district tolerance of stop—and—start state initiatives has long passed and is being sorely tested. Instead of moving ahead with more ini— tiatives the state should continue to incremen— tally improve alignment and information gath— ering. Work already underway should be con— tinued. Developing an English language learner test; adding standards—based items to the SAT 9; aligning textbooks, curriculum frameworks, professional development activities, and teacher preparation program strategies——these are all positive strategies that must be maintained. Governance and Accountability Even thOugh difficulties in implementing an accountability implementation abound, it is nevertheless critical that the state stay the course. PACE, therefore, does not advocate abandoning the accountability thrust, but urges the governor and the legislature to correct the problems which exist. If necessary they should provide additional resources, most notably time, to make certain the state “gets it right.” Most importantly, California must directly address the data problem. California’s K-12 data collection system is simply not appropriate for the kinds of high stakes consequences involved in a sophisticated and complex accountability system. Absent quality data, the best efforts of teachers and administrators will likely come to naught. The proposed California Student Information System (CSIS) is a move in the right direction; the state must work to get it fully implemented. As to the ultimate effectiveness of the vastly increased role of the governor in policy making, that too must fall in the “too early to tell” cate- gory. Part of Governor Davis’ early successes can partly be attributed to the extraordinary health of the economy which continues to gen— erate huge revenues, far exceeding even the most optimistic projections. There is also the eupho— ria among Democrats in the legislature who, after 16 years, finally have one of their own in the governor’s office. These Democrats were abundantly willing to support the new governor and his initiatives, many of which squared with their own ideas about education reform. However, early in the second year of the Davis administration, there were signs that the gover— nor’s hands—on, “my way or the highway” approach had begun to chafe legislative leaders PACE I ()3 94 and some gubernatorial appointees. The gover— nor offended many legislators with his remark that their job was to implement his vision. Gary Hart’s resignation as the Governor’s Secretary of Education is, by any account, a significant set— back. Hart was enormously successful in work— ing with the legislature. Furthermore, the sec- ond round of the governor’s education budget proposals have come in for much heavier criti- cism, as school districts and powerful statewide teacher’s union are decrying the loss of local flexibility and their inability to meet the collec- tive bargaining demands of their teachers, administrators and classified employees. The California Teacher’s Association is currently cir— culating an Initiative for the November ballot which would bring California’s expenditures for education up to the national level. The governor adamantly opposes this additional restriction on budgetary flexibility. Elizabeth Hill, the respected Legislative Analyst, in a particularly critical annual budget analysis, points to several concerns with the proposals in the governor’s budget: ° They are too centralized, leaving little flexi- bility for local education agencies directly responsible for delivering instruction. For example, the professional development insti— tutes are provided only by UC administered programs, only in the funding portions allo— cated and only for the purposes specified. The Analyst makes the case for allocating those dollars directly to districts through staff development block grants. ' They expand too rapidly—for example, the PDIs increase from about 25,000 trainees in 1999—00 to 93,500 in 2000—01. The Analyst argues that such an expansion severely over- taxes the capacity of the University to respond, thereby undermining program quality. 0 They make scores on the SAT-9 the sole cri- terion for eligibility for the bulk of the pro— fessional development dollars. Dollars instead should flow to schools based on demographic and socioeconomic factors. All of these issues will play out in the next few months of the current legislative session. While the road ahead may be bumpier than his first year, there are many factors which strengthen the governor’s hand. It may be that the governor, because of substantial budget surpluses, will be able to meet the demand for more dollars with no strings attached by pro- viding additional general fund revenue to school districts. He may also be able to enact many of his second year initiatives which do, we think, appropriately address the critical issues of teacher quantity and quality. Governor Davis enjoys strong approval ratings with the public, and his unwavering commit— ment to education will be helped along by a StI'OHg BCOHOIDy. Notes 1. Susan Fuhrman and Richard Elmore, “Understanding Local Control in the \Vake of State Education Reform,” Educational Evaluation and Policy Analysis 12 (Spring 1990): 88. 2. Bruce Fuller, et al., School Choice: Abundant llopes, Scarce Evidence of Results. (Berkeley, CA: Policy Analysis for California Education, 1999). CRUCIAL ISSUES IN CALiroRNiA EDUCATION 2000 Chapter 6 Teacher Quality The Center for the Future of Teaching and Learning The convergence of public opinion, policy focus, and system dynamics is thrusting teachers and the quality of their teaching to the forefront of education reform in California. The public opinion poll The Extremal] Prafi’n‘ion indicates that the public has a heightened understanding of how teacher effectiveness propels student achievement; consequently, it backs education reforms keyed to teacher quali- ty.‘ Policymakers, while still focusing intensely upon school accountability and the implemen— tation of explicit academic standards, are also making new efforts to expand the supply of qualified teachers. This will be a challenge, as the demand for teachers is soaring on account of student population growth, class—size reduc— tion, an aging teacher workforce, and a high rate of attrition. These forces tug in two very different direc— tions. On the positive side, public and political support for placing teachers at the center of school reform efforts is at an all—time high, as demonstrated in a variety of polls and by the creation of multiple programs and legislative initiatives focused on quality teaching. ()n the other hand, two critical issues present a threat to this positive momentum: 0 Most of the teacher quality initiatives under— taken at the state level are based on a belief in evolutionary change; policymakers often assume that better recruitment, larger public preparation programs, and support for new- comers in the early years will change the profession. While we should expect these efforts to have a positive impact over time, parents want consistently high quality instruction for their children now 0 The shortage of fully qualified teachers is being most severely felt in the most chal— lenging inner—city and rural schools with substantial poor and minority enrollments. In 37 percent of the state’s urban schools, 20 percent (one in five) of the teachers are under—qualified. Consequently, a snapshot of the condition of teachers and teaching in the year 2000 is great— ly affected by where the camera is aimed. For the “pretty” picture, we can look at rapidly expanding teacher preparation programs, full funding for beginning—teacher support pro— grams, and stipends/rewards that supplement teacher salaries and provide incentives for pro- fessional development in much needed direc— tions. For a much grimmer portrayal, we can PACE I 95 96 turn the camera on inner-city schools where class—size reduction has all too often spawned a mad scramble for anyone willing to teach. The poorest, most challenged schools are often left with little choice other than to hire untrained or under—prepared people with emergency per— mits or waivers, while their most skilled and experienced teachers are often recruited away by more affluent districts. As California continues to focus on the teacher workforce as a key to educational improvement in the coming years, we are for- tunate to have compelling research on what works, growing insight into what needs to be done, and unprecedented public support for the job at hand. But we are challenged by difficult and even deteriorating conditions that will make it difficult for us to catch up. Within this context, the Center for the Future of Teaching and Learning is nearing the end of a two—year initiative called Teaching and Californiak Fllfll7‘6, undertaken by a task force charged with understanding the issues that shape and mold the condition of teaching in California. Composed of a broad cross—section of education stakeholders and practitioners, the task force has as its central focus the develop- ment of high—quality teachers for all California classrooms. For the past two years, the task force has followed a two—fold strategy: taking an inventory on the current conditions of teacher development and building an effective coalition to promote and implement promising changes. Convened in January 1998, the task force spent much of the first year completing its inventory, resulting in the publications All Invenfom/ of the Storm of Tear/yer Developmem‘ in California, The Con.~‘1'.s‘fem:y and Coherence of Srmzd/Irrzls‘for Crllzfomio Woe/Jen; and The CRUCIAL ISSL'ES IN CALIFORNIA EDUCATION Essential Profession. During 1999, the task force concentrated on analyzing data, dissecting cur- rent policies, and formulating recommenda- tions for its final report in December. This chapter highlights the work of the Task Force on Teaching and California’s Future to date: its shared understanding of research on teaching, its findings about the status of the teacher workforce today, and its thinking about the issues that must yet be addressed. In the fol- lowing sections, we look at the academic case and public support for teacher quality; the poli- cy initiatives aimed at teacher quality that are already underway; and the system dynamics that are driving a growing gap between teacher sup- ply and demand. We then turn to the task force’s work (released in December 1999) that identifies key policy issues around reinvigorat— ing and reformng the teacher development sys— tem in California. Finally, we discuss some of the steps required to create a flexible, holistic system that will ensure that all of California’s children have qualified, effective teachers. Quality Teaching Counts: Research Shows It, the Public Knows It The Education Trust, a foundation that pro— motes high academic achievement, points out that sophisticated parents have long jockeyed to place their children with teachers who “every— one knows” are effective. Too often educators and policymakers have blamed low student achievement on external factors not within a school’s control: socio—economic background, lack of parental support and attention, language barriers, abuse and neglect at home, and so on. 2000 Certainly these factors come into play when student learning is measured. But a stream of studies is now demonstrating that there are sig- nificant differences in teacher quality—and that effective teachers have a marked effect on stu— dent achievement.ii A few examples of study results illustrate just how crucial teacher quality can be. Figure 1 comes from a 1991 Texas study that tracked student test scores with teacher quality, as mea— sured by scores on a licensing exam, possession of a master’s degree, and years of teaching experience. The study, which analyzed student test score variations in 900 Texas school dis— tricts, showed that more than 40 percent of the differences in math and reading scores in grades 1 through 11 could be accounted for by teacher expertise. Lower class size also had a small but noticeable affect. Combined, these two factors—teacher quality and class size had a greater impact on student learning than home and family factors.‘ A 1997 study in Dallas, Texas, also analyzed the relationship between teacher quality and Teacher Quality test scores, examining what happens when stu- dents are exposed to multiple years of highly effective or less-effective teachers. Figure 2 shows that when two groups of similar-ability students were assigned teachers of varying quality over three years, the difference in read- ing scores was immense. Each group started at around the 60th percentile in reading achieve— ment. At the end of sixth grade, the students who spent three years with highly effective teachers scored in the 76th percentile. The stu- dents who were assigned for three years to less effective teachers dropped to the ~12 nd per— centile.4 Closer to home, a recent study by the Los Angeles County Office of Education found that 25 percent of the differences in student perfor- mance on reading achievement tests could be attributed to teacher qualificationsv Similar results regarding the link between teacher qual— ity and student test scores have been found in many other states, including New York (1989), Alabama (1996), Tennessee (1996), and California (1999). Other studies have found 8% I Home and Quality Factors I Teacher Qualifications I Class Size 90 Beginning 4th Grade Score 60 Percentile (In Percentile) Students Assigned to 3 Very Effective Teachers in a Row Average Reading Score 3 Years Later I Students Assigned to 3 Very Ineffective Teachers in a Row 15 Figure 1. Influence of Teacher Qualifications on Student Achievement Figure 2. Teacher Effects on Student Reading Scores (Grades 4-6) PAC E I 9 7 0 8 that students score higher and are less likely to drop out when their teachers are certified in the subject matter they teach, have master’s degrees, or are enrolled in graduate studies. Another important perspective on teacher quality comes from analyzing the economics of investing dollars in different school improve- ment strategies to achieve higher test scores. Figure 3 displays the accunmlated results from ()0 production function studies, charting the rise in test scores against each $500 spent on class size, teachers’ salaries, and teacher experi— ence and teacher education. Investing in more education for teachers and hiring teachers with more experience both pay off in greater student achievement gains.“ The implications that can be drawn from the data in the above three figures is that enhancing teacher quality can be a pivotal, make—or—break reform for anyone trying to improve student achievement. Yet much of the action across the nation in the 19905 was focused not on teacher quality but on account— ability: setting explicit standards, administering rigorous tests, and delivering rewards and sanc- tions. The results of education reform in four states only add more fuel to the proposition that teacher quality is the key. North Carolina and Connecticut, where reform focused on improving pre-service teacher education, licensing, beginning teacher mentoring and ongoing professional development, posted Size of Increase in Student Achievement for Every $500 Spent on: 0.25 0.20 (n .t' 5 0.15 9 O a?) g 0.10 8 l.— 0.05 0 Lowering Increasing Increasing Increasing Pupil/Teacher Teachers“ Teacher Teacher Ratio Salaries Experience Education Achievement gains were calculated as standard deviation units on a range of achievement tests in the 50 states reviewed. Figure 3. Effects of Educational Investments on Student Scores Source: Rob Greenwald. Larry V. Hedges, & Richard D. Haine (1995). The Effect of School Resources on Student Achievement. Review of Educational Research 55(3), pp. 361-396. (:RL'CIAI. lSSL’liS l.\' CALIFORNIA EDL‘CA’I‘ION ZOOO some of the largest gains of any states on national reading and math tests. South Carolina and Georgia, which emphasized student testing as the engine for education reform, had flat results." This research—supported case for teacher quality is apparent in public attitudes, accord— ing to a 1998 national survey that looked close— ly at California. Conducted by Recruiting New Teachers, Inc. on behalf of the Center for the Future of Teaching and Learning, the poll asked Californians what needs to be done to improve schools, what role teachers should and could play, and which reform efforts are most likely to be productive. Over and over again, Californians gave answers indicating that they understand how important teacher quality is. Following are some key findings as reported in The Eowzz‘ial Profession? ' While keeping schools safe from violence was rated by 92 percent of Californians as the most important measure for increasing student achievement, ensuring that there is a well-qualified teacher in every classroom was not far behind at 89 percent. Issues such as reforming the curriculum, imposing strict discipline, reducing class size, and requiring school uniforms all scored far lower. 0 When Californians were asked to pick between requiring achievement tests, estab— lishing a system of academic standards, and improving the quality of teachers, a plurality (47 percent) chose the quality of teaching as the most influential factor in student learning. ° Californians felt that the most important qualifications for teachers were being well— trained and knowledgeable about how to Teacher Quality teach effectively (91 percent), thoroughly educated in the subjects he/she teaches (88 percent), knowledgeable about how people learn (88 percent), and able to establish good relationships with children and adolescents (88 percent). ° When asked about allowing parents to use public school funds for private education, Californians strongly rejected vouchers in favor of doing whatever it takes to get a fully qualified teacher in every classroom (84 to 14 percent). Similarly, they preferred getting fully qualified teachers to allowing parents to hire an outside organization to run public schools (83 to 14 percent). In summing up the results, the polltakers wrote: The California results provide an unusually in—depth portrait of public attitudes about teaching, one that reveals clear-cut apprecia— tion of the difference that quality teaching makes to student outcomes and the state’s future. Despite myriad challenges facing education today, recognition of the impor— tance of quality teaching cuts across all groups and socio—economic strata statewide. Once the issue of school safety is addressed, Californians conclude that qualified teachers are the heart of school reform and are will— ing to support a far bolder agenda to ensure a qualified teacher in every classroom." The report on the poll ends with a call to poli— cymakers to “mark well” the avid interest of Californians in high—quality teaching. And in many ways, new initiatives are demonstrating that policy makers are already heeding the message. PACE I 99 100 Strengthening Teaching: The Emphasis Is on Evolution, Not Revolution In the past few years, California has instituted several initiatives aimed at improving the capacity and effectiveness of the pipeline that turns college graduates into credentialed, high- quality teachers. These initiatives, considered in tandem, have the ability to directly affect the overall quality of the current workforce. Additionally, new accountability reforms call for improvements in the academic achievement of California’s diverse student population; both new and veteran teachers will need support and assistance if they are to help students meet higher academic standards. The pipeline into teaching begins with recruitment, followed by the preparation of teacher candidates through either a traditional or alternative certification program, and then extending to the support programs for begin— ning teachers. In each area, California is mak- ing strides: 0 Recruitment. The state created the California Center for Teaching Careers (CalTeach) as a one—stop information and referral recruitment center for people inter— ested in teaching careers. CalTeach works on drawing people into teaching through adver— tisements, newsletters, and an interactive web site. CalTeach also promotes creative approaches to teacher preparation by sharing information about the development of promising programs throughout the state. The state has funded a series of television spots that encourage young people, people from diverse backgrounds, and people looking for a career change to consider teaching. In (:RL‘t‘lAl. IssL'izs IN CALIFORNIA FDL'UATION addition, the state is forming reciprocity agreements with other states so that qualified teacher from other states can teach in California without having to attain further credentialing or navigate a bureaucratic maze. ' Preparation. The California State University system has been given the man- date and resources to increase its capacity to produce teachers. In addition, teacher prepa- ration programs are trying to become more flexible so that they can better draw in re— entry and career-changing candidates. Alternative certification routes are multiply- ing in number and expanding in capacity. Some school districts now have innovative programs to “grow” their own teachers, while institutions of higher learning are forging partnerships with each other and with districts to create new teacher education options. Regardless of just what paths teach— ers take into teaching, the Commission on Teacher Credentialing (CTC) is taking steps to ensure that all paths are aligned with the state—adopted content standards for students and the instructional standards for teachers. The state is also underwriting the cost of teacher education to a greater degree, signif— icantly increasing funding for a program— Assumption Program of Loans for Education (APLE)——tliat provides college student loan forgiveness when newly certified teachers enter and remain in the profession. ' Induction. National studies have demon- strated how valuable it is for beginning teachers to be guided by mentors and sup— ported intensively during the first few years of teaching. The payoff comes not only in higher quality teaching but also in a higher rate of teachers remaining in the profession. ZOOO California has proven the value of this approach with a long—running project, the Beginning Teacher Support and Assessment (BTSA) program. The recent focus on teacher quality has prompted policymakers to endorse and fund the rapid expansion of this program, theoretically making it avail— able to all new teachers. Studies by the Center of the program’s implementation at the local school district level indicate that, while the program is growing, the state has a long way to go before every new teacher is guaranteed a consistently high-quality induc— tion to the profession. Other state policies are clearly designcd to improve the quality of teaching in California’s classrooms. Perhaps the most visible policy has been the class—size reduction program. Implemented hurriedly in 1996—97, this pro- gram has cut class sizes to 20 or fewer students in the vast majority of the state’s K—3 class— rooms. While dramatic improvement across the board in student performance is yet to materi— alize, parent satisfaction with instruction has significantly improved with the reduction of class size, and lower—grade teachers have enthu- siastically embraced the reform. (Nevertheless, class—size reduction raises significant issues, which are discussed below.) The state has also begun to use incentives and rewards to improve the quality of teaching and shape the expertise of the available work— force. Programs in this category include a $10,000 stipend for completing 1 Tational Board certification; special programs to pay for retraining teachers in math and science; and funding to allow school districts to increase beginning salaries. Some education system observers also see the recently improved fund— Teacher Quality ing for modernizing school facilities and increasing security at schools as measures that can increase the quality of teaching, as a better work environment will draw a broader range of teacher candidates. Finally, California has taken major steps toward defining grade—by—grade academic con- tent standards and implementing rigorous assessment tools. Both set the context for improved teaching by ensuring that desirable outcomes are clearly articulated and that learn— ing is a continuous and cumulative grade-to— grade process. Understanding that teacher quality is the key to improving student achievement is only part of the battle, however. In the face of limit- ed supply and growing demand, ensuring that there is a quality teacher in every classroom will be an arduous task. Supply and Demand: Further Disadvantaging the Disadvantaged The demand for teachers, coupled with what is projected to be an insufficient supply, presents a bleak picture. After studying historical data and projected need on behalf of the Task Force on Teaching and California’s Future, SRI International concluded that California has long suffered from a shortage of fully qualified teachers. At the beginning of the 19905, 12,200, or about 5.5 percent of teachers, did not have a credential and were teaching on emergency permits. Class-size reduction dramatically increased the shortage. By 1998—99, the num— ber of emergency permits had risen to about 28,500—meaning that about one in every ten PACE I 101 102 California classrooms is staffed by an emer— gency—permit teacher. ‘” There is little sign that the shortage driving the use of umler—qualified teachers will dimin— ish any time soon. Today there are almost 260,000 teachers in California’s K-12 class— rooms. From 199‘) through 2007, California expects to need more than 282,000 new teach- ers (not counting the need to upgrade to cre— dentialed status the 28,500 teachers now teach— ing under emergency credentials). This need will be driven by an expanding student popula— tion, a possible mass exodus of retiring baby— boomer teachers, and the normal attrition of teachers leaving the field for other careers. Figure 4 illustrates the actual and projected annual rate of teacher hires. The sudden spike in 1996—97 and 1997-98 reflects the surge in teacher demand as class size reduction was implemented. What the figure shows is that about 25,000 new teachers will be needed each year—and that figure presumes that today’s policies remain fairly constant. The numbers could escalate easily if teacher-intensive reform poli— cies are implemented—for instance, an increase in the grade levels covered by class size reduc— tion, or the creation of a special corps of men- tor teachers, or the restructuring of teacher workloads to allow more planning time. The numbers could also be affected in the opposite direction if the annual attrition rate is reduced 40,000 35,000 30,000 25,000 20,000 15,000 Number of Teachers 10,000 5,000 1992—93 1994-95 1996-97 I Actual Teachers Hired Projected Teachers Hired 1998-99 2000-01 2002—03 2004-05 2006—07 Figure 4. Actual and Projected Teacher Hires 1992-2007 CRUCIAL Issuus [N CALIFORNIA EDUCATION 2000 by improving conditions in classrooms, if the annual retirement rate is lower than expected because of teacher willingness to work past the now average retirement age of ()0, or if improved hiring practices and beginning teacher support programs encourage more peo- ple to remain in the profession. Even in a best case scenario, California is not well situated to meet the expected need for classroom teachers. During the past five years, almost 17,000 people per year were creden— tialed, either through training programs or rec— iprocity agreements with other states. While teacher preparation capacity is expanding and new support programs may improve the teacher retention rate, SRI has concluded that “the most optimistic projection” of 282,000 fully credentialed teachers in the workforce by 2007 will fall short of the projected demand for 297,000. These statistics make it likely, under cur— rent policies, that California will continue to use under—qualified teachers. If this population was spread evenly across the 7,500 schools in the state, the challenge of compensating for a few under—prepared staff at each school might, in most cases, be manageable. But they are far from evenly distributed. Instead, the shortage is uneven across different geographical areas and subject areas. The most at—risk students— low—income, special—needs, English— impaired—are the least likely to have a fully trained, credentialed teacher in their class— room. Statistics show: ° 120 of the state’s 1,000 school districts have 20 percent or more under—qualified teachers on their payrolls. ° In 37 percent of urban schools, 20 percent or more of the teachers are under—qualified. Teacher Quality ° Schools with the largest populations of poor and minority youngsters consistently have the greatest numbers of under—qualified teachers. Figure 5 is a sampling of areas around the state showing the high usage of emergency creden— tials in inner cities and remote rural areas. In specific schools within districts, the numbers of emergency credentials can be even higher. In addition, teacher shortages are evident for bilingual education, special education, math and science, as well as for some secondary school subjects. Besides the numerical shortfall in the supply of teachers, there is also a mismatch between the ethnicity of teachers and students. While a teacher of any ethnicity can be effective, teach— ers who share the culture and even language of their students may ease the learning process, as well as serve as important role models. In California, the teaching work force is largely white, as Figure 6 reflects. The growing demand for well—qualified teachers, combined with the system’s current inability to supply them to the right places with the right specialties, is a challenge that threatens to overwhelm the best intentions of policymakers. Key Policy Issues: Stumbling Blocks Identified by the Task Force The Task force on Teaching and California’s Future has begun to identify key policy issues that need to be addressed for quality teaching to become the widespread norm rather than the occasional occurrence. Among the issues are: PACE % 103 104 Fragmentation in the teacher develop- ment system. Ideally, California would have a career continuum that would engage prospec— tive teachers early, train them in a blended, bal— anced approach emphasizing both subject con- tent and pedagogical theory, support them in their early years as they transform theory into practice, and then nurture their continuing professional growth through opportunities for collaboration and ongoing learning. The reality of the path to teaching in California is far more rocky. Except in a few bright spots where the various pieces are beginning to work together, the average California teacher has come to his Percent of Teachers Number of on Emergency Emergency Number of Waivers Permits and District Permits Issued Issued Waivers Northern California Sacramento City 156 33 8% Elk Grove 48 0 3% Modoc 0 0 0% Bay Area Oakland 352 88 1 6% Palo Alto 18 2 4% Cabrillo 14 5 9% Central Valley Fresno 172 29 5% Selma 2O 3 8% Firebaugh-Las Deltas 16 3 16% Greater Los Angeles Area Los Angeles 5,484 614 19% Ventura 35 7 5% Snowline 37 1O 14% Greater San Diego Area San Diego City 294 11 4% Vista 45 0 4% Imperial 21 5 23% California 23, 687 3,8 1 O 1 1 % Figure 5. Comparison of Emergency Permits and Credential Waivers Within a Sample of California Districts, 1996-97 Source: CTC (May 1998). Numbers include special education waivers and permits. CRUCIAL ISSUES IN CALIFORNIA EDUCATION 2000 Teacher Quality Percent of Total 100 ‘l 2 9 White Latino Black Ethnic Group I Students I Teachers 1111 10 Asian Filipino Am. Indian Pac. Islander Figure 6. California Student and Teacher Ethnicity, 1997-98 or her job from a “silo” education, with subject content learned separately from teaching theo— ry and with teaching theory largely disconnect— ed from the practical necessities of classroom management. If available, mentoring may be haphazard—or even more of a burden than a help. Professional development is often an iso— lated, day—long lecture on a topic that may or may not relate to what is happening in the teacher’s classroom. School scheduling leaves little time for collaboration or reflection. This fragmentation of teacher development has consequences that reach beyond the simple but compelling issue of effective teaching in the classroom. Without a clear and attractive career path, many bright and talented people opt for other professions. Without an instilled sense of teaching as a profession rather than as a job, it is difficult to inspire the continued growth and self—reflection that drives high—quality teaching. And without a holistic system that produces consistent results, sustaining public and policy— maker support is problematic. Innovative programs that tackle parts of the teaching career continuum are taking root around the state, and the progress they are charting should not be overlooked. However, a key challenge is to bring the many pieces together in a cohesive whole that will bridge the different components of a teacher’s career. Continuing teacher shortages. The state must continue to crank up the volume of teach— ers delivered into the system without losing the focus on the training and preparation necessary for quality teaching. The cumulative effect of policies designed in good faith—such as class PACE I 105 106 size reduction has been to continue and exac— erbate a long-term teacher shortage in California. The response to the growing shortfall has been to issue more emergency permits than ever before and to rush people through alterna— tive programs that emphasize learning on the job. This accomplishes filling the void at the front of the classroom but has major negative consequences. First, student achievement is lower when students are taught by an underpre— pared teacher—or one who is overloaded with working at a full—time job while carrying an aca- demic course load to complete the teacher cre- dential process. Second, attrition is high. Many teachers flee the profession within the first five years, and research suggests there is a strong link between attrition and poor preparation and inadequate on—the-iob support. It should be noted that a person with an emergency permit is not by definition a “bat ” teacher, just a person struggling with the com— plex task of teaching demanding subjects to students without training or support. The chances for high—quality teaching are enhanced when people are properly trained and support— ed. “bile insufficient numbers of such people is a compelling reason to look at other alterna— tives, one national teacher expert argues that many of the emergency hires and shortages occur for other reasons, especially poor local hiring practices.ll Another problem that has emerged as short— ages have deepened is the poaching of qualified staff by neighboring districts. Often districts siphon teachers away from other districts with signing bonuses, higher salary scales, more attractive work environments and other factors. For instance, in largely rural Imperial County, CRL'CIAL lSSL'Iis 1.\‘ CALiroRNL-x FDL'CA'I‘lox the relatively affluent El Centro district with eleven schools had almost no under-qualified teachers in 1997-98 (three percent of the work- force). But many of the far smaller and less attractive school districts in the surrounding areas had rates running from 20 to 38 percent. Rather than hiring emergency-credentialed teachers, El Centro has established incentives that attract experienced teachers from smaller districts. This serves El Centro’s students well, but is less helpful for the students in surround- ing areas.” It is impossible to tell hard—pressed inner city districts today that they must avoid hiring unprepared teachers since they are duty-bound to staff their classrooms. However, as the capac- ity to produce credentialed teachers expands and the supply of high-quality candidates increases, it is critical to set a standard of mak- ing the emergency-credentialed teacher the rare exception rather than the frequent solution. Mismatch of teacher expertise with stu- dent need. National research indicates that in any time and place, the least effective teachers are delegated to teach the children most at risk and with the highest level of need for expert teaching. But the shortages in California, exploding under the pressure of class size reduction and exacerbated by years of deterio— rating school settings, are particularly acute in the crowded, low—income, and culturally diverse inner city schools. As one researcher noted after on—site visits, even salaries of $100,000 a year couldn’t induce people to work in such environments with so many Challenges and so little support. California is far from the only state in which less—effective teachers tend to end up in schools with the highest concentrations of poor and 2000 minority students. Figure 7 is a national tabula— tion of data about the percentage of classes taught by teachers lacking a major in the field they are teaching, a key indicator of teaching effectiveness. As the figure illustrates, 25 per— cent of the classes in high—poverty schools and 22 percent of the classes in high-minority schools are taught by such teachers, far higher than the percentages for low-poverty (15 per— cent) and low—minority schools (16 percent)“ Other studies have shown that nationwide minority students are half as likely to get highly effective teachers and twice as likely to get low— rated teachers. In Texas, African American and Latino children are far more likely than white children to be taught by teachers who score poorly on the Texas licensing exam; in states such as Virginia, Pennsylvania, and Oklahoma Teacher Quality teachers without a degree in their subject area are more likely to teach in high—poverty or high-minority schools. But in California, the problem is particularly acute. Researchers evaluating the BTSA pro- gram found that 75 percent of new teachers in the Los Angeles Unified School District—the largest school district in the state with almost 700,000 children—are not credentialed. By far, most of them end up teaching in highly demanding classes where students speak a vari- ety of languages, come from impoverished backgrounds that impair their school readiness, and live transient lives that take them from school to school with little continuity in acade— mic exposure. These are students who need the most talented, experienced teachers. But these teachers instead gravitate to more pleasant, 4O 20 Low Poverty High Poverty Schools Schools (less than (more than 15%) 50%) Low Minority High Minority Schools Schools (less than (more than 15%) 50%) Figure 7. Percentage of Classes Taught by Teachers Lacking a Major in a Field, 1993-94 Source: Richard Ingersoll, University of Georgia, Unpublished, 1998. PACE 3% 107 108 affluent settings where children are often more motivated and better supported at home. The bell cannot be unrung; class-size reduc- tion is a popular reform with both parents and participating teachers. But for many of the state’s large population of at-risk children, this reform has had a devastating effect on teacher quality. Truly effective reform of the state’s education system will not be accomplished until equity of access to quality teaching is achieved across all income levels and ethnicities. Uneven implementation of teacher sup- port. There is a need for even, high—quality implementation of new teacher support pro— grams. One of the most promising programs is BTSA, which involves giving first and second year teachers intensive assistance through men— toring and collaborative opportunities with vet- eran teachers. The payoff comes in improved teacher effectiveness and higher retention in teaching careers. Policymakers in California have recognized the program’s effectiveness by providing funds to scale up the program to cover all beginning teachers in the state. \Vhile no comprehensive studies of full statewide implementation have yet been con— ducted, an assessment of BTSA in eight school districts in 1999 on behalf of the task force is raising concern. From these case studies, it is clear that individual school and district imple- mentation of the program is highly uneven. Nlany teachers have little or no contact with mentors, the supposed heart of the BTSA pro— gram. Several key problems are emerging: ° The short supply of teachers has a cascading effect, especially in schools with 20 percent or more under-prepared staff. In some poor urban and rural schools, there are so few vet— CRUCIAL lssL'Es IN CALIFORNIA EDUCATION eran teachers and so many uncredentialed or new teachers that there are simply too few mentors. This may mean that a teacher’s mentor is at a far-removed campus (if a men- tor is assigned at all), often posing a barrier to a close working relationship. ° New teachers who are given paperwork and assignments to complete as part of BTSA but aren’t given the supportive network BTSA is supposed to provide often View the program as an additional burden rather than a helpful process. 0 Mentors who are not given adequate release time from their own classes, or who feel they cannot take time away from their obligation to their own students, often feel too harried to give beginning teachers adequate support. The potential for mentor burnout is a con— cern on the part of those watching the pro— gram’s development. While districts need flexibility to mold the BTSA program to local conditions, there also needs to be careful evaluation of BTSA imple— mentation to ensure that the program is consis- tently and thoroughly implemented. The state and local school districts will need to focus on investing in a workable infrastructure that can result in quality mentoring for all beginning teachers. Lack of focus on current teachers. In the rush to identify, implement and perfect a model teacher preparation program for beginning teachers, it’s easy to overlook the needs of California’s veteran teachers. But a large major— ity of today’s 250,000 teachers will still be in the classroom for many tomorrows. It is critical to give them the opportunity and the incentive to upgrade their skills so that they can better ZOOO meet the challenge of teaching California’s diverse students. The task force’s recent teacher survey shows that experienced teachers are gen- erally dissatisfied with the quality of their pro- fessional development activities. One notable exception to the state’s focus on new teachers is the creation of a $10,000 reward for those gaining National Board certi— fication. Board certification is certainly one incentive that may prove effective as more teachers become aware of the opportunity. But far more needs to be done. Writing about pro- fessional development in a special paper for the task force, teacher quality expert Linda Darling-Hammond remarked: Yeachers learn well just as students do—by study— ing, doing, and reflecting; by collaborating with other teachers; by looking closely at students and their work; and by sharing what they see. Good settingsfor teacher learning provide plentiful opportunities for research and inquiry, for trying and testing, for talking about and evaluating the results oflearning and teaching. . ..Developing the type ofnecessaiy knowledge and skill requires that most teachers move far beyond what they themselves experienced as students, and thus that they learn in ways that are more powerful than simply reading and talking about new pedagogi— cal ideas. Learning to practice in substantially difi‘erent ways than one has oneselfeaperieiu'ed can occur neither through theoretical imaginings alone. nor through unguided experience alone. It requires a much tighter coupling of the two. Unfortunately, much of professional develop— ment today comes in the form of one—day lec— tures from an outside consultant, with no opportunity for teachers to interact and little prospect for follow-up and support once the Teacher Quality teacher returns to the classroom and finds that theory is not fitting comfortably with practice. Instead of this staccato approach, a common perception of teaching as a lifelong learning profession needs to be developed, spread and supported, with frequent opportunities for col- laboration, discussion and planning built into the teacher’s daily routine. A lack of equity in teacher preparation opportunities. As pressure increases for a greater supply of teachers, the private sector is stepping up its efforts to offer flexible teacher preparation options—especially for mid—career teacher candidates who may be working at full- time jobs already. While these options provide a much—needed increase in alternative opportu- nities for people to become teachers, they tend to be more expensive than public—sector options. Many of the people who would consid— er teaching as a career choice cannot afford to quit their present jobs and pursue the full—time teaching preparation schedule most frequently found in public universities. But they also may not be able to afford the options offered by the private sector. A two—track solution is neces- sary: public institutions must become more flexible regarding their course offerings, while the state should give financial support to teacher candidates wanting to attend private sector programs. A failure to recognize and rely on teacher expertise. The cumulative effect of multiple reforms on the classroom teacher is often overlooked. Such reforms as class size reduction, lengthening the school year (at the expense of staff development days), and imple— menting standardized tests (before content standards are in place) can have unintended PACE I 10‘) 110 consequences when they are layered one on top of another. Often such reforms are scarcely enacted before a new wave of reforms arrives. A critical voice in any reform effort, whether incremental or holistic, should be the class— room teacher’s. They are front—line experts on what works, what is unduly burdensome, and on what challenges yet need to be met. Next Steps: Addressing the Issues As its report goes to print, the Task Force on Teaching and California’s Future is setting pri— orities and getting ready to issue recommenda- tions to improve the teacher preparation and development system. Those recommendations, into which teachers have had substantial input, represent a comprehensive, balanced view of the teacher development system. The recom- mendations will be designed to help teachers and other educators meet the need of California’s diverse student population. While they wait for the task force’s specific recommendations, policymakers should consid— er the following principles: ° Be cautious about proposing new reforms. By almost anyone’s count, schools have been subjected to dozens of major reforms in the past few years. When reforms come so rapid— ly, there is little opportunity to assess their effectiveness. There is also little time to cor— rect the mistakes that occur with even the most well thought out reforms. Teachers now need time to make the new reforms work—— not a bundle of new initiatives. 0 Listen to the voices of experience. The distrust of the abilities and judgment of today’s classroom teachers is counterproduc— CRUCIAL lsswzs IN CALIFORNIA EDUCATION tive. Many are very knowledgeable, highly accomplished, and have a wealth of experi- ence that could guide reform efforts. We need to find teachers who are succeeding against the odds and overcoming challenges, such as those recognized for excellence as members of the California Teacher Leadership Forum. And we need to use their input to improve the quality of teacher preparation and professional development. 0 Focus on weaving the disparate pieces of today’s teacher development system into a cohesive whole. The present system does not need any more add-ons or new direc— tions. It needs to be unified and effective from beginning to end. Only policies and programs that support a holistic system should be considered and adopted. ° Use What we already know about high- quality teaching to inform preservice and in-service programs. As Linda Darling- Hammond defines it: Effective teachers must understand how to present critical ideas in powerful ways, sys— tematically organize a useful learning process, and adapt instruction to the differ— ent learning styles and backgrounds of their students. Expert teachers need to be alert diagnosticians and flexible planners who teach in reciprocal relationship to their stu— dents’ learning progress. 0 Use what we now know about the status of teaching in California to address the serious shortage of teachers, especially in poor urban and rural areas. There is an immense shortage of qualified teachers that will not be solved in the foreseeable future without significant changes in the teacher preparation process. There is a disparity of 2000 access to quality teachers that affects inner city and rural children adversely. The steps above speak to the major teacher workforce problems in California’s schools today. But there are signs of hope, too: public understanding of the need for quality teach— ers; a willingness on the part of policymakers Teacher Quality to take the necessary steps; and bright pockets of innovations throughout the state. Remaining to be done is the hard work of creating a unified and effective system that will guarantee all children a high—quality teacher. Now, more than ever, California’s children need this guarantee. Notes 1. This poll was conducted by Recruiting New Teachers Inc. in conjunction with the Louis Harris polling organiza- tion. On behalf of the Center for the Future ofTeaching and Learning, the nationwide poll was “oversampled” in California so that state—specific results could be obtained. 2. Kati Haycock, “Good 'l‘eaching Matters...A Lot," Thinking K—I6, Vol. 3, Issue 3 (1998), l. 3. Linda Darling—Hammond, Doing lV/mt Alarms Mort: [matting in Quality Yi’zlrbing. prepared for the National Commission on Teaching and America’s Future, (November 1997), 9. 4. Haycock 1998, 4. 5. Los Angeles County Office of Education, Teacher Quality and Early Reading Achievement in Los Angeles County Public Schools 1998 (May 1999). 6. Darling-Hammond1997. 7. Darling—Hammond 1997, 11—14. 8. David Haselkorn, Louis Harris and Elizabeth Fideler, The Ewemial PM; Zitx‘ion (Recruiting New Teachers Inc., 1998). 9. Haselkorn, 1998, 4. 10. Patrick .\I. Shields, Viki M. Young, Julie A. Marsh and Camille Esch, The Supply and Demand of Teachers for California’s Classrooms, prepared for the Task force on leaching and California’s Future (April 1999). 11. Linda Darling—I Iammond, “Doing \‘Vhat Matters Most: Investing in Quality Teaching,” a seminar sponsored by the California Education Policy Seminar and the California State University Institute for Education Reform, (A’Iay 1998), 16-17. h. 13. Haycock, 1998, 7. 2. Data developed by SRI International for the ’Iask force on Teaching and California’s Future. PACE I 111 Chapter 7 Student Assessment and Student Achievement in the California Public School System Joan L. Herman, Richard S. Brown and Eva L. Baker National Center for Research on Evaluation, Standards, and Student Testing (CHESS?) University of California, Los Ange/es Introduction M ore than 15 years ago, a prominent national commission declared us a nation at educational risk, noting (l rising tide (gfmedior— rify f/Jflf dire/Helix our veryfinure 115 {I nation ...‘ A decade later, California received its own spe— cial wake—up call when results from the 1990 and 1993 National Assessments of Educational Progress” state—by—state comparisons revealed that California students were scoring near the bottom nationally in eighth—grade mathematics and fourth—grade reading. California students surpassed only those in NIississippi, \Vashington, DC, and the Virgin Islands on the 1992 reading assessment. What of the situation today? How are California’s students faring? Are our students making progress toward the rigorous standards that have been established for their performance? Are our schools improv— ing? Are they better preparing our students for future success? As we strive toward excellence, who is being helped most and who the least by California’s educational system? Such seemingly simple, bottom line ques— tions are foremost in the minds of the public and its policy—makers. Yet answers are more complex to formulate, made more so by the history and current status of the state’s assess— ment system, the nature of other available indi— cators of educational quality, and the impreci— sion of all assessments. Below, we first provide a context for examining the progress of stu— dents and schools by reviewing California’s recent testing history and the state’s progress in creating a sound, standards—based assessment system. We then review available data about student performance, examining how schools are doing and the factors which most influence assessment results. We close by returning to the goals of accountability and standards by which such systems should be judged. Where California’s Assessment System Is Today and How It Got There California, as the rest of the nation, is creating statewide assessment systems intended not only to measure student learning, but to leverage its improvement. The system itself is intended as part of the reform: 1t signals what is important to teach and learn by providing specific learn— ing targets—i.e., the content of the test. The assessment also is intended to provide feedback PACE I 113 11-1 on how students are doing and thus enable school leaders to diagnose curriculum strengths and weaknesses. Coupled with sanctions and/or incentives, the assessment is expected to moti- vate educators, students and their parents to pay attention and act to improve their perfor— mance. As measurement experts have aptly put it, WYTIWYG—what you test is what you get, a phenomenon that any number of research studies have confirmed.3 How Does California’s Current Assessment System Measure Up? Put simply, California’s current system is still evolving toward a standards-based system, and the base requirements are not yet in place. As the result of the roclcy and changing story of the state’s plans over the last few years, the basic requirement for analyzing students’ progress—a consistent measure used over time—is not yet available and the system’s align— ment with state standards remains problematic. First, a short history. Beginning in 1993, the California Learning Assessment System (CLAS) was to be the primary measurement of student achievement in the state. CLAS was largely a performance—based assessment system, although it included both multiple choice and open-ended items. CLAS focused on the com— plex thinking and problem—solving aims of the state’s curriculum frameworks in place at the time. CLAS came to an early demise after just two years because of both technical quality and public credibility concerns. Following CLAS, instead of a common, statewide assessment, the state provided finan— cial incentives to school districts to select and administer assessments that best reflected their local standards. The result was a plethora of CRUCIAL ISSUES IN CALIFORNIA EDUCATION different standardized tests" being given across the state. Meanwhile, as the state embarked anew on establishing statewide standards for student performance, the testing plan changed again the next year. Impatient to establish a baseline and hold schools accountable, the state (and particularly then-Governor Wilson) initiated in 1998 the new California Standardized Testing and Reporting (STAR) program. The center- piece of STAR was and continues to be the Stanford Achievement Test Series, Ninth Edition, Form T (SAT—9) administered in grades 2 through 11. Thus, in contrast to an ideal scenario Where a testing system would be selected or developed based on a state’s stan- dards, the initial STAR test pre—dated the state’s standards. Unlike CLAS, the SAT-9, it should be noted, is a norm-referenced test, designed primarily to show how students or schools per— form on basic skills relative to others—others in the state, others in similar schools and districts, others in the national norm group (or average). California’s adoption of the SAT-9 occurred at a time when most other states were making progress in meeting federal expectations for state standards and assessments. The federal plan, originally designed in 1991-92,+ was given addi— tional impetus by the Improving America School Act,5 in which Title I, an act to support disad- vantaged students, was lodged. Title I made the receipt of funds contingent on the development of standards and assessments that met criteria, including the use of multiple measures, assess— ments for children with different language back— grounds, and measures of progress. With California’s Board of Education’s pass— ing of state standards in December 1998, plans to retrofit the testing system to the standards ZOOO Student Assessment and Student Achievement in the California Public School System began. To provide a comparable measure, the plan featured the continued administration of the same SAT—9 that had been administered in Spring 1998, but in 1999, additional items were included to bring the test into better alignment with the state’s standards. Thus was born the SAT—9 augmentation, additional items that the test publisher selected or developed to fill in some of the gaps between the existing SAT—9 and California’s content and performance stan— dards. With the augmented items, the SAT—9 would then eventually provide both norm-ref— erenced and standards—referenced scores. The norm—referenced scores would cmnmunicate to parents, the public, students, and educators how students were performing relative to other students nationally. The standards—referenced scores would tell those stakeholders if students were meeting state—defined content standards at advanced, proficient, basic, or below—basic lev— els. As we describe later in this chapter, although a first set of augmented items was administered in spring 1999, there are some questions about their appropriateness. Performance standards have not yet been estab— lished for them, so results from the augmenta— tion are not yet directly interpretable. Additional components of STAR are in the works to bring California’s assessment system into still closer alignment with the state stan— dards. The California Assessment of Applied Academic Skills (CAAAS), the so—called “Alatrix” test, is to be designed to focus on the disciplinary thinking and problem solving capa— bilities which are reflected in the standards, but not well assessed by the SAT—9. Since the SAT— 9’s multiple-choice items alone cannot assess the broad range of important thinking and communication skills, the Matrix test is to include open-ended and performance—assess— ment tasks, such as asking students to explain their thinking or write an essay. This compo- nent will model the types of learning which are expected of students and preclude an exclusive focus on “drill and kill” formats in classroom instruction that often are encouraged by multi— ple choice test formats. The Matrix test employs a matrix sampling framework where the accuracy and comprehensiveness of the assessment are improved by having some stu— dents within a school respond to some assess- ment tasks while other samples of students respond to different tasks. Given that each open—ended and performance assessment takes substantially more time to administer than a multiple choice item, matrix sampling improves the overall coverage for the school as a whole while minimizing the time each student is required to spend taking an assessment. While it has not been designed to yield a score for each student, it does provide school—level results for judging the quality of a school’s cur- riculum and instruction and students’ collective achievement and progress at that school. CAAAS currently is scheduled to go opera— tional in 2001. A high school exit exam in language arts and mathematics is the most recent addition to the state’s standards—based assessment arsenal. Enacted as part of Governor Davis’ first 100 days education agenda, the exit exam will be required for high school graduation and is scheduled to go operational in 2004. An English Language Development Test also is under development (see the chapter on English Learners in this publication). PACE I 115 116 Academic Performance Index The components of the STAR program thus are abundant and in fact, there is continuing debate about whether the assessment load is too high and whether all planned components are necessary to achieve the system’s goals. Even as the system components are under dis— cussion and development, California already has developed a high stakes school performance index based on them. The Academic Performance Index (API) is being used to rank schools across the state based on their SAT-9 test scores. More will be described about the API in our concluding section. The Assessment Context: Setting the Course with a System in Transition As we’ve noted, the ideal assessment system is developed after a state’s content and perfor- mance standards are established in order to be in alignment with what students are learning. Since California’s assessment system followed a differ- ent course, it remains a system in transition. The Importance of Alignment The alignment between what is tested and what students are expected to learn is a critical External Assessment System Assessments Performance Standards Standards‘i" .1 Learning Results p . Results ' School/Classroom Practice Curriculum Instructional ActiVItIes Assessment/Evidence Actual Learning Figure 1. Model of Standards-Based System CRUCIAL ISSUES IN CALIFORNIA EDUCATION 2000 Student Assessment and Student Achievement in the California Public School System criterion for any assessment or accountability system intended to promote the improvement of student learning and is the essence of cur— rent standards—based reform. As displayed in Figure 1, the idea is not really to teach to the test per se, but rather that both testing and teaching reflect the standards we hold for stu— dent performance. VVhen standards, testing and instruction are in synchrony, the logic of the system works to leverage better perfor— mance. \Vhen not, then holding schools accountable and encouraging them to use the assessment results may not promote the stan— dards we seek. Consider, for example, the case where the assessment doesn’t well reflect the standards. Under pressure to show improvement, schools and teachers may use test results to modify their curriculum and instruction, but moving toward the test does not mean movement toward the standards. Minimally, the test and the standards are sending conflicting messages, which can cause confusion and dilute the focus of school efforts. Or consider where there is a poor match between what is taught and what is assessed. Here, while the results may tell us about gaps in the curriculum, they tell us little about the quality of instruction and teaching in that school. Even under the best scenario, as Figure 1 portrays, assessment results reflect only a portion of what students have learned and what they know and can do. In other words, the test is a reflection of standards and goals, it is not the goal itself. Characteristics of Quality, Standards-Based Assessment Systems 0 Alignment. Does the assessment reflect content and performance standards that have been estab- lished for students? is the assessment content consistent with the best current understanding of the subject matter? Do it reflect the enduring themes and/or priority principles, concepts and topics of the discipline? 0 Instructional sensitivity. Can the test detect differences in the quality of instruction? Does the test measure learnable and teachable knowledge, rather than simply general factors such as general ability or language background? 0 Technical quality. To what extent are results reliable and consistent? Comparable over time and setting? Do the results enable accurate generalizations about student learning and achievement rel- ative to standards? 0 Fairness. Does the assessment enable students, regardless of race, ethnicity, gender or economic status, to show what they know and can do? Have students had the opportunity to learn what‘s being assessed? 0 Meaningfulness. Do parents, teachers, students and the public find the assessment worthwhile and credible? 0 Consequences. To what extent do the assessments model and encourage good teaching prac- tice? Are intended positive consequences achieved? What are the unintended negative conse— quences? 0 Multiple Measures. Does the mix of measures optimize alignment, technical quality, fairness, meaningfulness and consequences criteria? * Adapted from CRESST Criteria for Evaluating Assessment Quality (Linn, Baker and Dunbar, 1991) (National Center for Research on Evaluation, Standards, and Student Testing, Los Angeles, UCLA) Figure 2. Characteristics of Quality Standards-Based Assessment Systems P A (I E I 11 ,7 / 118 Other Criteria for Quality Standards-Based Assessment Systems How well the results of an assessment system represent student learning is a complex validity issue and one which has driven traditional con— cerns for technical quality. One asks about the reliability, accuracy and consistency of measure- ment, at the same time acknowledging that there is error in any measure and that all tests are falli— ble—some more than others. But even alignment and indices of technical quality provide an inade- quate base for evaluating the soundness of any assessment system. History shows that a number of other features of assessments are important to a quality system, the major ones of which are summarized in Figure 3. Consider the importance of instructional sensitivity. If the assessment does not measure efforts made in the classroom—even if it nomi— nally “matches” standards—it will be a poor device to provide feedback for improvement. Instead, scores will misrepresent the reality of serious educational reform. They may indicate improvement that might happen spontaneously with or without reform. Note also the final characteristic in our list—multiple measures—which is necessaly to achieve the other listed criteria. It is unlikely that a single measure can adequately capture our goals for student performance or enable all students to show what they know. Some types of measures are efficient and cost effective for some purpos— es but have unintended consequences for other purposes. For example, multiple—choice tests can be highly efficient, cost effective, and reli— able, but an over—reliance on such testing in the 1980s led to a narrowing of the curriculum to basic skills and an overemphasis on “drill and kill" types of instruction.“ Different constituen— (ZRL'CIAL lSSL'lis IN CALIFORNIA EDL‘CA’I‘IOX cies, furthermore, find different types of infor— mation meaningful and useful. For example, basic skills are high among the public’s priori— ties and parents and the public often want to know how their children compare with oth- ers—nationally and internationally. Educational reformers and futurists, on the other hand, emphasize the importance of all children achieving high levels of skill in communication, problem solving and ability to learn and change—abilities which may not be well assessed through multiple choice testing. Assessing Limited English Proficient Students While state code requires all students in grades 2 through 11—including those who are not fully proficient in English—to take the SAT-9f it also provides that limited English proficient (LEP)‘ students who have been in school less than 12 months also be tested in their primary language. Students who have been in school more than 12 months but are still classified as LEP may also be administered a primary lan— guage test. The state has selected the Spanish Assessment of Basic Education, Second Edition, (SABE/Z) as the statewide measure to be used for assessing students whose first language is Spanish, which begins this year. Currently, then, different districts are using different measures so it is not possible to know statewide how Spanish—language students are doing based on tests in their primaiy language. The assessment of LEP students continues to be highly controversial. On the one hand, test— ing students in a language they do not under— stand does not allow them to show what they know and can do in content areas such as math and science, raising questions about the extent to which fir/1710‘s criteria are being met in the state's ZOOO Student Assessment and Student Achievement in the California Public School System system. On the other hand, it is important that LEP students’ achievement and progress be monitored in publicly visible ways, and that schools be held accountable for all their stu- dents. The com‘eqnenres of not testing and report— ing LEP students’ performance is that their progress and their needs may be ignored. Testing in students’ primary language at first glance might seem a better and fairer option. However, research shows that primary language testing only helps those students who have been instructed in their native language"—a cir- cumstance which current education code pro- hibits for LEP students who have been in this country for more than a year. Statewide testing of English language proficiency will soon enable the state to at least monitor LEP stu— dents’ progress in acquiring English, providing another measure that is potentially more sensi— tive to individual students’ achievement and progress. Testing accommodations which attempt to reduce the language load of a test or otherwise compensate for students’ reduced language skills (e.g., allowing students more time to take tests) also are currently being researched, but a solution that is equitable and fair for all students has not yet been found. Measurement experts, however, largely agree that test results of LEP students should be sep— arated from those of English proficient stu— dents, and that the validity and utility of indi— vidual scores for LEP students on English lan— guage exams is limited.” Other Indicators of Quality Beyond the components of STAR, there are other statewide indicators that can be used to judge the quality of student performance." As mentioned above, multiple indicators are important to a balanced and valid View of any educational system. Some of these indicators act as counterbalances to others and are partic- ularly relevant for different sub-populations. For example, the high school drop-out rate is of interest in itself, but also to assure that schools are not achieving higher test scores at the cost of more children leaving the system. Advanced placement exams, which are given to high school students who take college-level courses at their high schools through the College Board,” provide an indicator of how schools are serving their highest—ability stu- dents. As described further below, both the number of exams taken and the proportion passing are of interest. Similarly, college entrance exams, such as the SAT, provide an indicator of both students’ expectations and preparation to attend college. Other indicators are external to the K-12 system and provide a validity check of its acade- mic quality. The National Assessment of Educational Progress (NAEP) periodically assesses national performance in the major sub- ject areas—reading, mathematics, science, writ— ing, etc. States participating in NAEP’s state— by-state program are able to compare their per— formance to that of other states as well as nationally. College placement tests, which are used to decide whether entering college stu— dents have adequate mathematics and writing skills to handle college coursework or need remedial help, provide another external com— parison point for judging the quality of the states’ pre—collegiate systems. Alignment and Consistency The alignment of these various indicators of student performance is an issue under current PACE I 119 120 discussion. Some believe that college entry tests, such as the SAT, and college placement tests ought to be aligned with the state’s stan— dards and with the state’s K—12 assessment sys- tem. Advocates believe that this would not only provide greater consistency and focus to California schools but would permit greater efficiency in testing. For example, they project scenarios where the state’s graduation tests would serve a role in the college selection and placement process. Consistency and alignment of each of these indicators with state standards aside, one looks for consistency in performance across various indicators to judge the quality of California’s academic achievement. Although any individual indicator is flawed, when multiple indicators show consistent direction, we can be more con— fident of the breadth of our perspective and the validity of our conclusions. We now turn to a consideration of those indicators. Student Achievement in California Public Schools A serious understanding of student performance in California requires in-depth knowledge of the wide variety of student achievement mea— sures we’ve outlined thus far. In the next few pages, we’ll describe those instruments, what they are intended to measure or monitor, and how well California schoolchildren are doing on them. VVe’ll review data from both the most recent testing period and over a longer period of time to help the reader understand the status and progress of California performance. CRUCIAL lssuus IN CALIFORNIA EDUCATION We’ll begin with a look at the state’s stan- dardized testing system, the program that applies to all students in the public educational system from elementary school through high school. Next we will analyze information regarding California’s performance on NAEP. From there, we’ll examine the results of a series of secondary school measures, including high school drop—out and graduation rates, advanced placement (AP) test results, course-taking pat- terns, and college entrance examination perfor- mance. To address the longer-term impact of public school, we will also present data on col— lege attendance and preparedness by consider- ing findings on reading remediation tests for college freshmen in the University of California system. Finally, we’ll comment on some of the demographic trends for California students over the last decade and venture a summary judgment across this collection of information on what the state of academic achievement in our California public schools is and whether there is evidence it is headed in the right direction. STAR Results As we’ve noted, California began to implement STAR in 1998 with the SAT—9. In the sections below, we’ll look at how well California students performed on the norm—referenced SAT—9 in reading, mathematics, language arts, spelling, science, and social studies in the 1997398 and 1998—’99 academic years, with some words of caution about the interpretation of the scores. We will follow with analyses of the perfor— mances of LEP students and students who are economically dis-.ulvantaged, and compare how ZOOO Student Assessment and Student Achievement in the California Public School System the performance gaps between these groups and others vary across different school contexts. How are California’s students doing on the SAT-9P Before examining how California students are doing overall, it would do us well to review what the results from a norm—referenced test mean. The results tell only generally what stu- dents know and can do. The real information they provide is how California students’ perfor— mance compares with that of a national norm— ing group. Results often are reported in terms of percentile scores, which reflect where stu— dents’ scores fall relative to the national distrib— ution. For example, if a student scores at the 40‘“ percentile, it means that the student’s per- formance equaled or exceeded 40% of the national norm group. A score at the 50‘h per— centile—which the public often considers “average”—means that the student’s perfor— mance equaled or exceeded half the national norm group. Thus the nature of percentile scores means that some students will be above and some below the “average” relative to the normng group. The national norming group is intended to represent students nationally. Ideally, for norms to be interpreted easily, the kinds of students tested in a particular state would be similar to those in the norming group. In the case of California, interpretation of the test results is difficult for a number of reasons. First, while no tested and norming groups are ever exactly alike, California’s student population differs substantially from the national norm group in its diversity and its urban concentrations. Plus, unlike other states, California assesses virtually all ofits students using an English—language examination, even though approximately a quarter of them are not fully proficient in English. It is not hard to predict that students who do not understand English are likely to fare poorly when compared to a national norm group consisting of only two percent of similar— 1y non—English proficient students. Thus, when we look on average at the results of all California students, it is not sur- prising to find that California students score below average (50th percentile) in practically all subject areas and in almost all grade levels com— pared to the national norm group. On the read— ing tests for grades 2 through 11, scores ranged from the 32nd to the 44th percentile in 1998, and from the 32nd to the 46th percentile in 1999. Average scores failed to exceed the 5 0th percentile at any grade level in either year, and performance shows a precipitous drop at the high school level.” Observed scores were somewhat better in mathematics, where scores ranged from the 39th to the 50th percentile (grade 9) in 1998, and from the 44th to the 52nd percentile in 1999. For 1998, only grade 9 showed average scores above the 50th percentile. In 1999, grades 2, 6, and 9 showed aver- age scores above the 50th percentile. In all other grades, average performance for California stu— dents was lower than the national average. The subject areas of language arts and spelling showed similar levels of performance. In language arts, only one grade level (grade 7) exceeded the national average in 1999. None did so in 1998. For spelling, no grade levels surpassed the 50th percentile in either year. Similarly, none of the three grade levels (grades 9—1 1) taking the science test demonstrat— ed average performance above the 5 0th per— centile in 1998 or 1999. In social studies, only PACE I 121 100 90 80 70 60 5O 4O 30— 20— 10— Percentile Rank of Mean Normal Curve Equivalent Grade 2 Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 9 Grade 10 Grade 11 I 1997-98 1998-99 Figure 3. SAT-9 Reading Scores 1:9 100 g 5 90 E a, 80 S o 70 E 60 E C 50 8 E 40 _ “5 x 30 __ C as CC 20 _ 2 E 10 _ (D 9 ea 0 ' Grade2 Grade3 Grade4 Grade5 Grade6 Grade? Grade8 Grade9 GradeiO Grade11 .1997-98 , 1998-99 Figure 4. SAT-9 Math Scores 122 CRUCIAL ISSUES IN CALIFORNIA EDUCATION 2000 Student Assessment and Student Achievement in the California Public School System 100 null—fin 90 1 80 il'uxfi 3—D!” ii 70 u l 60 -‘!~.LLl- 50 3 , 4O 30— 20_ 10— 3% O Percentile Rank of Mean Klormal Curve Equivalent Grade2 Grade3 Grade4 Grade5 Gra .1997-98 1998-99 de 6 Grade 7 Grade 8 Grade 9 Grade 10 Grade 11 , 7 ’igure 5. SAT-9 Language Scores _l O O 80 70 6O 50 4O 3O 20 10 Percentile Rank of Mean Normal Curve Equivalent Grade 2 Grade 3 Grade 4 Grade 5 Grade 6 I 1997-98 1998-99 Grade 7 Grade 8 Figure 6. SAT-9 Spelling Scores PACE A 123 12-1 E 100 9 g 90 3 o- UJ 80 32’ ,3 70 a E 60 o z 50 C 8 40 E ‘5 x 30 5 q: 20 2 E 10 o O 5 0 1 .1997-98 .1998-99 Grade 9 Grade 10 Grade 11 Figure 7. SAT-9 Science Scores E» 100 (U .2 3 90 0' Lu 0 80 E 3 O 70 R E 60 O 2 c 50 (U Q) E 40 “6 g 30 6:“ g 20 8 10 8 6‘8 0 .1997-98 @1998-99 Grade 9 Grade Grade 11 Figure 8. SAT—9 Social Studies Scores CRUCIAL ISSUES 1N CALIFORNIA EDUCATION 2000 grade 11 showed average performance above the national average, doing so in both 1998 and 1999. Average grade 9 performance in social studies came in at the 42nd and 43rd percentiles in 1998 and 1999, respectively. Average grade 1() performance was lower, reaching only the 38th percentile in both years. How are California’s English-proficient stu- dents doing on the SAT-9? One gets a slightly different picture, however, from looking solely at the results of California students who are fully proficient in English, a comparison that somewhat favors California stu- dents, since approximately two percent of the national norm group is not proficient. Here, the 1999 results show that California’s English—pro- ficient students are generally scoring at or above the national average. Differences between all students and English—only students are most pronounced in reading, as we might expect, at the elementary school level (grades 2—5). Yet student performance is still the best, relatively, in mathematics. And regardless of the compari— son group, California students are performing the poorest, relatively, in spelling at the elemen— tary school level and in science and social stud— ies at the high school level. Are Califomia’s schools improving? Comparisons between scores from the initial year (1998) of STAR and the most recent year (1999) are inevitable. Many claims of “improve— ment” or “progress” have been made based on such comparisons. However, a word or two of caution should be issued. First, observed test scoresH are not without error. That is, observed scores—the score students receive when they take a test—do not exactly represent their actual or true capability, due to errors of measurement. Student Assessment and Student Achievement in the California Public School System Percentile Rank of Mean Normal Curve Equivalent 100 90 80 7O 60 50— 40—. 30— 20— 10— O I English Only I Total 53 43 i i l, l-' 1 Grade 2 Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 9 Grade 10 Grade 11 Figure 9. SAT-9 Reading—All Students versus English Proficient Students Percentile Rank of Mean Normal Curve Equivalent 100 90 80 70 60 50— 40— 30— 20— 10—— 0 I English Only I Total 6O Grade 2 Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 9 Grade 10 Grade 11 Figure 10. SAT-9 Math-All Students versus English Proficient Students PACE I 135 100 90 80 70 60 59 50— 40— 30— 20.... ‘IO_ Percentile Rank of Mean Normal Curve Equivalent O Grade 3‘ Gra Grade Gradew Grad Grad Grad Gdrae11 I English Only I Total Figure 11. SAT-9 Language—All Students versus English Proficient Students _L O O LO Q 80 7O 60 Percentile Rank of Mean Normal Curve Equivalent Grade 2 Grade 3 Grade 4 Grade 5 Grade 6‘ Grade I English Only Figure 12. SAT-9 Spelling—All Students versus English Proficient Students 126 CRUCIAL lssuns 1N CALIFORNIA EDUCATION 2000 Student Assessment and Student Achievement in the California Public School System E 2 100 m .2 g. 90 w a) E 80 3 2 70 E 5 60 2 g 50 a) E 40 __ o ‘6 30 __ «s cc 9 20 __ 1% 8 10 _ a: Q 0 , , ....'.,< 17 Grade 9 Grade 10 Grade 11 - English Only I Total Figure 13. SAT-9 Science—All Students versus English Proficient Students 05,100 ‘1’ '5 90 o- LIJ o 80 S o 70 To E60 E C 50 cu m E 40__ “6 x 30__ C E 2 20.— $10— 2 ii’ 0 Grade 10 Grade 9 Grade 11 I English Only 3 Total Figure 14. SAT-9 Social Studies—All Students versus English Proficient Students The magnitude of this error varies, partly as a function of test reliability. One issue in inter— preting these norm—referenced test scores as indicators of student or group achievement (or progress), thus, is how accurately the observed scores represent students’ true achievement. Recent work by Stanford professor and CRESST researcher David Rogosa addresses this accuracy issue. In addition to technical reports that may be too complicated for the average citizen, Dr. Rogosa has created an easy- to—read guide titled, “How Accurate are the STAR National Percentile Rank Scores for Individual StudentSF—An Interpretive Guide.” The results of this work will surprise many. Although most of the results are presented in the form of tables of data, the guide does pro- vide a few samples in the form of responses to hypothetical questions. For example, the guide poses the question, “\Vhat are the chances that a ninth—grade math student whose actual capabili— ty or true score is at the 50th percentile of the norm group obtains a score more than five per— centile points away from the 50th percentile?" The answer—70 percentl That is, there is only a 30 percent chance that the observed score is between the 45th and 55th percentile points. \Vith respect to interpreting progress, Rogosa’s guide also provides calculations for the probabilities of certain increases or decreases for students whose true percentile ranks remain constant from one year to the next. In one example, a ninth grade math student who is actually at the ()0th percentile in both years has a greater than 50 percent chance of showing at least a ten percentile point change (up or down) in the second year! To state it differently, for this case it is more likely than not that student \\'l1()SC tl'UC SCOI’C ilCtLl'Jlly I‘CIH‘AIHS the S’JIUC PACE I 137 128 from one year to the next will result in an observed score difference of more than ten per- centile points. Given this level of imprecision in interpreting scores from one year to the next, it is advisable not to make too much of observed score differences, especially minor ones. While it is recognized that these analyses are based upon less precise student level scores and not state aggregates, it is nonetheless worthwhile to consider the issue of accuracy when utilizing standardized test scores to ren- der important judgments. Beyond the precision issues, there are also questions of the extent to which scores from one year to the next may be inflated by test preparation practices. That is, research suggests that under pressure to show improvement in test scores, teachers bring their curriculum more and more in line with just what’s on the test and not the broader domain the test is intended to measure. They also are likely to spend substantial time on test preparation. Thus, the extent to which gains reflect real improvement in learning is an open question.M Accuracy considerations aside, another issue to consider in comparing 1998 to 1999 observed scores is how progress is gauged. For assessing school-level progress, does it matter whether comparisons are made between mean— scaled scores or between the percentage of sm— dents scoring above a specified score point—two different ways of portraying “average” perfor— mance? And whose performance should be compared? What about comparing the perfor— mance of third graders in 1998 with the perfor— mance of third graders in 1999—commonly called cross sectional comparisons? Or should last year’s third grade performance be com— pared with the performance of fourth graders in 1999, an attempt to monitor the same group of students from one grade to the next? Does it make a difference? A series of school-level analyses conducted by researchers at CRESST indicates there is rather low agreement between the rankings of schools using these two different methods of assessing change. Thus, it matters which method is used if schools are to be ranked as a result of their performance on those year-to- year comparisons. In other words, school rank- ings differed dramatically depending on whether average performance was compared from one year to the next based on grade level (e.g., the third grade in both 1998 and 1999) or on student cohort (e.g., second grade in 1998 and third grade in 1999). Quintile rankings across these two approaches agreed only about a third of the time (see Figures 15). This find— ing held across the different types of test scores (mean-scaled score, percentile rank of the mean normal curve equivalent, and percent scoring about the 50th percentile) and subject areas (reading, mathematics, language arts, and spelling), although only reading results are pro- vided here.IT Such inconsistency in rankings across methods advises thoughtful considera— tion of the method and what it purports to measure before placing much significance on the results. How did students perform on the STAR augmentation? The rather poor showing by California stu— dents on the norm-referenced portion of the STAR system in 1998 has been attributed to many factors. As we’ve noted, one of the more widely discussed issues is the lack of alignment between the subject matter assessed by the CRUCIAL ISSUES IN CALIFORNIA EDUCATION 2000 Student Assessment and Student Achievement in the California Public School System SAT—9 test and what was being taught in the public schools. In an effort to better align the assessment with what is outlined in the state content and performance standards, an aug— mented version of the SAT—9 was created for the subject areas of English and mathematics.” It is difficult to interpret student performance on the augmented test since the state has yet to identify what constitutes various performance levels. However the general consensus was that the tests—administered in the spring of 1999—sampled the more difficult elements of the state’s standards, and student performance was very low. In most grade levels, students on average correctly answered about half of the items on the English test (see Figure 16). Generally, the percentage of correct answers was lower on the math test at each grade level (see Figure 17), with better performance in the lower grades.” Reports at open testimony at the October meeting of the California State Board of Education (1999) recounted anecdotes of students confronted with problems in math- ematics far beyond their capability. It is important to point out again that exact— ly what constitutes adequate or sufficient per— formance is undetermined at this time. Thus, not much should be made of student perfor— mance on the augmented tests until adequate performance standards are established and veri— fied. Of more concern is the content sampling model used for the augmentation examinations, particularly since they are now termed the “standards—based” element of the STAR.” It will be important to follow the extent to which these particular tests are curriculum referenced and thus will reflect appropriate Classroom instruction. Percent of Exact Ageement 100 90 80 70 60 50 4O 30 20 1O O Grade 2 Grade 3 Grade 4 Grade 5 I Mean Scaled Scores I Mean Percentile Scores PAC 50 Elem Grade 6 Grade 6 Grade 7 Grade 8 Grade 9 Grade 10 Middle Figure 15. SAT-9 Reading Quintile Agreements PACE I 139 100 90 80 70 60 54.93 54.8 56-89 57-“ 55.78 5033 51.11 53.33 53.11 53.67 50 4O 30 Average Percent Coriect 20 1O Grade 2 Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 9 Grade 10 Grade 11 Figure 16. SAT-9 Augmented English 100 90 00 O 7O 60 54.4 53 40 41.6 44.2 41 30 Average Percent Correct 20 1O Grade 2 Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Figure 17. SAT-9 Augmented Math 130 (:RL'HAL ISSL'ES 1.\‘ CALIFORNIA FDL‘ttx'rlox 3000 Student Assessment and Student Achievement in the California Public School System How is school composition related to a school’s SAT-9 performance? On average, California students scored below the national average on the norm-referenced portion of the SAT—9. But clearly this finding does not imply that all students in the state are performing poorly. In fact, many schools and districts showed exceptionally high levels of average performance. Usually, these schools and districts are those that are challenged least with the forces of poverty and limited English proficiency. Simply stated, California school— children with limited English skills and those from economically disadvantaged backgrounds tend to score lower on the state’s standardized test than students with English fluency or those from economically advantaged backgrounds. This relationship is even greater where the concentration of disadvantaged students Percent Scoring at or Above the 50th Percentile 100 90 80 70 60 57 50 .\ 40 \38 30 30 27 20 18 ‘\\,_\_ 12 9 10 was- flu“ g 6 T " Q. 0 0—25 26—50 51 -75 76—100 Percentage of School Designated as LEP I LEP I Non-LEP increases. Schools with high proportions of stu- dents receiving free or reduced—price lunch score considerably lower than schools with lower proportions of such students. Interestingly, the relationship holds for both economically disadvantaged and the more advantaged students at a particular school. That is, the average score for both groups of students tends to be lower in schools were there are high concentrations of poverty. Therefore, it appears the extent to which a school confronts the challenges of teaching impoverished chil— dren may affect not just the performance of poorer students, but of all students. The same result was found for limited English proficient (LEP) students. The average performance of both LEP and non—LEP stu— dents is lower in schools with higher concen— trations of LEP students. Thus, as in dealing g 100 ‘5 e 90 CD CL 80 5 § 70 66 .C E 60 \ 57 > O c 42 \ 3 40 L w 33 \ 07 “13..., 28 g 30 \\\\2:: \. 8 20 ‘\ \ 15 +-' \- § 10 a ‘1 0 0-25 26—50 5175 76-100 Percentage of School on Free/ Reduced Lunch I Economically Advantaged - Economically Disadvantaged Figure 18. SAT-9 Grade 3 Reading LEP vs. Non-LEP Figure 19. SAT-9 Grade 3 Reading Economically Disadvantaged vs. Economically Advantaged PAC F. I 1, j 1 with poverty, it appears the extent to which a school confronts the challenges of instructing children with limited English skills affects not only the performance of those students strug- gling to learn the language, but also the perfor- mance of students with sufficient English skills. The observed relationship between language proficiency, poverty, and achievement on test scores is not surprising, and, as mentioned above, partly explains the relatively low overall average achievement of students in California. Since the SAT-9 norm group and the California student population differ dramatically on these key measures, lower average performance for California students as a whole relative to the normative group should be expected. Figure 18 graphically presents how the average perfor- mance varies for both those with and without sufficient language skills}1 When there is a high proportion of LEP students at a local school site, all students perform at lower levels. Figure 19 illustrates similar findings for the problem of student poverty?2 The relationship between language profi— ciency, economic status, and test scores may not be as direct and clear, however, for students who are identified by our analyses as economi— cally advantaged and/ or fully English proficient because of the limits of the variables available to us. Clearly, those who are not eligible for free or reduced—price lunch (the advantaged or “non—disadvantaged” group in our analyses) represent a large range of socio—economic sta— tus (SES), from students whose families are just on the margin of qualification to those whose families reflect a very high level of SES. It may be the case that the relatively more advantaged students in schools that have high proportions of impoverished students are different from and CRUCIAL Issues IN CALIFORNIA EDUCATION relatively less economically advantaged than those who are in schools with low proportions of children qualifying for free or reduced—price lunch. It may well be that these actual SES dif— ferences account for the differences in “non- economically disadvantaged” groups across the different types of schools. Similar conclusions could be drawn for dif- ferences between the non—LEP population in schools serving a large proportion of LEP stu- dents compared to those that serve few or no LEP students. In the former case, a large pro- portion may be non—native English speakers who have relatively recently transitioned to English proficiency, but whose English lan- guage skills still are not totally secure; poverty may be another intervening variable. And it may be that it is these differences in the nature of the non—LEP group across the various types of schools that cause the observed performance differences. In any event, the relationship between school composition and the perfor— mance of different subgroups is vitally impor— tant and merits additional scrutiny. NAEP Results The National Assessment of Educational Progress (NAEP) is a federal effort at a nation- wide assessment of educational achievement, conducted every few years nationally and including state—by—state comparisons in recent years for most states across the country. Generally, California students have performed poorly compared with the rest of the country. For instance, for the 1996 assessment of eighth-grade mathematics, California ranked 3lst out of 41 states. The state did even worse in fourth—grade reading, coming in dead last out of 38 states. As indicated in Figures 20—23, 2000 Student Assessment and Student Achievement in the California Public School System Figure 20. NAEP Grade 4 Reading 1992, 1994, and 1998 50 50 *5 4a .9 8 o .— ._ o '5 30 =5 ii _ 30 (D Q > a) .8 S < 20 .Q a E 20 *5 O E F: a) 4‘ o 10 C 5 § 10 0' a) Q o 1992 1994 1998 0 I Nation I Nation 1998 I California a California Figure 21. NAEP Grade 8 Reading 1998 50 40 30 20 1O Percent at or Above Proficient 11f 1992 1996” I Nation I California Figure 22. NAEP Grade 4 Math 1992 and 1996 50 4O 30 20 10 Percent at or Above Proficient I Nation 1996' 1992 1996 California Figure 23. NAEP Grade 8 Math 1990, 1992, and 1 996 1’1\ (2 1i i 133 134 Percent at or Above Proficient 50 4O 30 2O 1O O I Nation I California 1 998 Figure 24. NAEP Grade 8 Writing 1998 California lags the nation in grades 4 and 8 in both reading and mathematics achievement. Only 17 percent of California students per— formed at the proficient level in eighth—grade mathematics and 11 percent achieved that stan— dard in fourth-grade mathematics—both of which are much lower than the national rates. Similarly, in eighth—grade writing, only one in five California students achieved at or above the proficient level, compared to one in four nationally (see Figure 24). Clearly, California students’ performance does not compare favor— ably to either the national sample or the stan— dard of proficient performance. Comparisons often provide a clear way to understand the meaning of performance. One way to understand California’s NAEP perfor— mance is to compare it to other states with sim— ilar characteristics. For example, with relation to poverty, 16.5 percent of California schools in the 1992 NAEP reading sample showed 75 per— CRUCIAL ISSUES IN CALIFORNIA EDLmA'i‘ION cent or more students on free or reduced-price lunch, and in the 1994 assessment, the figure was 16.6 percent of the California school sam- ple.“ In those two assessments, only 12.7 per- cent of those sampled in poverty-stricken schools scored at or above basic (the lowest level of achievement) in 1992, increasing to only 14.8 percent in 1994. Looking only at 1994 findings, ten states had higher percent— ages of schools in poverty than California. All of these states—Alabama, Arizona, Florida, Georgia, Louisiana, Mississippi, New York, New Mexico, South Carolina, and Texas—had higher proportions of disadvantaged students reaching the basic level than did California. In fact, some states with significantly higher pro- portions of schools in poverty, for example Georgia with 22.3 percent, Mississippi with 39 percent, and New Mexico with 26 percent, were substantially superior to California on this metric (Georgia and Nlississippi with 29 per- cent scoring at basic and above, and New Mexico with 32 percent of students scoring at basic or above). Only one entity, the District of Columbia with about 62 percent of the schools meeting this poverty definition, scored below California, at 13.9 percent. Even so, the’ District of Columbia is doing a better job pro- portionally for its students when one looks at poverty and performance conjointly. These numbers show that the US. overall has a long way to go in educating its poor students, and California is clearly lagging behind the countly. 1n mathematics, the situation is comparable for the 1996 data. Twenty—one states have high— er proportions of impoverished students than California and of these only the District of Columbia performed more poorly. For exam— ple, West Virginia with 29.7 percent poverty ZOOO Student Assessment and Student Achievement in the California Public School System had more than 60 percent of its students reach- ing or exceeding the basic level in mathematics. Not all of the news from the NAEP assess— ment in California is bad. Other NAEP perfor- mance data” indicate the performance of low- income students in fourth—grade math is increas— ing. Education \Vatch reports a 7.8 percentage point increase in the number of these students scoring at or above the basic performance level from 1992 to 1996. In terms of cohort growth, furthermore, when one examines how fourth— graders performed on the 1992 mathematics assessment compared to the same cohort as eighth-graders in 1996, we find California in the top third of the states on this progress measure.“ Clearly, California needs to continue to make progress and has a long way to go. Drop-Out/Gmduation Rates“ Despite the extensive focus placed on standard- ized test scores, other indicators of student per- formance have been collected and will be incor— porated into the state’s accountability index at some undetermined future date. Two of them are the drop-out rates and completion rates for high school students. Definitions of dropouts often vary. California officially defines a dropout as a student at or above seventh grade who misses school for 45 consecutive days and does not enroll in another school. School completion rates tell us the proportion of high school seniors who graduate relative to those enrolled at the beginning of the year. Both of these indi— cators represent important ends in themselves, but also enable us to assure that improvements in test scores are not coming at the expense of more students being pushed out of school. 100 90 8O 70 60 50 4O 3O 20 10 6.62 6.38 5.97 5.30 4.76 4.29 4.14 3.72 3.26 2.76 O 0 #g + + + r . + 4 1995—96 1996—97 1997-98 1988-89 1989-90 1990—91 1991 —92 1992—93 1993—94 1994-95 Figure 26. California High School Dropout Rates 1989-98 PACE I 135 136 Percent 100 90 92.86 93.79 94.16 91.95 80 m 91.67 ¥- 4 91.18 89.86 90.33 90.54 89.71 70 60 50 40 30 20 10 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 Unfortunately, data regarding these two indicators are often unreliable or inaccurate because schools across the state do not use uni— form definitions or share equally careful proce— dures for collecting the data. Poor data man— agement may record students as dropouts when they have simply moved their home, or dropped out and then returned, after an extended hiatus. California is moving to a statewide student data system that will permit more precise understanding of these indicators. Nonetheless, in Figures 35 and 26, we use data from the California Department of Education to present ten-year trend lines of drop-out and graduation rates for California high school stu— dents. Drop—out rates are steadily declining and have done so in each year of the period. Graduation rates, on the other hand, have remained fairly stable, at around the 90 percent to 91 percent, though the rate was a few points higher at the beginning of the decade. CRL'CIAL ISSL'ES IN CALIFORNIA EDL‘CATION Figure 26. California High School Graduation Rates 1989-1998 High School Course-Taking Patterns 3‘ In California, high school students may choose to take a series of courses specifically defined to meet the University of California and California State University entrance require— ments. These courses include the following: 0 I'Iistory/Social Science—two years required. ' English—four years required. ' IVlathenmtics—three years required, four years recommended. ' Laboratory Science—two years required, three years recommended. ° Language Other than English—two years required, three years recommended. ° College Preparatory Electives—two years required. Two years (four semesters) in addi— tion to those required in “A—E” above. How many graduating students have actually completed this course series is, in some ways, a good indicator of how well the high schools in the state are preparing students for college in 2000 Student Assessment and Student Achievement in the California Public School System the state’s university system. It’s also a marker for students’ plans for college. Over the past ten years, the rate at which graduating seniors have met these course requirements has been consistently climbing. As shown in Figure 27, whereas fewer than 30 percent of graduates met the requirement in 1988, more than 38 percent did so in 1997. Interpreting these changes depends upon how serious course titles match with actual course content. There is considerable evidence that actual topics covered and difficulty of con- tent may vary in different courses with the same name. So at the least, increased college preparatory course—taking reflects better moti— vation if not always an increase in student per— formance. Advanced Placement Examinationf“ Another secondary school measure of interest is the availability of and participation in advanced placement courses and examinations. Advanced Placement courses reflect college—level course work, and students passing advanced placement exams receive college credit. Thus, the percent— age of students taking AP courses and passing the exams is an indicator of the extent to which students are being prepared for, pursuing, and are being successful in rigorous academic coursework. Because of the rigor of the cours- es, students receive extra points for their grades in these courses (5 for an A, 4 for a B), which in turn advantages their grade-point averages for college admissions. Recently civil litigation was brought against at least one California school district for allegedly providing disproportionate AP opportunities to students of varying ethnic backgrounds.” Though we do not present data on the availal'nlity ofAP courses here, we do have some data on the frequency with which various ethnic groups take AP examinations, and contrast those numbers with the percent- Percent 100 90 80 70 6O 5O 40 34.32 34.12 33.11 34.97 35.78 37.48 29.93 30 , 3029 32.97 38.52 2O 1O 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 Figure 27. High School Graduates Meeting UC/CSU Course Requirements 1998-1997 ages of each group in the student population. For instance, African—American students com- prise 8.8 percent of the public school popula- tion, 3.5 percent of students taking the English Advanced Placement examination, and 2.5 per— cent of students taking the calculus test. In con— trast, students of Asian ethnicity comprise 11.2 percent of the student population, but account for 28.1 percent of the English AP test takers and a whopping 42.8 percent of those sitting for the AP calculus examinationxxx Clearly, the ethnic makeup of students taking Advanced Placement examinations is not representative of the California student population as a whole. One positive finding regarding the advanced placement data is the increased fre— quency with which California students are meeting the AP qualification standards. Since the 1991—")2 academic year, this rate has steadily improved every year, going from 9.2 percent at the beginning of the decade to 14.8 percent last year (see Figure 28). College Entrance Exar'r/zimltio11s‘I College entrance examination scores are anoth- er measure of who California high schools are preparing for college. As the figures below indicate, the performance of California’s col— lege-bound student population on the Scholastic Achievement Test (SAT) has been fairly stable over the last ten years. For both the math and verbal components of the test, statewide average scores dipped in the early part of the decade, but have steadily climbed back near the levels attained at the end of the last decade. For math, the achievement levels of the late 1980s have actually been surpassed in the last two years. 9 16 '9 14.8 $14 13.4 /. 12.6 E 12.1 <0 12 g 10.8 11-1 5:310 8 7 9.2 9.9 91 91 9.4 O 71 8.4 ' 81 ,1hwr/' $3 8 - 73 7.4 ell/’// g 0/ 6 5 - ”l" '1 G.) 6 52 5.8 f 4H , 13' 4.9 _., J” II 41 fl/Ir ''''''''' J" g) 4 a ’ E‘ g 2 0 3r 0 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 - California I Nation Figure 28. Advanced Placement Qualifying Rate 1989-1999 .8 CRL'ch, lsstiis 1.\' CALIFORNIA liot‘cx 1‘10\' ZOOO Student Assessment and Student Achievement in the California Public School System 100 90 80 70 60 g 50 8 g 40 30 26.98 27.16 27.7 2999 29.75 29.671 k r 0/' ’ ’ 9 20 16.41 17.06 15.84 17.35 17.41 29.67 F i‘ 4 a k a 10 13.79 14.08 13.77 14.17 14 14.16 0 1993 1994 1995 1996 1997 1998 I Non-Minority I Mixed Minority Figure 29. Students Meeting SAT Criteria in California 1993-1998 100 90 80 70 60 40E; 50 44.27 44.95 44.84 46.55 46.11 46.03 2 M C . g: 40 35.84 38.2 38.05 38.02 37.49 37.78 Mfr—MW“ m» a 4 4| 30 34.55 36.47 35.97 35.95 35.57 35.55 20 1O 0 1993 1994 1995 1996 1997 1998 I Non-Minority I Mixed Minority Figure 30.SAT Test Takers in California 1993-1998 P.\(ZI{ I 13‘) 140 As in the average scores for the math and verbal component of the SAT, there appears to be a rebounding trend in the percentage of test takers meeting or exceeding the combined IOOO—point threshold. This measure, too, expe— rienced a slight dip in the early 1990s but has inched up to remain between 18 and 19 percent over the last three years, levels comparable to the latter part of the 1980s. Of course, these rates vary by high school, with schools com- prised of high minority populations achieving rates at roughly half those of low minority schools (see Figure 29). And as Figure 29 sug- gests, the differences in these rates do not appear to be decreasing over time. Similarly, the rate at which high school seniors are taking the SAT has changed little over the last six years. As shown in Figure 30, the percentage of twelfth—graders taking the SAT in low minority schools has remained sta— ble at around 46 percent, while that rate has hovered around 36 percent for high minority schools over the same time period. College Attendance Data from Education Whit/.7 1998 indicate that 66.4 percent of high school graduates in 1996 went on to enroll in college (full or part—time) by the time they were 19 years old. This rate ranked California fifth out of 50 states in pro— viding students access and opportunity for col— lege. However, college completion rates for minority students entering as freshmen— decmed the equity rate—is not so rosy. The equity rate for California is 58.4 percent, which is below the national average of 65 percent. California’s four—year graduation rate is 41 per— cent, meaning that less that half of entering freshmen graduate within a four—year period. CRL'CIAI. lSSL'ES IN CALIroRNIA EDUCATION College Remediation Rates-‘3 Part of the reason for the lower completion rates may be the fact that California has a high number of part-time community college stu— dents. Another reason may be the fact that many students enroll in college with severe limitations in their basic reading and mathe- matics skills. In the California State University system, more than 54 percent of incoming first- year students are required to take remedial math and more than 47 percent need remedial reading classes. In the state’s elite University of California system, more than a third of the stu- dents fail to meet the minimal standards of writing proficiencyxxxiii This number has improved in the last year, from 38.9 percent in 1997 down to 33.3 percent in 1998. However, this indicator still suggests that although more high school graduates are completing the required sequence of high school courses, a great many are not at the basic levels of reading and mathematics ability that successful transi- tion into a university education requires. Snmmmy of Achievement California student achievement is low compared to the rest of the nation. This is true based not only on SAT—9 scores but also on the NAEP. Average student performance in some schools is better than in others, and it is fairly easy to identify which schools these are by who is going to them. Although students now take an addi— tional test designed to address their mastery of state—determined subject—matter standards, it is not ready for widespread implementation. Essentially across—the—board minimal gains on observed scores from the SAT—9 in 1999 corn— pared with 1998 probably signify familiarity with the process more than “real” improvement. IOOO Student Assessment and Student Achievement in the California Public School System Moreover, different measures of improvement greatly disagree with one another. At the secondary level, we have seen improvements in reducing drop—out rates and maintaining graduation rates. Graduating stu— dents are taking more nominally challenging course loads, and greater numbers of them are meeting advanced placement requirements, although the rates of advanced placement test— taking vary markedly by ethnicity. Students are scoring higher on college entrance examina— tions, but the percentage of seniors taking the examinations is holding steady. ()n the positive side, California does a good job of providing college opportunities to high school graduates. Unfortunately, these students often are not pre— pared for the fundamental academic require— ments for success in higher education. On a more troubling note, the relationship between the socio—demographic complex of poverty, language skills and ethnicity and stan— dardized student achievement measures is immense and getting stronger.” Over the past six years, this relationship has strengthened, not diminished (see Figure 31). These background measures relate to average school performance on the SAT at an extremely high level, account- ing for greater than two-thirds of the variation in scores among schools. Similar evidence is found for the SAT-9 test, particularly at the lower grades, where background measures account for 60 to 80 percent of the variance in average school scores in reading, spelling, and language arts (see Figure 32).“ A somewhat weaker relationship is found between back- ground measures and mathematics, although a majority of the variance is still accounted for at each grade level, from a low of 56 percent in grade 2 to a high of67 percent in grade Jr. The relationship is clear. Alore poverty relates to lower average scores. More limited SAT Combined R2 1.0 0.9 0.8 ‘ 0.7 0.67 0.675 0 583 0.602 0643 0.617 0.6 - 0.5 0.4 0.3 0.2 0.1 0.0 1992 1993 1994 1995 1996 1997 Figure 31. Relationship Between Socio-Economic Measures and SAT Scores 1992-1998 PACE I 141 142 SAT 9 Reading R2 1.0 0.9 0.8 0.788 0-802 0.788 0.711 0.7 0.739 0.698 0.613 0.6 0.5 0.4 0.3 \O.367 N286 0.248 K. 0.2 0.1 0.0 Grade 2 Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 9 Grade 10 Grade 11 Figure 32. Relationship Between Socio-Economic Measures and SAT-9 Reading Scores English skills relate to lower average scores. Greater minority representation in the student population, sadly, also relates to lower average scores. Poverty is increasing as the percentage of students qualifying for free or reduced—price lunch has risen from 32.19 percent in 1989 to 47.61 percent in 1999. Similarly, the percentage of California students with limited English pro— ficiency has jumped from 16.29 percent in 1989 to 24.89 percent in 1999. Both of these increas— es represent about a 50 percent jump over the past decade (see Figures 33—34). Over the same period, the minority population has risen only slightly, but consistently (see Figure 35). And poor performance doesn’t just affect those stu— dents who lack language skills or sufficient monetary resources. Students fully proficient in English and those not eligible for free or reduced—price lunch in schools with high con— CRUCIAL Issues IN CALIFORNIA EDUCATION centrations of LEP and economically disadvan— taged students perform more poorly than their counterparts in schools with lower numbers of these disadvantaged students. The goal of California schools is to prepare all students to reach high academic standards. To do so, the educational system should seek to reduce the impact socio—demographic measures have on student achievement. Student achieve— ment should relate more to what students learn in the classroom than to their background. Unfortunately, we currently are not seeing the desired effect. Why? VVe have to consider the sensitivity of our measures to instructional change, to the capacity of the schools and school districts, to the motivation of students and parents, and to the period of time (fewer than two years) that California standards have been in place. 2000 Student Assessment and Student Achievement in the California Public School System 100 90 8O 70 60 50 Percent 40 30 24.39 24.8 24.72 24.89 16 29 18 32 2014 21.34 22.35 23.28 23.84 10 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 Figure 33. Limited English Proficient Students in California 1989-1999 100 90 80 7O 6O 50 44.33 46.37 47.07 47.43 47.61 40] 42.83 Percent 40 33.51 35.32 38.16 32.19 30 20 1O 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 Figure 34. Students Receiving Free or Reduced Lunch in California 1989-99 PACE I H?» 144 Percent 100 90 80 70 60 50 40 52.3 53.1 53.82 51.36 43.77 45.21 46.53 47.55 48.45 49-5 50-42 30 20 1O 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 Figure 35. Minority Students in California 1988-1999 The Future: Assessment and Accountability With the adoption of the Academic Performance Index (API) in November 1999, California has moved into a new level of educa- tional accountability. It has adopted a general plan to use assessment and other key school data, e.g., student absences and graduation rates, as part of a system to hold schools accountable. The plan is supposed to support ytmzd/zrds-based refirm. Over a six—month period, a committee of school policy—makers, academic experts, and practitioners met and prepared the requirements of the API. The details are avail— able on the Department of Education Web page (www.cde.ca.gov) and will eventually cover how growth targets are set (based on the distri— bution of performance of students at the school), how comparisons are made, the expec— tations for identifiable subgroups, and sanctions and rewards. What is of most relevance here is the degree to which the API relies on assess— ments, and related to that, the degree to which the assessments represent and propel progress on the state’s standards for student perfor— mance. The original plan for the API involved phasing in various assessments as they became available to bring the assessment into closer alignment with the standards. However, for the 1999—2000 year, only performance on the SAT— 9 component of STAR enters into the account— ability index. Prior to adopting the API details, the California State Board of Education adopted a framework that enunciated principles to guide the use of the accountability system. The crite— ria comprising this framework are reproduced in Figure 36 below. The relevance of these principles to con— cerns we have raised earlier about assessment and criteria for quality assessment systems is CRUCIAL ISSUES IN CALIFORNIA EDUCATION 2000 Student Assessment and Student Achievement in the California Public School System Academic Performance Index Framework 0 The API Must Be Technically Sound . The API Must Emphasize Student Performance, Not Educational Processes 0 The API Must Strive to the Greatest Extent to Measure Content, Skills, and Competencies that Can Be Taught and Learned in School and that Reflect the State Standards (our emphasis) o The API Must Allow for Fair Comparisons . The API Should Include as Many Students as Possible In Each School and District . The API Must Measure School Performance and Growth as Accurately as Possible 0 The API Should Strive in the Long—Term to Measure Growth Based on Student—Level Longitudinal Data 0 The API Should Be Flexible and Its Component Indicators Should Be Stable - The API Should be Understandable, Particularly to Educators and Parents 0 The API Is Part of an Overall Accountability System That Must Include Comprehensive Information Which Incorporates Contextual and Background Indicators Beyond Those Required by Law 0 The API Should Minimize Burden 0 The API Should Support Local Accountability Systems 0 The API Must Conform to the Requirements and Intent of the Public Schools Accountability Act of 1999 as Well as Related Legislation Figure 36: Academic Performance Index Framework36 (Adopted by the California State Department of Education at their July, 1999 meeting) Relevant Standards for California from the Standards for Educational and Psychological Testing 0 State Purpose(s) and Minimize Negative Consequences of the test 0 Give Evidence of Technical Quality of the test for Each Purpose 0 Document Relationship to Content Standards 0 High Stakes (Promotion) Requires Match Between Instruction and Test Content 0 Give Evidence of Suitability of Test for Program and for Test Population 0 When Use of a Test or System lmplies a Specific Outcome, Provide Basis and Evidence for Expectation 0 Minimize Possible Misinterpretation of Data with Appropriate Context 0 No Student Decision Should Be Made on the Basis of One Test 0 Test Preparation Should Not Adversely Impact Validity of Results 0 Reports Should Include Classification Error and Error in Measurement of Change 0 Public Interpretation Should Be Handled by Trained Personnel Figure 37: Relevant Standards for California from Standards for Educational and Psychological Testing (Adopted by the California State Department of Education at their July, 1999 meeting) PA (I E I HS 146 clear. In addition, the evolution of assessment for accountability in California calls for careful analysis. In general, California is starting with a measure—the SAT—Q—that has only limited relationship to the state’s standards. While there are plans to add more elements down the line, the current accountability provisions may work to encourage a near exclusive focus on the SAT—9, since it was the first and most salient measure in use. In adhering to the principles articulated by the state board, which in a preamble explicitly commit to continued studies of the validity of the state’s assessment system, it may be relevant to reference yet another set of guidelines for the design and use of assessments. From the recently published Standards for Educational and Psychological Testing", the following para— phrased standards are applicable to California planning and evaluation. As California moves forward with its assess- ment and accountability system, it will be important that it do so in line with its own principles and those of the testing profession. Conclusions Starting with the available data, the story about California is mixed. \Vhen examining the over— all performance on the SAT—9, we find that the state average, over all grades and all subject matters, is below the national average. However, when we account for the state policy requiring that all students who have been in school for one year take the test—whatever their English proficiency—we find California students positioned around the national aver— age. In fact, given the difference in the compo— CRL'CIAL ISSL'ES IN CALIFORNIA IZDL‘CA’I‘IOX sition of the tested population and the norming groups, this result is somewhat better than we might expect. However, when we move to standards-based measures, of which NAEP is a general example, California performance looks poor indeed. California especially falters when one addresses the performance of children in poverty. Also, it is important to recall that on the NAEP, only students who can comprehend the examination are tested. What will be important to watch in the future is whether California students, like those in many other states, at the outset have lower performance on new standards-based tests. We would expect lower performance if the tests are measuring and students are in fact attempting to meet more challenging goals. We would also expect to see test performance to rise over time as instruction becomes more relevant to the standards the assessments are measuring. California has a number of important tasks to consider. We believe that there is direct action that can be taken to support the best possible development of the assessment system, of the accountability structure it supports, and of California education. First and foremost, it is desirable to focus on the appropriateness and validity of the assessments planned to be in the system, as they are under development. In sim— ple terms, any test is usually not exchangeable for any other. For example, as we have seen, the SAT—9 is a general achievement test and not fully aligned with the state’s content and per— formance standards. It cannot simply be exchanged for a rigorous standards—based assessment system. Similarly, a high school graduation test presumably must make distinc— tions between those who are qualified and those who are not qualified, relative to explicit ZOOO Student Assessment and Student Achievement in the California Public School System standards, for a high school diploma, implying assessment items primarily focused on making that distinction. A college admission test, on the other hand, must make distinctions at a higher ability range, thus implying a different item focus and test—taker differentiation. A sin— gle test of limited duration probably cannot well serve both these purposes. This is not to say, however, that these various measures themselves should not be consistent with the state’s standards, albeit representing levels of performance and sophistication. Nor is it the case that a single test strategy for decision making is a good one or that all students neces— sarily should have to pass the same test. For example, a number of people have advocated using course—based exams for California’s High School Exit Exam, as a direct way to align cur— riculum and testing and better assure that stu- dents have the opportunity to learn what is expected. Two examples of such course—based exams already exist — the Advanced Placement Exams, which were discussed earlier, and the Golden State Exams, a series of state—developed, academically rigorous, voluntary exams which are linked to specific high school courses. Both of these assessments probably represent a higher level of proficiency than can be expected from all high school students in the short run, but one might imagine a system where passing one or the other of these tests would count for the HSEE requirement, while still requiring stu— dents who were enrolled in other course to take the actual HSEE. Assessments can be designed to serve various policy purposes, but there are times, such as we are seeing in other states, where policy impera— tives have swamped technical capacity to deliv— er the assessments. Time frames have been 1nsufficient to assure a quality assessment or to prepare the educational system and its students for a new set of expectations. The result is usu— ally some form of retrenchment. In California, we would hope to avoid this cycle. Recommendations These recommendations will be brief and illus- trative rather than exhaustive. Validity studies examining the extent to which California’s assessment system is achieving intended purposes (school accountability, instructional improvement, consequences) must be undertaken immedi— ately. These studies must address the impact of the assessment on various subgroups of students and schools. Evidence that the assessments detect instruc— tional effects is needed. Efforts should be made to describe which standards are not measured by statewide pro— grams (and are, therefore, appropriate for local scrutiny). Studies of side effects are needed, for exam— ple, to determine whether the developed form of accountability supports or interferes with the recruitment and retention of high— quality teachers for all children. Careful decisions need to be made about weighting of new measures as they become available for inclusion on the API. .Vlodeling studies of potential volatile effects on APT status by school and group will be required. Detailed studies of the relationship among all measures, those used for school report cards and the API, should be conducted to determine whether and how various out— PACE I 147 1—18 comes operate at cross purposes to one another. ' Smarter studies of alignment are necessary, including alignment of planned and enacted curriculum, resources, and preparation of teachers. 0 Studies of the accuracy of the test are need- ed. In addition, strategies to help parents, the community, and the teaching force to understand the meaning of assessment—and what it does not mean—are essential. Finally, well—designed assessments may tell us where we are and may communicate where we want to be. As we hope we have made clear, California’s assessment and accountability sys— tem will need to continue to evolve to more fully achieve these goals and to support a stan— dards-based system. We can all agree that the current status of student performance in California is insufficient, and that California schools need to improve. The real question is not where we are, but where we need to be and how we will get there. We should he look— ing for assessment results to show progress toward excellence—toward truly rigorous stan- dards for student accomplishment—as well as progress toward equity. That is, we need to both raise our expectations for what children should know and be able to do, and assure that as we move forward, we do not continue to leave some students—indeed a growing pro— portion—behind. We need to move all chil- dren ahead and reduce the gap between our least and most economically advantaged stu- dents. We need to find better ways to assure that poor students and students who start school without full English proficiency have effective opportunities to learn and are given what they need to make steady progress. Certainly dramatic changes will not come overnight. Improvement will not come easily or quickly if we keep to high standards. It will take more than accountability and clear communica— tion of expectations to change practice at a sig- nificant, meaningful level. It will take impor- tant and coordinated changes in capacity; in teacher quality; in curriculum, instruction, and assessment; in parent and community involve— ment; and in district and local capacity to sup— port change—to name just a few, as the other chapters in this volume make clear. It also will require that we align and focus educational resources, policies and practices at the state, district, and local levels to assure all students achieve and learn what they need to be success— ful citizens of the future. We look to California’s assessment system to be able to provide sound guideposts on how we are doing. Notes 1. National Commission on Excellence in Education. A Nation at risk: The imperative for educational reform. A report to the nation and the Secretary of Education. (“ashingtom DC: US. Government Printing Office, 1983). 2. See for example: Corbett, II. D., & \Vilson, B. L. Testing, reform, and rebellion. (Norwood, N]: Ablex Publishing, 1991); Dorr—Bremme, D. “7., & Herman, J. L. Assessing student achievement: A profile of classroom prac— tices (Los Angeles: UCLA Center for the Study of Evaluation, 1986); Kellaghan, T., & .\Iadaus, G. F. “National testing: Lessons for America from Europe,” Educational Leadership, 493(1991): 87—93; Koretz, D., Stecher, 13., Klein, S., McCaffrey, D., 8: Deibert, E. “Can portfolios assess student performance and influence instruction?” in The 1991-93 Vermont experience, RAND (Santa Monica, CA: RAND, 1993; reprint from CSE Technical Report 371, Los Angeles, University of California, Center for Research on Evaluation, Standards, and Student Testing, December.); Koretz, D. AL, Barron, S. 1., Mitchell, K. _l., & Stecher, B. M. Perceived effects of the Kentucky Instructional Results Information CRUCIAL ISSUES IN CALIFORNIA EDUCATION 2000 Student Assessment and Student Achievement in the California Public School System System (KIRIS). MR—.79Z.PCT/FE (Santa Monica: RAND, 1996); Koretz, D. M., Mitchell, K. J. Barron S. I. 8: Keith S. Fmal report: Perceivedetfects of the Allaryland School Performance Assessment Program (CSE Tech. Report 409). ’ (Los Angeles: UCLA National Center for Research on Evaluation, Standards and Student Testing, 1996): McDonnell L. NL, & Chmsser, C Testing and teaching: Local implementation of new state assessments (CSE Tech. Rep. No. 4-13). (Los Angeles: University of California, National Center for Research on Evaluation, Standards and Student Testing 1997)- Smith, M. L. Reforming schools by reforming assessment: Consequences of the Arizona Student Assessment Prograiif (CSE Technical Report). (Los Angeles: UCLA Center for Research on Evaluation, Standards, and Student Testing (CRESST), 1996); Stecher, B. M., Barron, S., Kaganoff, T., 8: Goodwin, ]. The effects of stai‘idards-based assessment on classroom practices: Results of the 1996—97 RAND survey of Kentucky teachers of mathematics and writing (CSE Tech. Rep. No. 482). (Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing, 1998). 3. While most of these standardized tests were of the norm—referenced, multiple variety, some districts chose criteri— on-referenced tests that included some performance-oriented items. Selected tests had to meet technical quality criteria that were established by the state. ' 4. See NCEST report, 1992 5. IASA, 199-1 6. See for example: Corbett, H. D., & \Vilson, B. L. Testing, reform, and rebellion. (Norwood, NI: Ablex Publishing, 1991); Dorr—Bremme, I). “I, & Herman, J. L. Assessing student achievement: A profile oficlassroom prac— tices (CSE Monograph Series in Evaluation, No. 11). (Los Angeles: UCLA Center for the Study of Evaluation, 1986); Kellaghan, T., 8: Madaus, G. F. “National testing: Lessons for America from Europe," Educational Leadership, 49.3(1991): 87—93. Shepard, L. “ill national tests improve student learning? (CSE Technical Report 3-13). (Los Angeles: University of California, Center for Research on Evaluation, Standards, and Student Testing, 1991). I 7. All students are required to take the test unless specifically exempted by an Individual Education Plan (IEP) or a written parent request. 8. Limited English Proficient (LEP) is the term used by California in its directives and reports. Many practitioners and researchers prefer the term English Language Learners (ELL) because of its accuracy and is more commonly found in the recent literature. 9. Abedi, ]., Lord, C. & IIofstetter, C. “Impact of Selected Background \r'ariables on Students’ NAEP Math Performance.” CSE Technical Report # 478, University of California. (Los Angeles: National Center for Research on Evaluation, Standards, and Student 'I‘esting, 1998). 10. The LEP designation applies to the full continuum of students from virtually no English proficiency to almost fully proficient. As students progress to the latter end of this continuum, scores from English language tests become more meaningful, though the point at which such meaning occurs is currently under investigation. 11. In addition to statewide measures, many districts have curriculum sensitive district assessments which are used to evaluate student achievement. 13. The Advanced Placement Program is conducted by the College Board in a total of 33 possible subjects. 13. Some believe that this drop may be an artifact of the norm group at this level, rather than representing an actual decrement in performance. Technical data that would more definitely determine the cause has not been available. 14. Measurement contrasts “observed” scores with “true” scores. “Observed” scores are the scores which students attain (and which are “observed”) when students take a given test. Their true score is the score they would attain if the test were a perfect measure of their capability. \Ye use “observed” scores to estimate what students “true” performance capability. 15. “How Accurate are the STAR National Percentile Rank Scores for Individual Students? — An Interpretive Guide” by David Rogosa is available to download at the CRESST “'ebsite, w\vw.cse.ucla.edu. PACE I 1-19 150 16. See, for example, Shepard, L. A. “Inflated Test Score Gains: Is the Problem Old Norms or Teaching the Test?” Educational Measurement: Issues and Practice, 9 (1990): 5-22. 17. For reasons of space, onlv 3rd grade reading results are presented here. For analyses of other subjects, please see full technical report available through CRESST. 18. See CDE Stanford 9 Augmentation information at www.cde.ca.gov 19. The SAT—9 mathematics augmentation was controversial at grades 8—1 1. Only students taking particular courses were required to take the test — e.g., Students enrolled in Algebra at 8th grade, and critics raised serious questions about the technical and content appropriateness ofthe items. Because the test is still under development, we are not reporting results for these grades. 20. Hearing bv the Joint Senate and Assembly Education Committee, November 1999. 21. Observed relationships were similar across grades and subject areas. Only 3rd grade results are displayed here. See full technical report for other subjects and grade levels. Herman, J., Brown, R, and Baker, E. “Student Assessment and Student Achievement in the California Public School System.” CSE Technical Report (forthcoming). (Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST)). 22. Observed relationships were similar across grades and subject areas. Only 3rd grade results are displayed here. See full technical report for other subjects and grade levels. Herman , J., Brown, R, and Baker, E “Student Assessment and Smdent Achievement in the California Public School System.” CSE Technical Report (forthcoming). (Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST)). 23. United States Department of Education, 1998. 24. Education Watch 1998: The Education Trust State and National Data Book, Volume 2 (Washington, DC: Education Trust, 1998). 25. Barton, PE. and Coley, RJ. Growth In School: Achievement Gains from the Fourth to the Eighth Grade. (Princeton, NJ: Policy Information Center, ETS, 1998). 26. Data from California DOE Website; 1988—89 — 1998—99 state summary numbers; One year dropout rate is calcu— lated by the sum of the number of dropouts from grades 9—12, divided by the enrollment in grades 9—12 and un—graded secondary. 27. Data from California DOE Website; 1988—89—1997-98 state summary numbers. 28. Data from California DOE Website www.cde.ca.gov; 1991—92—1998—99 state summary numbers for public schools; rate ()f passing exams per 100 juniors and seniors in public high schools. See also, The advanced placement pro— gram: California’s 1997—98 experience. (Sacramento, CA: California State University Institute for Education Reform, 1999). 29. David Hoff, “Inglewood ACLU Lawsuit,” Education Week, 4 August 1999, Volume 2, 13. 30. Education Watch 1998: The Education Trust State and National Data Book, Volume 2. (Washington, DC, Education Trust, 1998). 31. Data from California DOE Website www.cde.ca.gov; 1988—89—1997—98 state summary numbers for public schools; SAT verbal, SAT math, percent meeting SAT criterion (>=10()() on Verbal and Math sections), and percent of 12th graders taking the SAT. The percent ofminority (American Indian + Black + Filipino + Hispanic + Pacific Islander/total enrollment) was calculated —and schools were designated as minority (>3()%) or non—minority (<3000). The percent meeting the SAT criteria (>=1()00) and percent taking the SAT are presented. 32. Data from University of California for 1997 and 1998. Summary scores were created as the percent of students not meeting the requirements after taking the Subject A English examination. CRUciAL ISSUES IN CALIFoRNIA EDUCATION 2000 Student Assessment and Student Achievement in the California Public School System 33. The UC system does not have a consistent measure of mathematics preparation. Each campus uses its svstem for assessment] placement. 34. Regressmn analysis on SAT Combined scores (1993-97). The amount of explained variance from regressing aver— age test scores onto the school measures of percent of students receiving free lunch and percent Limited English Proficient 18 plotted. 35. Results were similar across subject areas. For reasons of space, we only present figures for reading here. See full CRESST technical report for other subject areas. Herman, _l., Brown, R, and'Baker, E. “Student Assessment and Student Achievement in the California Public School System.” CSE Technical Report (forthcoming). (Los Angeles: Universitv of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST)). ' 36. Adopted by the California State Department of Education at theirjulv 1999 meeting. 37. See Joint Committee on Testing (AERA, APA, NCME), Standards for Educational and Psl'chological Testing. (\Wshington, DC, 1999). PACE I 151 Chapter 8 Connecting California’s K-12 and Higher Education Systems: Challenges and Opportunities Andrea Venezia Stanford University In recent years, policymakers have begun to realize that California’s K—12 and higher edu— cation systems can no longer be approached as separate entities. Historically, there has been a disconnect because of funding strategies, mission and stmcture. The K—12 and higher education systems have been two separate spheres moving in different directions with no mechanisms in place to bring them together. This disconnect has led to all kinds of confusion and disjunctures between such policies as undergraduate admis~ sions requirements, high school graduation exams, and university placement tests. High school level exams such as the aug— mented STAR and the Golden State Exams‘ ask students to know and be able to demonstrate skills in areas that are different from those test— ed by college entrance and placement exams such as the SAT—I, SAT—II, and the California State University’s mathematics and English placement exams. VVhile there is a lot of over— lap between some of the assessments and stan— dards, there has not been enough purposeful alignment. In 1995, the California Education Round Table” brought some of these issues to the fore by proposing that the state undertake several collaborative initiatives between the K— 12 and higher education sectors. These includ— ed: 1) developing more agreement around the standards necessary for high school graduation, 2) clarifying the expected competencies for uni— versity admission, and 3) assessing student progress more uniformly to determine if stan- dards have been met.1 While much progress has been made within education sectors, more work between sectors must be done to tie K—12 and higher education together. We propose that policymakers look toward developing deeper connections between the K— 12 and higher education systems to create more coherence. This is particularly important in an environment in which over 70 percent of the state’s graduating high school students will attend some kind of postsecondary education institution, and in a diverse state, with an ever— gr wing population, that is grappling with issues pertaining to equitable access to postsec— ondaiy education. (liven the high skills nature of the labor market, and growing economic dis— paritics in the state, it is imperative that the two education levels work together to increase student opportunities to enter and succeed in higher education. In this chapter, we assert that one way to strengthen the opportunity struc— ture for all students is to develop greater align- ment between the systems, and thereby 'JJ PACE I 15 decrease the fragmentation and confusion sur— rounding high school curriculum requirements and undergraduate admission and placement policies. In the following chapter, we begin by pre— senting recent changes that have occurred in K—12 education instructional policy. We then look at the current policy landscape in California higher education. Those two sec— tions provide the context for the closing section in which we suggest ways in which policymak- ers can work to bring coherence to the entire K—16 system by connecting such policies as content standards, admission standards, and placement exams. \Nhile there are logical and historical reasons why the two systems are so separate—K-IZ is a mass education system for minors, while higher education is a more elite and often more specialized form of education for adults — we propose that strengthening the alignment between the two systems would be beneficial for all stakeholders. Both K—12 and higher education stakeholders would benefit from having a more academically prepared stu— dent body. Also, students, parents, and K—IZ educators would receive clearer signals regard— ing college preparation and expectations, while higher education stakeholders could use infor— mation from K—IZ assessments as one factor in admissions decisions. Changes and Challenges in California’s K-1 2 Instructional Policy Over the last twenty—five years, the state’s K-13 education system has evolved from one largely controlled by local school districts to one that CRUCIAL IsscEs IN CALIFORNIA EDL'CA’I‘lON is more heavily controlled by centralized state decision-making. This transition occurred in a somewhat desultory manner, as the state devel- oped no master plan, or road map, to guide policy toward a systemic end.3 Both school gov- ernance and education policy remain fragment- ed and confused. As a consequence, the Senate Office of Research wrote that, “California’s schools operate in an environment of signifi- cant instability that impacts their ability to plan and perform in a manner that maximizes stu- dent achievement. At the core of this instability is a convoluted, multi-layered system of gover- nance in which roles and scopes of authority are not clearly defined.” One district superin- tendent, undoubtedly voicing the concerns of others, stated that California has a K-IZ educa- tion system with no conceptual framework.4 In recent years, California has been working to reverse this situation, taking tentative but significant steps to putting a framework in place. The centerpiece of this campaign is the state’s effort to develop an accountability sys— tem based on rigorous academic standards aligned to curriculum frameworks and assess— ments. The hope is that this accountability sys- tem will result in higher expectations and greater accountability for students and schools alike which will lead to improved student achievement. In such an environment, assess— ments will have higher stakes than ever before. Students could face not graduating from high school if they fail a proposed high school exam— ination. Individual schools and their staffs may receive rewards for success and harsh penalties for failure. Progress regarding the development and alignment of components within the account— ability system has been made on several fronts. ZOOO Connecting California’s K—12 and Higher Education Systems Academic content standards were developed (however, performance standards that will be used to interpret student success toward meet— ing the content standards are still under discus- sion), followed by the development of the cur— ricular frameworks and the augmentation of the Stanford 9 assessment used in the state’s Standardized Testing and Reporting (STAR) assessment program. The Stanford 9, a basic skills test, has been augmented with rigorous testing items reflective of California’s standards. However, the state board of education and poli- cymakers are aware of the need to further align the assessment program with the standards. In any case, implementing the standards will not be an easy matter. To take just one example, the standards have been layered on top of an exist— ing curriculum, and categorical programs are often tied to the curriculum. Thus, the curricu— lum and the funding stream often have little to no connection with today’s standards. Consequently, there are two different incentive systems — one based on categoricals and one based on standards; these systems are often in conflict. As this chapter highlights, many reforms and policies have been layered on top of each other without a rational plan. Curricular Standm-d-Setting The California Education Round Table helped to lay the foundation for the development of the current standards with its 1995 report enti— tled, “Collaborative Initiatives to Improve Student Learning and Academic Performance, Kindergarten Through College.” Its first rec— ommendation was that the state needed to agree on standards for high school graduation and clarify expected competencies for universi— ty admission.‘ The standards commission authorized by Assembly Bill 265 developed content standards for all core curriculum areas in all grade levels. While the standards are not mandatory, the hope is that all districts will be held accountable for preparing their students for the state assessment; the results reflect stu— dents’ achievement toward the standards. The state board has approved content standards in English, math, science, and social studies. These standards outline what students need to know and be able to do to be considered profi— cient in each subject area. The state has yet to adopt performance standards stating what level of achievement is expected of students for each of the content standards." Curriculum frameworks have been approved by the state board of education in English and mathematics, and are being developed in other subjects. The augmented STAR is aligned with the content standards, and current plans are for STAR to include Stanford 9 and the augmenta— tion. The state board of education plans for the performance standards (showing whether stu— dents meet the content standards) to be aligned with the content standards (showing what stu— dents know and can do) and assessments.— Currently, the state’s curriculum frameworks are being updated to be aligned with the state’s content standards. There are not state adopted textbooks for high schools.“ Unless the curriculum, standards, and assessments are carefully aligned, there will be serious ramifications for the effectiveness of the proposed accountability system." For example, if funding is tied to high academic perfor— mance, or if local educators face firing or school reconstitution if students perform poor- ly on the STAR, and if the standards, curricu— lum, and assessments are not aligned, the sys— JI JI PACE I 1., 156 tem will unjustly penalize the schools. Although it is impossible to align every aspect of a state’s K—12 system at one time, if students are being assessed on and held accountable for knowledge and skills that are not taught in the classroom. The Need to Align Assessments with the Standards As the assessment chapter describes, California’s K-12 assessment environment has been tumultuous during the past decade. From 1972 until 1990, California’s only testing pro- gram was the California Assessment Program (CAP), which focused on school site scores. Former Governor Deukmejian canceled this low stakes testing program in 1990. In 1992, the State Department of Education began work on California’s first performance assessment, the California Learning Assessment System, or CLAS. CLAS was supposed to provide school scores initially and would eventually provide individual scores. But then Governor Wilson vetoed CLAS funding due to concerns about the test design, in addition to the lack of indi— vidual pupil data. The legislature passed legisla— tion in 1997 authorizing a new testing system that would provide individual, school, and dis— trict assessment data in relation to statewide performance and academic standards. In response to the legislation, plans for the California Assessment of Academic Skills— another test that would provide only group scores—were announced. Again, Governor Wilson intervened and withheld fimding, ask- ing for a basic skills test that would provide individual scores. Thus, the Standardized Testing and Reporting Program (STAR) pro— gram was introduced through gubernatorial flat CRUCIAL ISSUES IN CALIFORNIA EDUCATION in 1997 and was authorized by Senate Bill 376 in October 1997. While the assessment system was originally designed to provide primarily diagnostic information, a new high stakes accountability focus has been introduced, man- dating that the test results determine whether students can graduate from high school or if school staff will be rewarded or sanctioned.” All students in grades 2—11 in California’s public schools are required to participate in the STAR program. The STAR program includes the Stanford 9, a nationally normed basic skills test; the Stanford 9 Augmentation, a set of test questions aligned to California’s new content standards; and SABE 2, a Spanish language test taken by fIrst year limited English speakers to assess content knowledge. The Department of Education hopes to develop a new test, the California Assessment of Applied Academic Skills (CAAAS), which would be similar to the Augmented Stanford 9 in its purpose, design, and type of scoring (criterion—referenced). Unlike the Augmented Stanford 9, in which all students respond to the same set of questions, students taking the CAAAS would respond to different sets of questions.H This is called a matrix test; it allows for more questions to be used. This creates a deeper assessment of gen— eral knowledge acquisition at the school level — not just at the individual student level. Yet another assessment, a statewide high school exit examination in Reading, Writing, and mathematics, is scheduled to be adopted by the State Board of Education in 2000 and implemented for students in senior high school starting in the 2003-2004 academic year. One rationale behind the legislation creating this test is that, since localities are not mandated to adopt the state’s content standards, they may be 2000 Connecting California’s K-12 and Higher Education Systems left with local standards that may not be high enough. A high stakes statewide graduation exam would, de facto, ensure that every school aligns some of its standards with the state stan- dards. Although it would seem logical to use the STAR test as a graduation exam, rather than to create a new test as the exit exam, the STAR is “a generic test of basic skills that does not necessarily have ‘curricular validity’ as yet...the basic test is still not assured to match the state standards, and therefore would not be ‘aligned’ to the standards based exit exam.”” The debate surrounding the exit exam has been a contentious political issue — especially around the stakes of the exam. The statute is clear, though. In 2004, students will not be able to graduate from high school unless they pass the exam. Once the test is given in 2004, there will be an evaluation to determine whether there should be an additional, alternative, assessment.” The results from the 1999 STAR assessment are discussed at length in the assessment chap— ter. A major issue for this chapter is that the current STAR test is only partially aligned with the state’s content standards or the current cur— riculum. Conseduently, the Stanford 9 is not testing the same knowledge and skills that stu— dents are being asked to learn in the content standards and statewide curriculum. A piece of the Star program, the augmented STAR, administered in 1999, is more aligned with California state standards.H The California Department of Education also administers the Golden State Exams (GSE) — the state’s end—of—course exams. The tests are offered in key subject areas in grades seven through eleven. The GSE program recognizes students for outstanding levels of achievement on each examination, culminating with the new Golden State Seal Merit Diploma established by AB 3488." The UC System’s BOARS (the Board of Admissions and Relations with Schools) committee members testified at a leg- islative hearing in Sacramento that it will be analyzing how well the GSE’s predict grades for students in the UC system. After complet- ing the research, the committee will consider whether it will use GSE scores to inform admissions decisions, as admissions criteria, or not at all. The GSEs, however, are not current— ly aligned with the content standard, the aug— mented STAR, or higher education admission and placement policies. It is legislated that the proposed high school exit examination will be aligned, as far as con— tent is concerned, with the state’s academic content standards, and work is being done to align the Golden State exams with these stan— dards.“ The result of all of this testing is that high school students spend many hours prepar— ing and taking K—lZ tests that are irrelevant for higher education admission, and higher educa— tion—related exams that are irrelevant for high school graduation. An eleventh grader takes six hours of standardized tests in writing, social studies, mathematics, and reading comprehen— sion. Those scores account for nothing when that student applies to a public institution of higher education in California.“ NIu/eing Students and Schools Accountable for Results There is great momentum behind the develop— ment of a statewide accountability system, which is intended to drive the improvement of California’s schools. Policymakers and the pub— lic have shown growing concern over much— PACE I 157 publicized problems such as low test scores, stu— dent apathy, and poor national rankings. The current accountability movement is concerned with outputs (i.e., performance on tests), not on inputs (i.e., student/teacher ratios, length of class periods) as was the case in the past.“ Senate Bill (SB) 1570 required the superin- tendent of schools to submit a plan for “posi- tive and negative incentives” for the state board of education’s approval by the last day of 1997.1U SB 1x and SB 2x address strict accountability measures, including the establishment of a four-part $192.3 million umbrella entitled the Public Performance Accountability Program. The two pieces of legislation include: ° An Academic Performance Index (API) that will be the basis for ranking all public schools’ performance; ° Intervention for underperforming schools, based on the API, that will include financial assistance and sanction, including the possi— ble reassignment of school personnel; 0 $125 million for schools that meet and/or exceed their performance targets; and ° the development and implementation of a state high school exit examination.” In addition, Assembly Bill 1626 requires every school district to approve a policy regarding the promotion and retention of students between grade levels and to ensure that students are identified who “should be retained or are at risk of being retained in their current grade level.”1 Senate Bill 1370 increased the funds available to districts for summer school instruc— tional programs, or remedial programs in order to deal with the probable influx of students needing remedial assistance."3 CRL'CIAL lSSL'ES IN CALIFORNIA l‘lDL'CATlON The California Public Schools Accountability Act (PSAA), part of Governor Davis’s school reform program, went into effect in the fall of 1999. It is designed to set targets for improving all schools and for forcing low-performing schools to shape up. The main measure of suc- cess or failure will be results from the STAR assessment. The first API results that were released used only STAR results, raising an out- cry from educators that one measure is not enough for an accountability program. In the future, rates of attendance, teacher absen- teeism, and graduation rates will also be used in the performance indices. Every school that falls below the median is eligible for a $50,000 state grant to hire a consultant in addition to funds to implement reforms. Schools that do not improve within three years will be subject to sanctions, such as the reassignment of teachers, the removal of principals, or take—over by the state. All of California’s schools must improve five percent per year. In addition, failing schools must identify the underlying causes of low student performance and figure out how to rectify them.“ California is behind most other states in developing and implementing an accountability system. At least 32 states and 3—1 large city school districts have accountability systems based, in part, on test scores. Alany educators are concerned that the system is moving too quickly given its fragmented nature — and that holding schools accountable for success using a measure that is not entirely aligned with the content standards is poor policy. At the end of this chapter, we explore issues related to the development of a K—16 accountability system. 2000 Connecting California’s K-12 and Higher Education Systems Such an accountability system would tie together data from both systems. working Toward a K—12 Master Plan Policymakers and legislators are increasingly concerned about the state’s lack of a compre— hensive, coherent vision for K—12 reform.” Responding to this concern, the Legislative Analyst’s Office has proposed that the state develop a Niaster Plan for K—12 education (see www.1a0.ca.gov/). The proposed plan would provide a conceptual framework for K—12 edu— cation, local control over the implementation of standards, and local fiscal control. The state would provide schools and the public with funding, flexibility, and information}5 To address the concerns about the lack of a coher— ent vision for K—12 education in the state, the Joint Committee to Develop a Master Plan for K—12 and Higher Education was been formet .3“ Absent a plan, however, many of the current reforms, standards, and assessments will be seen as ad hoc, with each major policymaker advocating the use of his or her favorites. It remains to be seen if all the pieces of the California education policy puzzle will be put together correctly, or if they will evolve into a misaligned assortment of policies. Currently, the layers of categorical programs, assessments, standards, curriculum frameworks, and accountability measures —some aligned and some not—create a confusing environment through which students must navigate in order to graduate from high school and attend insti— tutions of higher education. The picture becomes even more confusing when higher education standards and assessments are added to the mix. Higher Education in California: Negotiating the Maze Context: The Master Plan and Current Policy Environment The state’s higher education institutions have well-articulated agreements between them regarding their admission policies. In 1960, California’s Master Plan for Higher Education established student eligibility criteria for the three segments—the community colleges, the California State University System (CSU), and the University of California (UC). The criteria are as follows: 0 The community colleges, 106 in total, are to accept all applicants eighteen and older who can benefit from attending. ° CSU is to draw from the top one—third of high school graduates and all qualified trans— fer students from the community colleges. ' UC is to draw from the top 12.5 percent of high school graduates and accept all transfer students from the community colleges?- Since the passage of Proposition 209 on November 5, 1996, the state has strengthened its efforts to ensure that diverse groups of stu— dents will be prepared to enter and succeed in its public colleges and universities. Proposition 209 bars the use of “preferential treatment” in public employment, public education, or public contracting.3N The proposition mandated an end to the use of a fflrmative action in UC’s and CSU’s admission policies and procedures. This change highlighted the need to provide equal, high quality K~12 educational opportunities to all students—including clear signals about what students need to know and be able to do to enter higher education—in order to maintain PACE l 159 160 diverse student bodies in the state’s public uni- versity systems. In response to Proposition 209, Governor Davis proposed the Top Four Percent rule, which will allow students in the top four percent of their graduating high school class to gain admission to the UC System. The UC Board of Regents approved the proposal on March 19, 1999. The UC System estimates that it will increase the pool of eligible students by 3,600, or 1.4 percent.” The Governor has also increased the state’s commitment to provide financial aid for college students. The Governor’s 2000 budget, released on January 10, 2000, includes a $26.5 million expansion of the Cal Grant Program. Slightly more than half ($14.1 million) would be used to add more than 7,700 new Cal Grant A, B, and C awards for financially needy students; $2.4 million would be used to increase the maximum awards for students attending independent insti— tutions. The Governor proposes to increase the budget for merit aid as well. In addition, 1,000 new awards are authorized for the Assumption Program of Loans for Education, a program that forgives student loans for individuals who become teachers in schools districts facing a shortage of qualified teachers.” These efforts show a commitment to expand programs that benefit more than the traditional elite pool of prospective college students. Other measures have been taken to try to offset the effects of Proposition 209. The state has dramatically expanded funding for its pre— college outreach programs in the wake of Proposition 209; however, the evaluation com— ponent of the many outreach programs is lag— ging. The UC System has begun a large scale evaluation of its outreach programs and, when fully developed, it will have a comprehensive CRUCIAL Issues IN CALIFORNIA EDucA'rION database of student-level data for students who participate in UC’s outreach programs. The expansion in outreach-related services has come in every direction—school improve- ment, after school programs, mentoring, tutor— ing, teacher preparation—but little is known about which strategies are effective and which are not. The 1998-1999 state budget provided an extra $33.5 million in state support, plus $5 million from the UC System, for outreach. The budget required a $31 million match from the schools; therefore, the total amount of new money was $69.5 million. The UC spent approximately $137 million on outreach efforts in 1998—1999; this was more than double the $65 million spent in the previous year. The additional money expanded 1) partnerships with schools that focuses on improving long- term student performance ($15 million); 2) stu— dent-centered academic programs such as Math, Engineering, Science Achievement (MESA), Puente, and Early Academic Outreach Program (EAOP, $15 million); 3) services that promote the community college transfer func— tion; 4) programs in the Central Valley; 5) out- reach to students, families, teachers, and coun- selors; and 6) UC’s evaluation of the effective— ness of its outreach efforts.‘liii Also, the UC Office of the President is administering a $25 million federal GEAR UP (Gaining Early Awareness and Readiness for Undergraduate Programs) grant to encourage and prepare more middle schools students for college. In addition to this grant, many K—12 districts and higher education institutions in the state received one—time GEAR UP partnership grants. Finally, the UC System is requesting a $6 million increase in its proposed 2000—2001 budget in order to expand outreach initiatives.” ZOOO Connecting California’s K-12 and Higher Education Systems From the CSU System perspective, the Trustees adopted the Cornerstone Implementation Plan in January of 1998. Principle 5 of the plan states that, “The California State University will meet the need for undergraduate education in California through increasing outreach efforts and trans- fer, retention, and graduation rates, and provid— ing students a variety of pathways that may reduce the time needed to complete degrees.” In January 1996, the CSU Trustees passed a policy to reduce the need for remediation in English and mathematics. The policy calls for the CSU to work with public schools to strengthen the preparation of high school grad- uates and reduce the need for remediation for incoming students by 10 percent in 2001 and a total of no more than 10 percent in both sub- jects by 2007. The policy does not call for the elimination of remedial studies at CSU cam— puses. Strategies to be used include: strength— ening teacher preparation, setting clear stan— dards and assessing performance to ensure that students meet high school graduation and uni— versity admission standards, communicating university competence standards to K—12 stake— holders, informing high schools and communi— ty colleges about first year student performance at the CSU, developing early intervention pro— grams for high school students, using CSU stu— dents to tutor and mentor K—12 students, and providing early assessment after university admission and before enrollment.“ A priority for the campuses it to continue and expand their programs to reach tradititmal— ly underrepresented student groups.” An out— reach effort which has impacted the CSU’s approach regarding working with high schools is California Academic Partnership Program (CAPP), which was established by the legisla— ture in 1984. CAPP promotes intersegmental partnerships by awarding grants to higher edu- cation institutions, public schools, and busi- nesses to improve academic programs and increase the number of students who are pre- pared for college. CAPP focuses on identifying strategies and activities that improve college preparatory curriculum and strengthening teachers’ capacities to help all students learn the curriculum.‘5 Also, the CSU system has identified approx- imately 240 high schools that traditionally send the most students to the CSU. The system has allocated approximately 10 million dollars to its campuses to work with those high schools and focus on faculqr—to-faculty interaction. A goal is to have faculty from each system interact with each other about what they teach and what their expectations are in order to reduce the need for remediation.M Other current statewide K—16 efforts include the following: ' The California Subject Nlatter Projects (CSIVIP), a professional development net- work funded by the legislature, link UC campuses, CSU campuses, Independent Colleges, County Education Offices, Community Colleges, and high schools. The CSIVIPs develop teachers’ content knowledge and expand their teaching strategies, create a pool of expert teachers to conduct CSMP programs, and bring universities and schools together. The CSAIPS started as the Bay Area \Vriting Project, but now include math— ematics, art, foreign language, literature, and science. CSMPs serve over 500 California school district where 87 percent of the state’s teachers and 90 percent of the students are PACE I 161 located. This effort has historically been a major commitment on behalf of the state.” . SB 1697 established the College Preparatory Partnership Program as a $13 million grant program for high schools to contract with providers for “the provision of preparation courses for college admissions tests for eligi— ble high school pupils.” Priority for inclusion in the grant program will be given to schools with student populations that have low col— lege attendance, high numbers of low- income students, and demonstrated efforts to improve their college preparatory curriculum and college attendance rates.“ This bill does not, however, challenge the misaligned sys— tem. Rather, it is an attempt to work within the status quo to prepare students for the current standardized college entrance exams. Programs such as those mentioned above assist students in a wide variety of ways — from help— ing students navigate through undergraduate application processes, to academic tutoring and test preparation. Rarely, though, do outreach programs prepare students to take higher edu— cation placement exams. A Proliferation of Unaligned Placement Exams Each college and university system, and sometimes each individual campus, develops or adopts placement exams to determine which core courses students should take during their first year. These exams are not well—publicized to high school students and their parents or to teachers, nor are they aligned with current K— 12 standards and assessments; this creates a sit— uation in which it is difficult for students to prepare adequately for the placement exams. CRUCIAL ISSUES IN CALIFORNIA EDUCATION The placement tests given by most institu- tions of higher education are administered after students are already accepted by a particular campus, so students have no way to prepare specifically for the tests. Students who fail placement exams must take lower level courses that do not count for graduation and many are not informed of the tests prior to matriculation into the university.” Below are descriptions of placement exams administered by the California Community College System, the California State University System, and the University of California System. 0 Coinmunity College System. Community college leaders are concerned about the inability of many of their students to complete credit level or transfer level work when they enter a community college. Remedial rates vary greatly depending on the high school the student attended. For example, approximately one—third of the students who graduated from higher performing (on standardized tests) high schools in the Santa Barbara area and matriculated into Santa Barbara community college could not do credit level mathematics work; that number for lower performing high schools is approximately two—thirds.40 Community colleges in California administer approximately fifty—eight different placement tests; there are approximately 678 combina— tions of exams given each year by the campus— es. This does not include exams developed by individual faculty members. The list does include many tests developed by ACT, the UC and CSU systems and campuses, and the College Boart .*' 0 CSU System. The CSU System has two placement exams that are used by every uni- ZOOO Connecting California’s K-12 and Higher Education Systems versity in the system: the Entry Level Mathematics Examination (ELM) and the English Placement Test (EPT). The tests were designed to assess the skills of entering CSU students in mathematics and in reading and writing, respectively, so that students can be directed to the appropriate courses or programs to help them attain the necessary skills. All entering undergraduates must take both exams. The tests are not used in the admission process. Campuses and individual faculty members also develop and use their own placement exams in core subject areas. In 1998, the CSU system reported that a record 47 percent of its freshmen had to take remedial English; 54 percent enrolled in remedial mathematics.43 0 UC System. The UC system administers the Subject A English examination. It does not have a mathematics placement exam. Campuses and individual faculty members also develop and use their own placement exams in core subject areas. In sum, each system, and often each campus, has its own placement criteria, creating a con— fusing and misaligned set of policies. Admission Requirements University of California System. UC System standards have long had an affect on the K—12 system. In a previous report, PACE researchers wrote that, “Historically, high schools have been attentive to changes in admission requirements of the postsecondary institutions and other higher education—initiated curricular directions. . . .The University of California’s entrance requirements have long been viewed as a prima— ry determinant of high school curriculum.” The UC System and CSU joined forces in 1999 to align their required course sequences; the new requirements are entitled “a-g” and include Vlsual and Performing arts. The “a-g” require— ments, listed below, have historically driven the college preparatory curricula in the state’s high schools. The intent of the subject area require— ments is to assure that students can participate fully in the first year program at the University in a wide variety of fields.“ In order to gain entry into an institution in the UC System, applicants must 1) complete the required “a—g” courses sequence, 2) meet the Scholarship Requirement, and 3) meet the Examination Requirement. The Scholarship Requirement specifies the grade point average (through the use of an Eligibility Index) that applicants must earn in the “a-g” courses to be eligible for admission. The Examination Requirement stipulates that applicants must submit SAT—I or ACT scores and SAT—II scores-18 Three SAT—II tests are required by the UC institutions: 1) Writing, 2) mathemat- ics, and 3) a subject in an area appropriate to a student’s chosen area of study in college (cho— sen from English Literature, Foreign Language, Science, or Social Studies). There has been a recent increase in the per- centage of California high school students taking a rigorous sequence of courses. In 1998, over 36 percent of high school graduates have finished the “a—f” requirements (as they were previously an increase of over called) for the UC System 3 percentage points since 1996. Also in 1998, over three times as many California students took and passed Advanced Placement exams than in 1984—1985. Over 13 percent of all juniors and seniors passed the exams, compared with just over 9 percent nationally.“ A major problem exists, however; there is not enough PACE I 163 16-} Course History/Social Science English Mathematics Laboratory Science Language Other Than EngHsh College Preparatory Electives Visual and Performing Arts Years 2 years, including 1 year of US history or .5 year of US history and .5 year of civics or American government; and 1 year of world history, cultures, and geography. 4 years of college preparatory English that includes frequent and reg- ular writing, and reading of classic and modern literature. Not more than 2 semesters of 9th grade English can be used to meet this requirement. 3 years required, 4 years recommended, of college preparatory mathematics that includes the topics covered in elementary and advanced algebra and 2 and 3 dimensional geometry. Math courses taken in grades 7 and 8 may be used to fulfill part of this require— ment if your high school accepts them as equivalent to its own courses. 2 years required, 3 recommended. 2 years of laboratory science pro- viding fundamental knowledge in at least 2 of these 3 areas: biology, chemistry, and physics. Lab courses in earth/space sciences are acceptable if they have as prerequisite or provide basic knowledge in biology, chemistry, or physics. Not more than 1 year of 9th grade laboratory science can be used to meet this requirement. 2 years required, 3 recommended. Courses should emphasize speaking and understanding and include instruction in grammar, vocabulary, reading, and composition. 2 years. Courses must be chosen from the following areas: visual and performing arts, history, social science, English, advanced math, laboratory science, and language other than English. 1 year. Courses must be chosen from the following areas: art dance, drama/theatre, or music. Table 1. A-G Requirements consistency in terms of what is offered in a—g courses across the state. The UC System certi— fies that a course meets the necessary criteria by ensuring that the course title meets its standards. If substantial changes are made, the UC System must be notified. Content of the courses varies and, over time, does not always match the titles of the course?“ Community College System. The California Community Colleges must admit CRUCIAL Issues IN CALIFORNIA EDUCATION any California resident, and may admit any person, who is at least eighteen years old and capable of profiting from the instruction offered. The community colleges may also admit a nonresident who has a high school diploma or the equivalent.“ California State University System. Freshmen are admitted based on courses taken in high school and their ranking on the eligibil— ity index, a combination of high school grades ZOOO Connecting California’s K—12 and Higher Education Systems and either the SAT or ACT composite scores. To qualify for admission 3 student must satisfy the following criteria: 1) be a high school grad— uate, 2) have completed the course subject requirements with a grade of C or better, 3) and earned a qualifying ranking on the eligibili- ty index. Again, the course subject require— ments, the a—g requirements, are the same as those required by the UC System. California residents with a school grade point average of 3.0 or better are not required to submit test scores, but are encouraged to do so. The SAT—I, like the CSU math placement exam, covers algebra and geometry; however, the SAT—I also covers topics not stressed in the CSU exam such as ratios and data interpreta- tions. A score above 550 on the mathematics section of the SAT exempts students from tak— ing the CSU’s mathematics placement exam. The SAT and Advanced Placement Results from the traditional measure of college preparation, the SAT", are encouraging‘. California’s state standards and assessments, however, are not aligned with the SAT; SAT results are not indicators of students’ overall academic performance in school. SAT results are intended to measure students7 readiness for college—level academic work. In 1997, 47 per— cent of California’s high school seniors took the SAT, four percentage points higher than the national average. Percentage—wise, substantially more California students were not native English speakers and were from economically disadvantaged families. Nevertheless, average scores increased to the highest point since 1973—1974: 497 in the verbal section (the national score was 505) and 516 in math (the national score was 512).“ The assessment chapter provides more detailed information and data regarding the SAT. Although causality can not be proven with these data, the table below shows that the more academic courses a student completes, the higher the SAT scores.” Currently, because there are three required SAT—II portions and only two SAT-I portions. the SAT—II factors more heavily for campuses that choose to use all three SAT-II exams in calculating their academic index. It is unknown what percentage of non—honors or non-AP track students are aware of, and prepare for, the SAT-II. If the SAT—H is not a commonly known assessment, it could be a barrier for students Academic Courses California SAT Taken Math Scores 20-plus 567 19-195 525 18-185 503 17—175 481 16—165 472 15-155 470 Less than 15 456 Academic Courses California SAT Taken Verbal Scores 20—plus 550 19-19.5 508 18—185 488 17—175 468 16—165 458 15-155 452 Less than 15 434 Table 2. Number of Academic Courses Completed and SAT Score, 1998 P ;\ C E W l()5 166 with regard to becoming eligible for admission to the UC System. In 1997, the UC Office of the President conducted a study to investigate the likely result of removing the SAT—I as an admission criterion. The study considered how eliminat- ing the use of the SAT—I would affect UC eligi— bility rates. It found that such an action would produce small changes in the eligibility of African Americans (from 2.8 percent to 2.3 per- cent), Asian Americans (from 30 percent to 29 percent), and Latinos (from 3.8 percent to 4.0 percent); the largest change would be an increase in the eligibility rate for whites (from 12.7 percent to 14.8 percent). A California Postsecondary Education Commission report on eligibility found that only 2.5 percent of California’s public high school graduates are ineligible to attend a UC campus solely on the basis on inadequate test scores. Approximately 63 percent of graduates were ineligible because they had major course omissions or grade defi— ciencies, or had attended a high school that has a college preparation program that is approved by the UC system?x While many high schools offer honors courses, they can vary in quality depending on the pedagogy and curriculum. The Advanced Placement (AP) program, sponsored by the College Board, included college preparation courses and tests gauged to an external stan— dard. AP exams are given to high school stu- dents who take AP courses in core subject areas. A passing grade on an exam is accepted by many colleges and universities for college credit. There is little consistency, though, in the system. Students can score well on an AP exam, but depending on the higher education institution, credit can be denied. High AP CRUCIAL ISSUES IN CALIFORNIA EDUCATION scores on the English and mathematics exams also exempt students from taking those place- ment exams at UC and CSU institutions. The UC System also allows extra weighting of grades earned in AP courses completed during a student’s sophomore and junior years in high school. California’s 1999 Advanced Placement exam passing rate of 14.8 (per 100 test—takers) is above the national average of 9.4.“ Bringing it All Together: K-16 Policy Alignment The issues of articulation and of connecting K- 12 and higher education standards affect a large proportion of California’s college age popula- tion. A greater percentage of California stu- dents are attending the state’s two and four- year institutions of higher education than ever before. In 1996, over 66 percent of California high school seniors enrolled in a postsecondary program within two years of graduation, and that is expected to grow in the 2 lst century.“ Two major issues that arise when K—16 alignment discussions are held is that K—12 and higher education have different missions and not all students who graduate from K—12 enter higher education institutions. But aligning poli— cies between the two systems would not change the mission of either entity, nor leave out the noncollege-bound. Rather, if all high school students are held to high standards that are aligned with college entrance requirements, it could increase standards for all students and could help ensure that all students receive clear messages about what they need to know and be able to do to enter college. K—16 reform as described in this chapter could provide previ- ZOOO Connecting California’s K-12 and Higher Education Systems ously noncollege bound students with the information and opportunity necessary to make the choice to pursue postsecondary education. It could also address equity concerns in the wake of Proposition 209 by ensuring that every high school senior’s curriculum relates to col— lege or university entrance standards. This is crucial, given that approximately 70 percent of all high school graduate will attend some form of postsecondary education. This, in addition to a well—conceived and implemented K—16 data and accountability system, could cut down on the need for remedial or developmental education. Approximately 50 percent of all col— lege and university students in the state enroll in some form of remedial education during the course of their college—going years. Challenges California is striving to align standards and assessments within the K—12 policy arena. Another challenge is to link K—12 and higher education policies. The lack of compatibility between the two systems is evident in two ways: 1) policy—making bodies in the two education sectors have minimal interaction and opportuni— ties to collaborate, and 2) assessments and other policies are not aligned across the K-16 system in terms of what they are asking students to know and be able to do. None of the state’s public higher education admission and placement exams is aligned with the California State Board of Education’s curriculum frameworks or the a ug— mented STAR. Compounding this is the fact that the various tests high school students take all have different purposes, including preparation for college, freshman placement, prediction of uni— versity performance, determining trends in per— formance on statewide K—lZ standards, and com— paring state test results to national norms. While K-12 and higher education have different pur- poses and, consequently, the assessments will differ—we argue here that the process can be streamlined, data can be made more useful for all stakeholders, and that all stakeholders should be brought to the table to discuss these issues. It remains to be seen whether the STAR assessment system for eleventh graders and col- lege entrance level policies will be aligned in order to send clear messages about what stu- dents need to know and be able to do. This challenge will, perhaps, be more difficult than the one facing the K—12 system since the suc— cess of the effort will depend not only on the success of the alignment efforts in the K—12 system, but on clear articulation of what stu— dents need to know and do to enter college at all levels of the higher education system, agree— ment among K—16 stakeholders, and an institu— tional center from which reforms can be made and implemented. Currently, many of the reforms in one system are made in isolation from the other system, although recent discus— sions of test consolidation have moved toward a K—16 inclusive reform environment. Thus, from an assessment perspective, much work remains. The table below pro- vides brief snapshot data regarding the major assessments and standards utilized in California during the transition between high school and college. The Education Trust, a national nonprofit focused on K—16 reform, recommends that states take the following actions: ° Make sure that the content of all assess— ments used at the high school exit level and college entrance level are made public and shared widely. PACE I 167 168 0 All high school students should be required to complete a rigorous, college preparatory, academic core. ' Redundancies and mixed messages in assess- ment at the juncture of high school and col- lege must be eliminated. 0 Reward high performing students by enabling them to begin college work early; provide extra time and help for high school students who are struggling.H The California Education Roundtable took a slightly different perspective than did the Test or Standard STAR CA Augmented STAR STAR SABE/2 CAAAS Matrix Golden State Exams Exit Exam Tests of General Educational Development California High School Proficiency Exam State Content Standards High School Graduation Standards PSAT PLAN NAEPV’ Advanced Placement Purpose K—12 Assessments and Standards Measures knowledge of broad content areas; allows for district and national comparisons. Augmented items are developed to test performance against CA standards. Spanish language test given to students to gauge command of content in primary language. Proposed as a measure of progress toward standards. Matrix sample would assess a broad array of content items. End—of—course exams. Student recognition for high performance and seal of merit. Will be required to pass in order to earn high school diploma. Provides an alternative way to complete high school. Proficiency test that allows students to graduate early from high schooL Outlines what students need to know and be able to do at each grade level. Defines what students are expected to learn by the time they gra- date from high school. Preparation for SAT—l; selection of National Merit Scholars Preparation for ACT; measurement of student knowledge. National exam that allows for comparison of performance across states. High Scores on individual subject tests exempt students from cer- tain baccalaureate courses. Table 3. K-16 Assessments and Standards CRUCIAL Issues IN CALIFORNIA EDUCATION 2000 Connecting California’s K-12 and Higher Education Systems Higher Education Assessments ACT SAT-l IV" Approximately 58 different Community College placement exams UC English Language Arts placement test CSU Entry Level English Placement Test (EPT) CSU Entry Level Math Exam (ELM) SAT To predict readiness for college-level work. Used for college admission. Alternative to ACT. Measures knowledge of subjects areas and predictor of college success. Used for college admission. Alternative to SAT-l. Measure knowledge of subject areas; used for college placement. Places entering students into appropriate courses. Places entering students into appropriate courses. Places entering students into appropriate courses. Places entering students into appropriate courses. Table 3 continued. K-16 Assessments and Standards Education Trust when it analyzed the state’s K— 16 policy environment and outlined the follow— ing problems and disjunctures: ° For many seniors, too little is demanded of them academically. ° The options for acceleration to college and the opportunities to sample college—level work are not equitably available for high school seniors. ° The senior year is not effectively employed to assist students in catching up and/or becoming fully prepared for college—level work or for the workforce. ' There could be more higher education courses offered at high school . ' By sending out early acceptance notification, colleges foster a slacking off of academic effort on the part of many seniors. ' The high school senior who is uncertain of his or her academic options has few opportu— nities to meet individually with an academic counselor. ° High school seniors are often unaware of their level of preparedness for college—level work, particularly in mathematics and English language arts. The Roundtable recommends coordinating the multiple K—lo standards—development efforts.“ Another crucial issue that is rarely addressed is that of the different teaching, learning, and assessment styles used in the K—12 system as compared to the higher education systems. Could one contributing factor to the need for remedial or developmental education he that students are underprepared for lecture styles and multiple choice exams utilized in many col— leges and universities? \Ve propose that policymakers address the following questions in order to determine the depth of the disjunctures between the K—12 and higher education systems in the state. \Vhile we can address some of these issues, others need to be examined in close detail by educators, poli— cymakers, and researchers. PA C E I 100 How do the state’s academic content standards and graduation requirements compare to the content of beginning level courses at the state’s colleges and universi- ties? If there is not a continuum of learning when a student leaves high school and enters higher education, that student can suffer, acad- emically, in college. While we do not have data to address this question specifically, we can state that there is a gap between what is required to graduate from high school and what is required to become eligible for admis— sion to the UC and CSU systems. While the curricular requirements have been well-aligned between the UC and CSU systems, they are not aligned with high school practices across the state. In 1997, the State Department of Education surveyed nearly 300 high schools regarding the nature of their high school pro— grams. It reported that California high schools are expecting to increase their high school graduation requirements in the areas of requiring 2 years of math, including algebra and geome— try. Currently on 5 Jr percent of high schools have this requirement, with 70 percent indi— cating that they will have such a requirement for the class of 2004. Currently about 80 percent of high schools require 4 years of English and 85 percent require 2 years of laboratory science. If all students are to pass a rigorous exit exam, the percent of students taking these courses, whether required as graduation criteria or not, may need to be increased.50 Thus, high school graduation requirements often fall short of higher education admission requirements. This has implications if, for CRUCIAL lSSLTES IN CALIFORNIA EI)L'(:.-\'I‘I()N example, the minimum high school graduation standards do not permit students to do credit level work in higher education, or if high schools’ curricular offerings are gauged to low level graduation standards. Also, students are not aware of this disconnect, they might not be preparing adequately for higher education. Is the STAR program assessment, including the augmented test, asking stu- dents to know and be able to do the same knowledge and skills that are required by UC and CSU admission and placement policies? How do the individual institutional placement exams relate to each other in terms of content, and to the STAR assessment? More specific details of differences between the K-12 and posr—ycmml/I'Iy assessments currently admin— istered are outlined in a forthcoming study by RAND Corporation researchers. Researchers analyzed the alignment of mathematics content and format between the SAT-l, ACT, SAT—H, augmented STAR, Golden State Exams (High School mathematics, First year Algebra, and Geometry), and the CSU mathematics place- ment exam. Fewer than ten percent of the SAT— I and augmented STAR items required a mem— orized formula, in contrast to 25 percent of the GSE Geometry problems. Whereas the GSE Algebra and SAT—H mathematics level IIC assessments made little use of figures and tables, the augmented STAR and GSE Geometiy exams included many illustrations. No more than 25 percent of the items found on California’s university admission and placement assessments were seen as “authentic” or applied, as compared to 58 percent of the aug— mented STAR items. For college entrance exams such as the SAT—ll and ACT, relatively ZOOO Connecting California’s K-‘I2 and Higher Education Systems greater emphasis was given to Geometry and trigonometry, whereas trigonometry was large— ly absent from the CSU, SAT—I and augmented STAR exams.64 As this brief discussion illus— trates, the list of knowledge and skills required by the various exams is diverse, extensive, and confusing. While some content and structural misalignments may be necessary due to legiti— mate differences between the tests, California students are subjected to a babel of tests and standards that could be aligned better. In addi— tion, unless the curriculum, standards, and assessments are carefully aligned across the K— 16 system, the effectiveness of the proposed accountability system could be undermined.“ Should the statewide accountability pro— gram currently being developed hold high schools accountable for offering college preparatory work including AP courses? Should it hold higher education institutions accountable for graduating their students? As discussed earlier, the only data currently includ— ed in the accountability index are the STAR assessment results, although there are plans to increase the number of indicators and create a more comprehensive accountability system. Can state education agencies link their databases in order to assess needs through- out the K-16 continuum? Can researchers and policymakers use the data to understand any current inequalities in terms of who enters and who graduates from higher education insti— tutions in the state? Can issues such as college preparation be addressed by tracking student success in higher education by district or high school? It is impossible to address needs when there is not a comprehensive, linked, K—lo data system in the state. It is too soon to tell if the state’s education databases will be linked. In 1999, AB 1570, sponsored by \Hllaraigosa, was chaptered. It requires the California Postsecondary Education Commission, in developing a comprehensive student database, to, “ensure that the database supports longitu— dinal studies of individual students as they move through the state’s educational segments, that it provides for the interactive use of data, and that it provides each of the educational seg— ments access to the data...”“’ A data exchange agreement has been formalized between the UC, CSU, and Community College systems to track UC outreach program participants into public higher education.67 This is a start, but, in order to address needs statewide for all stu— dents, more work needs to be done create a K— 16 data system. Do the state’s schools have a sufficient number of counselors whose main role is to advise students about college options? Do all students have early, repeated, access to col— lege preparation information? Counselors in every state, and California is no exception, are overwhelmed by the variety of responsibilities they must fulfill every day. Counselors face a long list of roles, including scheduling classes, administering tests, and assisting students with emotional and psychological issues. Often, they do not have time to do one—on—one college counseling. California has one of the worst counselor—to—student ratio in the country. Are there university outreach programs that are connected with local schools and districts? Are they connected with national, state, and local outreach programs? One of the challenges of the outreach environment in California is the proliferation of unconnected outreach programs. There is not a clear sense of the level of overlap, or the gaps, in types of PACE I 171 services or populations served. The joining of MESA, Puente, and EAOP is a good start. Additional research needs to be done in this area, in additional to more evaluative work. Are there articulation agreements between public universities, community colleges, and high schools? California has strong articulation agreements between its higher education segments. What is lacking is a {(—16 data system that can track student progress through the segments. Is there an institutional center or mecha- nism that will allow K-12 and higher educa- tion stakeholders to work together and overcome fragmentation concerning such areas as policy alignment, faculty interac— tion, teacher training and pedagogical issues, and K—16 information systems? No one is held accountable for K—16 reform in California. There is no entity charged with developing and implementing K—lo reform. Many of these issues, such as equitable access to college preparatory courses and to higher education in general are politically charged issues that can quickly turn into turf battles. A group needs to be identified that will have the authority to represent all stakeholders. Groups such as the Education Roundtable and the lntersegmental Committee of the Academic Senates are not appropriate bodies since they do not include a representative from the Governor’s office, or from the community at large. Until such a group is put together, many of these reforms will be difficult to tackle. Oppm'tunitiesfor Change Unlike many other states, California has several intersegmental groups that can begin to work CRL‘CIAL lssi'ias IN CALIFORNIA l‘il)L'(‘.;\'l‘l(),\‘ together across sectors. Such groups as the Education Roundtable, the California Postsecondary Education Commission, the lntersegmental Committee of the Academic Senates of the UCs, CSUs, and community colleges are all in good positions to work together with other K-16 stakeholders to create a more aligned system. The K-16 policy envi- ronment is slowly evolving into a more coher— ent whole, rather than disparate pieces. It will be crucial to bring K-12 and higher education stakeholders together, including Governor Davis’s key education aides, to create a more cohesive system of education in California. California educators and policymakers must cre- ate an institutional center for these reforms. If improved alignment is to occur, the K—12 and higher education systems need to determine whether one system will adopt the others’ assess— ments; whether yet another assessment will be adopted; if a separate K—12 and higher education assessment will be acceptable, but data will be shared and utilized across systems; or if the sta— M tus quo will remain. Curricular alignment needs to be addressed as well. We applaud California’s educators, researchers, and policy— makers for the efforts made to date, and urge them to address the issues and questions raised in this chapter. This work must be completed while balancing all the other facets of education, especially 1) creating incentives for all K—16 stakeholders to take an active role in K—16 reform, 2) ensuring that changes improve oppor— tunities for all students to enter and graduate from higher education, and 3)making sure that capacity issues are addressed. This is a tall order to fill, but if California succeeds it could become a national model for K-16 reform. ZOOO Connecting California’s K—12 and Higher Education Systems Notes 1. T he augmented STAR is the statewide K—lZ assessment that is aligned with the state’s content standards. The Golden State exams are the state’s end of course exams that are not aligned with the content standards. ii. The Round Table is an association of the chiefs of the systems (or segments) of education in California. Its mem- bers include the State Superintendent of Public Instruction, the President of the University of California, the Chancellor of the California State University, the Chancellor of the California Community Colleges, the President of the Association of Independent Colleges and Universities, and the Executive Director of the California Postsecondarv Education Commission. The Round Table focuses on issues affecting all segments of education. Its agenda is implemented through working committees composed of staff, faculty and students managed by its operating arm, the Intersegmental T \ Coordinating Committee. ll]. EAOP, MESA, and Puente have joined forces to increase their efforts in the development of individual academic plans and preparation of students for college tests. The new entity is called the EMP (Earlv Mesa Puente) Outreach Collaborative. iv. Ar’Iost students in California who proceed to a public university in the state take the SAT, rather than the ACT. v. The SAT, administered by the College Board, is used by instimtions of higher education as an indicator of stu- dents’ readiness to take college—level work. This is a different exam than the Stanford 9, also called the SAT 9, which is used as a statewide K—12 assessment in California. vi. NAEP is the National Assessment of Educational Progress. vii. Universities in the UC System require that applicants submit three SAT—II tests: \Vriting; Mathematics level 1, 1c, or Be; and one of the following: English literature, foreign language, science, or social studies. These are used for admission and placement purposes. 1. California Education Round Table, “Collaborative Initiatives to Improve Student Learning and Academic Performance, Kindergarten Through College,” 30 October 1995. 2. Legislative Analyst’s Office, “Ii—13 Master Plan: Starting the Process,” May 1999, 5. 3. Senate Office of Research, “Master Plan Overview,” August 20, 1999, —l. 4. Legislative Analyst’s Office, “K—12 AIaster Plan: Starting the Process,” May 1999, 5. ‘JI . California Education Round Table, “Collaborative Initiatives to Improve Student Learning and Academic Performance, Kindergarten 'l‘hrough College,” 30 October 1995. 6. Gerald C. Hayward, “Creating State Curricular and Graduation Standards,” PACE Newsletter, Fall 1997. 6—7; Ed Source, “How California Compares: Indicators and Implications for Our Public Schools,” November 1998, -l. 7. Gerald C. Hayward, “Creating State Curricular and Graduation Standards,” PACE Newsletter, Fall 1997, 6—7. 8. Dave Jolly, director of the CSU System’s California Academic Partnership Program, telephone conversation with author, 2-1 February 3000. 9. Ed Source, “How California Compares: Indicators and Implications for Our Public Schools,” November 1998, —l. 10. California Business for Education Excellence, “l listory ofCalifornia’s l\'—l2 Assessment System,” CBEE Backgrounder, n.d.; California State Department of Education, “Chronology of State Testing in California," 8 June, 1990, 29. 11. California Business for Education Excellence, “History ofCalifornia‘s K—lZ Assessment System," CBEE Backgrounder, n.d. PACE I 173 12. http://www.leginfo.ca.gov/; Ed Source, “An Update: Special Legislative Session, 1999,” May 1999, 1, 13. Dave lolly, director of the CSU System’s California Academic Partnership Program and member of the state’s exit exam panel, telephone conversation with author, 24 February 2000. 14. Ed Source, “How California Compares: Indicators and Implications for Our Public Schools,” November 1998, 4. 15. California Department of Education, “Golden State Examination,” October 1996, 1-2. 16. http://www.legin fo.ca.gov/. 17. Richard Lee Colvin, “Education System Faces a Test of Bridge Building,” Los Angelcs Times, 19 May 19, 1999, see. B, p. 2. 18. Ed Source, “Shifting the Focus to Learning: California’s Accountability Debates: State Experts Discuss How Accountability Can Improve Student Achievement,” June 1998, 1. 19. Gerald C. Hayward, “Creating State Curricular and Graduation Standards,” PACE Newsletter, Fall 1997, 6-7, 20. Ed Source, “Governor’s Proposed Budget for 1999—2000,” February 1999, 5—6. 21. http://www.legi n fo.ca.gov/. [\J Iv . http://www.leginfo.ca.gov/. 23. Peter Schrag, “School r’\ccountability — Fast Road to a Voucher?,” Sacramento Bee, 25 August 1999. Available at wysiwyg://4/http://www.capitolalert.com/voices/schrag/schrag-augustZ 5-99.html. 24. Ed Source, “Governor’s Proposed Budget for 1999—2000,” February 1999, 1. 3 5. \I‘ww.lao.ca .gov/. 26. Jennifer Best, “Superintendents: Quit Burying Us in Legislation,” Santa A'Iaria Times, Santa AIaria, California, 25 August 1999. 37. Ed Source, “An Overview of Current Challenges: High Schools in the Hot Seat,” June 1999, 11. 2 8. http://wwwchre.vt.cdu/. 29. http://www.governorca.gov/s/issues/education/education6.html. 30. http://www.governoInca.gov/s/statc/ynr99.doceducation. 3 1. http://www.ucop.cdu/outreach/factsheets/LKI_budget.html. 32. http://www.uc<)p.edu/outreach/outl()(Ik/()L3_inbrief.html. 33. California State University System, Chancellor’s Office, Board ofTrustees meeting minutes, IV/Iarch 17—18, 1998. 3—1. http://www.calstate.edu/cornerstoneS/reports/implIIIent.htIIIl. 35. http://wwwco.calstate.edu/aa/ar/()utreach .html. 36. Dave Jolly, director of the CSU Systems’s California Academic Partnership Program, telephone conversation with author, 2~l February 2000. 174 CRUCIAL ISSUES IN CALIFORNIA EDucATION zooo Connecting California’s K—‘l2 and Higher Education Systems ‘ 37‘ Inverness Research ASSOCIMCS, “California Subject Matter Projects: A Statewide Professional Development System, January 1998. 38. http://\mvw.leginfo.ca.gov/. 39. Michael W. Kirst, “Conflicting Signals: the Need to Align St . 1 _ ate Assessments,” Policy Analysis for California Education newsletter, Summer 1998, 3. ' ' 40. Santa Barbara City College, scores of local high school graduates on the Santa Barbara City College Mathematics, Reading, and Writing Placement Exams, Fall 1999. i T 41. CaliforniaCommunity College Chancellor’s Office, “Assessment Instruments Used in the California Community Colleges, Academic Year 1997—1998,” October 12, 1999. i 42. iVIichael W. Kirst, “Bridging the Remediation Gap,” Education “'eek, September 9, 1998, 1. 43. Policy Analysis for California Education, 1989. 44. http://www.ucop.edu/ pathways. 45. Sam Bersola, Amy I’Iightower, and Michael \V. Kirst, forthcoming publication entitled, “The California Case Study for Stanford University’s Bridge Project,” Stanford University, November 1999. 46. Ed Source, “How California Compares: Indicators and Implications for Our Public Schools,” November 1998, 4. 47. Dave Jolly, director of the CSU System’s California Academic Partnership Program, telephone conversation with author, 24 February, 2000. 48. http://\Avw.ccco.edu/mission/htm. 49. Ed Source, “How California Compares: Indicators and Implications for Our Public Schools,” November 1998, 4. 50. Tanya Schevitz, “California SAT Scores Below US Average in Verbal Skills: Students” Math Scores Better Than in Other States,” San Francisco Chronicle, 31 August, 1999.: http://sfgate.c<)m/cgi— bin/articlecgi?file=/chronicle/archives/1999/08/31/.\l.\'106986.DTI.. 51. Rebecca Zwick. “Eliminating Standardized Tests in College Admissions: The New Affirmative Action?," Phi Delta Kappan, December 1999, 320—324. VI 2. vwsiuyg://story.content.1 1/http://\nwv.ne.ngtrfng/99—09—0 1%clas3 01 .asp? PUID—4902. 'J] .u . . California Postsecondary Education Commission, “Factsheet 99—1,” February 1999, 1. 54. The Education Trust. “'l‘hinking K-16,” Vol. 3, Issue 3, Fall 1999, \\ashiiigt()n, DC, 39—30. 55. California, “Policy Recommendations for Transition from High School: Taking Responsibility for the High School Senior Year,” prepared by the Intersegmental Coordinating Committee, December 1996, 3, 6. 56. Cathy George, “Research Brief: High School Reform: \Vlaere Are We N()\\'?,” California State Department of Education, Office of Policy and Evaluation, February 1999, Sacramento, California. Available at http://wuuzcdeca. gov/ ope/ sse/ hi gh.htm. 57. Le Ni—Nhuan, Abby Robyn, and Laura Hamilton. Forthcoming paper entitlet , “Alignment Analysis: California Case Report,” by the RAND Corporation, Commissioned by Stanford University, November 1999. 58. Ed Source. “How California Compares: Indicators and Implications for 0111‘ Public Schools,” November 1998, 4. 59. http://www.ucop.edu/outreach/outlook/OLLlegislativehtinl. PACE I 175 60 University of California Office of the President, “Annual Report on the Outreach Efforts of the University of California,” May 1999. 61. The Education Trust. “Thinking K—lé,” Wil. 3, Issue 2, Fall 1999, “fishington, DC, 30. 176 CRUCIAL ISSUES IN CALIFORNIA EDUCATION 2000 Chapter 9 Alignment Among Secondary and Post-Secondary Assessments in California Vi—Nhuan Le, Laura Hamilton, and Abby Robyn The RAND Corporation Background As students progress through high school and into institutions of higher education, they take numerous tests that vary in scope, content, and purpose. At the K—12 level, almost all of the states are currently using or develop— ing assessments that are aligned with state stan— dards.1 Some of these assessment programs rely on commercially available, standardized, multi— ple—choice tests administered to every student, whereas others hire their own staff to develop items in multiple formats (including, for exam— ple, essays or portfolios) which are then admin— istered in a matrix sampling scheme (i.e., not every student completes every item). In some states, scores on these tests are used to inform decisions about grade promotion and gradua— tion. Students who plan to attend college also take one or more admissions tests, such as the ACT or the SAT I and H, and may take Advanced Placement (AP) exams, which pro- vide college credit for high school coursework. \Vhen they arrive at college, many students are asked to take placement exams which are used to assign them to appropriate courses. These exams are especially prevalent in large state higher—education systems. Assessments play a prominent role in the transition from high school to college. In most cases, test scores are among the major criteria used to determine who is accepted into an institution and who is assigned to remedial courses. Although these scores are imperfect, indirect measures of what students have accom- plished, they often provide valuable informa— tion that may improve the decision—making process. A well—constructed test of achievement in a particular subject area constitutes a sample of performance from a larger domain to which the user wishes to generalize. This domain will vary depending in part on the purpose of the assessment. A statewide achievement test might be designed to sample from a range of topics and to cover material learned across several grades. A college placement exam, in contrast, may have a narrower focus, reflecting the cur— riculum of a particular course. Consequently, these tests may not resemble one another close— ly in the constructs that they measure. In other words, they may not be well aligned. The goal of the present study is to investi— gate the degree of alignment among these dif— ferent types of tests in six case—study states, and to explore the potential consequences of any misalignment. We will compare assessments PACF I 177 used for college admissions, college placement, and K-IZ system monitoring and accountability in each state, classifying items along several dimensions. For each state, we will obtain a summary of the ways in which the assessments are and are not aligned with one another, and discuss possible implications. This report pre- sents the results for California. It is important to note that we do not claim that all tests must be well aligned. The conditions under which alignment is important are discussed later. This analysis is part of a larger study com- missioned by Stanford University. “The Bridge Project: Strengthening K-16 Transition Policies” is a national study funded by the Pew Charitable Trusts and the US. Department of Educations Office of Educational Research and Improvement. It focuses on the need to increase the alignment between higher educa— tion admissions-related requirements and K-12 curriculum frameworks, standards, and assess- ments. The study was prompted in part by a perceived disjuncture between standards for college admission and placement, on the one hand, and high school curriculum and instruc— tion on the other.2 The salience of this problem is underscored by a 1995 National Education Association survey in which 82 percent of House and Senate Education chairs polled viewed the improvement of connections between colleges and schools as among their highest priorities for higher education.‘ Admissions policies are a primary way in which colleges influence the education of secondary students, and the tests that are given as part of the admissions and placement processes are a major component of these policies. CRUCIAL lSSL'l-ZS IN CALIFORNIA EDUCATION Importance of Alignment There are at least three major ways to think about alignment among different assessments. First, the content and format of test items send messages to students who take them. Particularly when tests have high stakes attached, such as graduation from high school, selection into college, or placement into a remedial program, they can be expected to influence the behaviors of examinees and, in some cases, their instructors. For example, multiple-choice tests are often criticized for encouraging an emphasis on memorization of discrete facts rather than extended problem solving. It is important to determine whether tests are sending a consistent message to stu- dents regarding what kinds of knowledge and skills are valued by the institutions they wish to attend. It is also critical that students have ample opportunity to prepare in appropriate ways for high—stakes assessments. If students enter college unaware of what skills they will be expected to demonstrate on a placement exam, they may not perform as well as they would if given the opportunity to prepare. It is impor- tant to note here that the signals a test sends are somewhat distinct from the measurement properties of the test. For example, it is possi— ble that a multiple—choice test does indeed measure complex problem—solving skill, but that examinees and instructors perceive the test as being focused on memorization or recall. The importance of aligning the various aspects of the educational system to support a common set of goals has been recognized by advocates of systemic reform,+ promoters of test—based accountability systems,‘ and many ZOOO Alignment Among Secondary and Post-Secondary Assessments in California others involved in educational reform efforts. Especially important to standards—based reform efforts is the degree to which the standards and the assessments used to measure progress toward them are consistent with one another. A recent study by Webb found varied degrees of alignment between tests and standards in math and science in four states." A content analysis of teacher licensing tests conducted by the Education Trust showed that most such tests required little more than high school level knowledge but that some were more rigorous than othersf Standards and assessments that are not aligned with one another or that encourage a focus on low—level skills create mixed messages and confusion for students, teachers, and others involved in promoting stu— dent learning. The second aspect of alignment involves the consistency with which students are rank ordered or classified into categories or pro— grams (e.g., remedial instruction) by different tests. If two tests are designed to measure the same abilities, evidence must be gathered to show that students who do well on one tend to do well on the other. Although most tests of academic achievement tend to correlate highly with one another, even when subject and item format differ, it is nonetheless important to evaluate the magnitude of this correlation and the consistency of any classification that results from test use. Scores on a high school math exam should, for example, correlate highly with scores on a math placement test administered by the higher education system. Finally, it is essential that the standards used for decision making be comparable across assessments and set in a technically sound and credible manner. The placement process often involves selecting a cut score on an exam and assigning students to programs or courses based on whether or not their scores exceeded this cut score. Statewide assessment programs are increasingly reporting student performance in terms of standards similar to the achieve- ment levels used on the National Assessment of Educational Progress (NAEP). These efforts have been criticized in part because the process of mapping performance to descriptors relies heavily on judgments that are often error— prone.8 Even so, assessment results continue to be reported in terms of standards, and it is therefore important to determine whether the standards set on different tests provide reason- ably consistent information about students. If a student is labeled “Advanced” or “Proficient” on a state test but is unable to reach the level of performance on a placement test necessary to avoid remedial coursework, there is reason to believe that the standards used on one or both tests are inappropriate. The current project is designed to provide information concerning the degree and nature of alignment among tests used for K—12 system monitoring and accountability, college admis— sions, and college placement in six states. The project is limited in scope and will not be able to address all forms of alignment. We will rely on expert judgments regarding the features that characterize test items, thereby addressing the first aspect of alignment discussed above. Because we will not have access to test score data, we will not be able to examine item char— acteristics or relationships among scores on dif— ferent tests and criterion measures (such as first—year grade point average). A comprehen« sive study of standard—setting across instru— ments is also beyond the scope of this project. PACE I 179 180 Importance of Considering Purpose of Assessment The degree of alignment among different sets of tests will undoubtedly vary substantially. Even when assessments are designed to be par- allel, as with alternate forms of the SAT, we would not expect perfect alignment. Because the assessments we are comparing in this study were designed for different purposes, the align— ment is likely to be much less than perfect. This is not necessarily a problem, if the differ- ences result from appropriate efforts to tailor the measure to the situation for which it was designed. For example, a low-stakes K—lZ sys- tem monitoring exam (i.e., one that is used to track achievement but that has no conse- quences for individual students, teachers, or schools), might be designed to include a broad variety of topics and therefore may not sample adequately from college—level material. There may be no discernible negative effect of this on students’ efforts to prepare for other exams. If, however, scores on this K—lZ exam were used to determine which students should graduate or which teachers should get bonuses in their pay— checks, there would be a significant risk of “teaching to the test” that might result in teachers and students neglecting material that is not tested. This type of response has been observed in states with test—based accountability systems." Thus the purposes of the tests, and how they are viewed by school personnel and students, influence the degree to which mis— alignment may pose a problem. The nature of the misalignment is also important. In the example presented above, the issue was primarily one of content sampling. The problem may be more serious when two tests reflect different philosophies concerning CRL'CIAI. IssL'Es IN CALIFORNIA EDL'CATION what students should know and what kinds of skills they should be able to display. In many cases, the misalignment among K-12 and uni- versity-level tests results from reforms that have taken hold at one level of the educational system but not another. This is particularly true in states where new tests have been developed to reflect state standards or frameworks that emphasize inquiry-based teaching and open— ended problem solving. In such cases, the skills and knowledge students are expected to demonstrate on the state exams may differ sub- stantially from what is expected on college admissions and placement exams. This creates a confusing set of signals for students concerning how they should prepare for the admissions and placement process. It is this signaling function of tests that is the primary motivation for this alignment study. Finally, the examinee population for which the test was designed, and the ways in which scores are used, must be considered. Exams that are intended to make fine distinctions among high—ability students need to include a large number of difficult items and may include topics that are covered in advanced courses. Such items would be less appropriate for a test that is administered to the entire public school population. So it would be reasonable to expect some misalignments. All of the results we dis— cuss below should be interpreted with this in mind. Later we provide further discussion of the importance of considering purpose. California’s Assessment Environment The current policy environment with respect to standards and assessments in California is 2000 Alignment Among Secondary and Post—Secondary Assessments in California described in the chapter of this volume by Venezia. Students in California high schools, par— ticularly those who plan to attend college, take a number of tests that differ in format and purpose. Below we discuss each of the assessments that we examined in this study. We study only mathemat— ics and English/language arts tests, though many of the assessment programs discussed below include tests in other subjects as well. Several of the tests we examined, including the SAT 1, SAT II, ACT, and AP exams, are used nationally to aid in college admissions decisions. The SAT I, a three—hour, mostly multiple—choice exam that measures general mathematical and verbal reasoning, is intended to help predict success in college. Evidence of its validity for this purpose typically focuses on correlations with freshman grade point average. The SAT II is a one—hour multiple—choice test that assesses in—depth knowledge of a particular subject, and is used by admissions officers as an additional measure with which to evaluate stu— dent subject—matter competence. The SAT II is used primarily at the more selective institutions and is taken by far fewer students than is the SAT I. For this study, we examined the follow— ing SAT 11 tests: Mathematics IC, Mathematics IIC, Literature, and Writing. The ACT is an approximately three—hour exam consisting entirely of multiple—choice items. Used as an alternative measure to the SAT I in evaluating applicants chances of success in college, it assesses achievement in several academic sub— jects, including science, reading, writing, and math. The AP tests are used to measure col— lege—level achievement in several subjects, and to award academic credit to students who demonstrate college—level proficiency. We examined two AP exams: Calculus AB and English Language and Composition. Students are encouraged to take the ACT or SAT 1 within their junior or senior years, where- as the most optimal time to take the SAT II or AP exams is within months of completing a rele— vant course. Students are typically required to take either the SAT I or ACT, and, at certain schools, several SAT II exams as part of the admissions process. While the AP tests are not a requirement, admissions officers are likely to View students with AP experience as better-pre— pared and more competitive applicants. In addition to the college entrance tests, California students encounter several other assessments during their high school years. As part of its Standardized Testing and Reporting (STAR) program, California currently requires public schools to administer the Stanford Achievement Test, Version 9 (Stanford 9) in grades 3 through 11, published by Harcourt Educational Nleasurement. Scores on this one— hour multiple—choice test are used to monitor student achievement in basic academic skills, and allow comparisons to be made to a nation— al sample of students. In spring of 1999 a set of augmentation items was administered to supplement the Stanford 9. These included 35 language arts items and 35 math items, which were designed to assess progress toward the state—adopted content standards. In grades 8— 10, the specific math items administered were determined by the math course in which the student was enrolled. The augmented portion of STAR is still evolving, and we were unable to obtain the actual items administered to stu- dents. Therefore these items are not included in our analysis. Results from the 1999 STAR PACF 181 182 administration indicate that the augmented items were difficult for students. The gover- nor has proposed tying merit—based college aid to performance on these items; this and other proposed high—stakes uses of STAR make it highly likely that both students and teachers will increasingly focus their efforts on this testing program. Students also have the option of taking the Golden State Exams (GSE), which are volun— tary tests allowing high schools students to earn special recognition when they graduate. The GSEs are 90—minutes tests con— taining both multiple— choice and open-ended items. They are intended to assess student achieve— ment relative to state— adopted content stan— dards in particular subject areas. We included five of these tests in our study: High School Nlathematics, First Year Algebra, Geometry, Reading/Literature, and “Tritten Composition. Some of the GSE assess— ments are similar to end— of—course exams (e.g., Algebra or Geometry), and are best taken while the students are currently enrolled in the course. CRUCIAL lssrias IN CALIFORNIA EDL'CA’TION ()ther GSEs are comprehensive tests that cover the content of several courses (e.g., Reading/Literature, Written Composition, and High School Mathematics). Students wishing to take these tests are advised to wait until their junior or senior year of high school. California State University Entry Level Mathematics Placement Exam Golden State Exam (Algebra) Golden State Exam (Geometry) released exam Sample items Sample items Sample items sections 75 minutes Two separate 45—minute sessions Two separate 45-minute sessions 6 Free response 65 MC 30 MC 2 OE 30 MC 2 OE Table 1. Structural Characteristics of the Tests: Mathematics 2000 Materials Number of Test Examined Time Limit Items Tools ACT Full sample form 60 minutes 60 MC Calculator AP Calculus AB Fuli form, 1997 Two 90~minute 40 MC Graphing calcu- ,_ Iator on last 15 MC items Calculator Calculator, Ruler . 1 Calculator, Ruler Alignment Among Secondary and Post-Secondary Assessments in California Finally, examinees applying to any of the 31 colleges under the California State University (CSU) and the University of California (UC) systems may be required to take a placement exams. These tests are used to determine whether admitted students possess entry-level math and English skills. CSU has placement tests for both math and English, whereas UC exam in math and/or English. Many of the administers a system—wide test only for community colleges also administer placement English. The CSU system requires its students Purpose Framework Content as Specified in Testing Materials Selection of students for higher education Provide opportunities for HS students to receive college credit and advanced course placement Assess whether admitted students possess entry level math skills Monitor student achievement toward state—approved con— tent standards, pro— vide special diploma Monitor student achievement toward state—approved con— tent standards, pro— vide special diploma High school mathe- matics curriculum AP Calculus Course Description Statement on Competencies in Mathematics Expected of Entering College Students reviewed by faculty from CA community Colleges, CSU, and UC systems Mathematics Content Standards for California Public Schools, Kindergarten Through Grade 12 adopted by the State Board of Education Standards Mathematics Content Standards for California Public Schools, Kindergarten Through Grade 12 adopted by the State Board of Education Standards Prealgebra (23%), ele— mentary algebra (17%), intermediate algebra (15%), coordinate geometry (15%), plane geometry (23%) and trigonometry (7%) Calculus Algebra l and II (60%), geometry (20%), data interpretation, counting, probability, and statistics (20%) First-year algebra Geometry obtain a minimum achievement level on the SAT 1, SAT II, or ACT in order to be exempted from taking a placement exam. UC requires a mini- mum achievement level on either the SAT II or AP exam. Students not meeting the minimum standards under the CSU guidelines must take a 75— minute multiple-choice math exam, and/or a IDS—minute English test, which contains both multiple-choice and essay items. Examinees not meet- ing the UC standards for English are required to take a two—hour essay exam. The community colleges admin— ister a range of exams; we include the Santa Barbara City College English exam in this analysis as an example. All students planning to enroll in an English course at the Santa Barbara City College must take the 85—minute College Tests for English Placement before registration. The test, consist— ing of both multiple—choice and essay items, is used to place students in an appropriate English course. Tables 1 and 2 list these testing programs and the type of information we were able to obtain for this study. For most tests, we used a single form from a recent administration or a full— length, published sample test. In a few instances where full—length forms were P.\(ZF, H 183 184 unavailable, we used published sets of sample items. This was the case for the CSU place- ment tests and the GSEs. As mentioned earlier, we were also unable to obtain the STAR aug- mentation items, but instead looked at the STAR Test Blueprints provided by the California Department of Education. For the En glish/ langua ge arts (ELA) tests, the table specifies whether the test includes each of three possible types of items: reading, objective (e.g., multiple—choice) writing, and essay writing. When interpreting results, the reader needs to keep in mind that the percent— ages we report for the CSU and GSE exams are not necessarily the same percentages that would be obtained ifwe had examined an actual test form. They do, however, provide rough indicators of the emphasis placed on various topics in the materials that are used by students to prepare for the exams. Methodology The alignment analysis involved two major phases. In phase 1, we developed a framework Materials Number of Test Examined Time Limit items Tools Golden State Exam Sample items Two separate 30 MC Calculator, Ruler (High School 45-minute 2 OE Mathematics) sessions SAT | Full sample form Two 30-minute 35 MC Calculator sessions 15 QC One 15—minute 10 GR session SAT II—Level IC Full sample form 60 minutes 50 MC Calculator SAT II—Levei llC Full sample form 60 minutes 50 MC Calculator Stanford 9 Full form 60 minutes 48 MC Calculator, Ruler Stanford 9 Test blueprints augmentation items Notes MC : multiple—choice OE = open—ended GR 2 grid—in QC 2 quantitative comparison Calculator, Ruler 1‘. Table 1 continued. Structural Characteristics of the Tests: CRUCIAL Issuas lN CALIFORNIA EDUCATION 2000 Mathematics Alignment Among Secondary and Post-Secondary Assessments in California of spec1fications for each subiect. We examined them to produce a set of specifications that several ex1st1ng assessment frameworks, such as addressed the range of topics and item types appearing on the tests included in this study. We then applied these frameworks to our set of those used to develop the National Assessment of Educational Progress (NAEP), and combined Purpose Framework Content as Specified in Testing Materials Monitor student achievement toward state-approved con— tent standards, pro— vide special diploma Selection of students for higher education Selection of students for higher education Selection of students for higher education Monitor student achievement toward CA standards Monitor student achievement toward CA standards Mathematics Content Standards for California Public Schools, Kindergarten Through Grade 12 adopted by the State Board of Education Standards High school mathe- matics curriculum Three—year college preparatory mathe— matics curriculum More than three years of college preparatory mathematics curricu- lum National Council of Teachers of Mathematics Standards CA standards Algebra I and II, geometry, probability and statistics Arithmetic (13%), algebra (35%), geometry, (26%), and other (26%) Algebra (30%), geometry (38%, specifically plane Euclidean (20%), coordi— nate (12%), and three— dimensional (6%)), trigonometry (8%), func— tions (12%), statistics and probability (6%), and miscellaneous (6%) Algebra (18%), geometry (20%, specifically coordi— nate (12%) and three— dimensional (8%)), trigonometry (20%), func- tions (24%), statistics and probability (6%), and miscellaneous (12%) Two subtests: mathemati- cal problem—solving and mathematical procedures 23% algebra l, 31% geometry, 31% algebra II, 14% statistics tests, and made several rounds of mod— ifications in response to difficulties we encountered in conducting the align- ment. The process was similar to one that we use for developing scoring rubrics for open-ended assessment items. The resulting frameworks are described later in this report. Phase 3 consisted of the actual alignment exercise. Two raters who had expertise in both the relevant subject area and in the application of scoring criteria to assessment results conducted the alignment analysis for each subject. The raters worked through several of the assessments together. “Then raters differed in their interpretations of the framework components, they discussed the dif— ference until agreement was reached. In cases where a disagreement could not be resolved, a third rater deter— mined the final categorization. This process resulted in reasonably high levels of agreement (kappa values of approximately 85 percent tolOO per— cent) for most categories. Two excep— tions were content area in math, where items often assessed skills in more than one area, and passage topic in reading, because passages often could be coded as addressing more than one topic. A final exception was the cognitive process category in math, discussed further below. For PACE 185 186 these categories, agreement tended to be approximately 70 percent. Results for Mathematics In this section we describe the results of the alignment exercise for math tests. First we pre— sent the framework that was developed. We then describe the major areas of alignment and mis— alignment, and discuss the implications of these findings for the signals that students receive. Framework The math framework consisted of three major dimensions: technical features, content, and cognitive processes. This set of dimensions was used in an earlier study of the alignment between state tests and NAEP,” but we modified the defi— nitions of these dimen— sions to some degree to reflect unique character— istics of some of the tests we examined in this study. The technical dimension covered fea- tures of the test that could be described through simple examina— CRUClAL ISSUES tion of the test and items—number of items, time limit, format (e.g., multiple-choice, essay), provisions for the use of tools such as calculators or protractors, the use of diagrams or other graphics, the use of formulas, and whether each item was embedded in a context (as in a word problem). The use of formulas AP Language and Composition California State University Entry Level English Placement Exam Golden State Exam (Reading/ Literature) Golden State Exam (Written Composition) (35 minute reading 75 MC objective section, 45 minute writing objective writing section) Sample questions 60 minute MC 52 MC section 3 essays 120 minute essay section Two 30—minute sections Sample items 45 MC reading 45 MC objective (one section each writing for reading and 1 essay objective writing) 45 minute essay section Sample items Two separate 45- 30 MC minute sessions 2 essays Sample Items Two separate 45— 30 MC minute sessions 2 essays Test Materials Examined Time Limit Number of Items Purpose ACT Full sample form 80 minutes 40 MC reading Selection of stu— dents for higher education Provide opportuni-«ii ties for HS stu- dents to receive college credit and l: advanced course i placement Assess whether admitted students possess entry level English skills Monitor student achievement toward state- approved content ‘3 standards, provide-Iii special diploma Monitor student achievement toward state— approved content standards provide special diploma Table 2. Structural Characteristics of the Tests: English/Language Arts IN CALIFORNIA EDUCATION 2000 Alignment Among Secondary and Post—Secondary Assessments in California was sometimes dlfficult to determine because rounding the assessment, particularly the problems can be solved 1n multiple ways, and in degree to which high stakes are attached to some cases an item could be solved either with performance. This is important because it or without a formula. Items were coded as affects examinee motivation. requiring a formula only if it was determined The content dimension included several cat— that the formula was necessary for solving the egories of math topics, from pre-algebra (e.g., problem. Finally, we examined the context sur— numbers and operations) through calculus. Almost all of the tests we examined Reading Objective Essay had specifications that included manv Framework Section? Writing Section? Section? ' or all of these categories. \Ve listed High school y Y N sub-categories as a means of making mathematics curriCU'um the distinctions among the main cate- gories clearer, but we coded using only the main categories. Finally, the cognitive dimension was AP English Language Y N Y identical to that used for NAEP, and and Composition . . Course Description 1ncluded three categories—conceptual understanding. procedural knowledge, and problem solving. As is typical with studies like this, the raters found this dimension to be the most difficult to CSU English curriculum Y Y Y . . code.H The cognltive process cate— gories cannot always be separated neat— ly: According to the NAEP framework, “These abilities are...descriptions of the ways in which information is struc— . tured for instruction and the ways in English-Language Arts Y N Y ' Content Standards for which students mampulate, reason California Public .- l1 . . 1 .. l . Schools, Kindergarten w it , or communicate tien mat iemati— Through Grade Twelve, adopted by the State Board of Education be no singular or unanimous agree— Standards cal ideas. As a consequence, there can ment among educators about what constitutes a conceptual, a procedural, English—Language Arts N Y Y Content Standards for California Public Schools, Kindergarten Through Grade Twelve, adopted by the State Board of Education Standards or a problem—solving item. “hat can be classified are the actions a student is likely to undertake in processing infor- mation and providing a satisfactory response.“ PACE u 137 188 In addition, items can often be solved in multiple ways, sometimes as a function of the examinees proficiency. What might be a prob- lem-solving item for one examinee might require another to apply extensive procedural knowledge. For instance, consider an item ask- ing students for the sum of the first 101 num- bers starting with zero. A procedural knowl— edge approach might involve a computation- intensive method, such as entering all the num— bers into a calculator to obtain the resulting Aspects of Alignment and Misaligwment in Mathematics To evaluate alignment, the degree of consisten— cy among the measures in connection with structural and content characteristics was stud- ied.‘ Table 3 provides more details on the struc- tural and content features of each test. The measures shared some features, particularly those related to format and administrative con— ditions. Every assessment included multiple- choice items, and all but the GSE were admin— sum. However, the problem-solving approach would entail a recognition that all the numbers, except the number 50, can be paired with another number to form a sum of 100 (100+0, 99+1, 98+2, etc.). The total sum is then simply computed by multiplying the number of pairs (i.e., 50) by 100 and adding 50. Clearly, depending upon the cho— sen approach, the same item can elicit varying levels of mathematical sophistication. The cogni- tive processes required by the items affect the con— struct that they measure and, as a consequence, examinee scores. However, for the purpos— es of this study, which focuses on signals sent to examinees, clear distinc— tions along this dimension are arguably less critical. CRt'ciAL lssrrs istered in a single testing session that took Materials Test Examined Time Limit Number of Items Purpose Santa Barbara Full sample form 85 minutes 35 MC reading Assess whether College Tests for English Placement SAT I SAT ll-Literature SAT ll—Writing Stanford—9 University of California Subject A Examination IN CALIFORNIA EDL'cATION Full sample form Full sample form Full sample form Full form Sample questions 2000 (30 minutes read— ing section, 35 minutes objective writing section, 20 minute essay) Two 30-minute sessions One 15—minute session 60 minutes One 40—minute MC session One 20—minute essay session 60 minutes 2 hours 70 MC objective writing 1 essay 78 MC 60 MC 60 MC 1 essay 84 MC (54 reading comprehension items, 30 vocabu- lary items) 1 essay students possess entry level English skills Selection of stu-' dents for higher education Selection of stu- dents for higher educafion Selection of stu-IK dents for higher ‘_ education ' Monitor student achievement toward CA standards Assess admitted .5; students’ writing ‘? skills Table 2 continued. Structural Characteristics of the Tests: English/Language Arts Alignment Among Secondary and Post-Secondary Assessments in California approximately one hour. Students were allowed the use of a calculator, although most questions did not require extensive computation. Familiarity with basic formulas and mathemati— cal identities was generally assumed as back— ground for the questions, but knowledge of more complex formulas was seldom necessary. The assessments, however, tended to have many more differences than similarities. There was a great deal of structural variation among the exams, especially with regard to the per- centages of items containing formulas and illus— trations. Fewer than 10 percent of the SAT I and Stanford 9 items required a memorized formula, in contrast to 25 percent of the GSE Geometry problems. Whereas the GSE Algebra and SAT II Level IIC assessments made little use of figures, the Stanford 9 and GSE Geometry exams included many illustra- tions, with 42 percent and 75 percent of their items, respectively, containing a diagram. Differences in the degree to which tests require interpretation of spatial or figural information are particularly important as they can affect gender and other group differences. Reading Objective Essay Instances of misalignment were . 9 . . . 9 . . Framework Section. Writing Section. Section? also observed Wlth respect to the amount of contextualization provided. High school Reading Y Y Y In spite of reform ideology that rec- and Language Arts 1 1 . 1 ‘. f . 11 Y Curriculum ommem 5 tie inc usion o persona } relevant items that require applica— tions of mathematical principles to real—life situations, many of the exams High 5°h°°' Reading Y Y N continued to measure student achieve— and Language Arts . . ‘ Curriculum ment With abstract questions—that 15, questions that included only numbers 2 7 ‘ I 2 2 i — High school English and Y N N 1nd 5} mbols. I\() more th in _ per American Literature cent of the items found on the college Curriculum . . admissions and placement assessments High school Reading N Y Y were contextualized (i.e., embedded in and Language Arts a story),whereas more than half of the Curriculum ‘ f . . fl Stanford 9 items were claSSified as being contextualized. Al'gned With NAEP Y N N Perhaps more important than the framework - . percent of contextualized items is the nature of the contextualization. In this respect, only the GSE open—ended UC English curriculum N N Y , . , , . questions were in line With the reform movement. Although 58 percent of the Stanford 9 items were framed in realistic situations, the presented sce- PACE I 18‘) l ()0 narios were brief, and had limited practical applica- tions. ()n the other hand, the GSE open—ended items allowed examinees to impose their own meanings and constraints, and bore some relevancy to “real—world” skills. The GSE open—ended items will be discussed more fully in a later section. W idespread reform efforts have also been direct- ed toward the format in which test items are pre— sented. Despite frequent criticisms that multiple— choice items are limited in both make use of an open-response format, the cognitive demands differ dramatically. An analogous problem arises with similarly named tests that assess very different sets of skills. Although all the exams are considered measures of mathematics achievement, there is a great deal of variation in the constructs assessed. Approximately 52 percent of the GSE Algebra items and 37 per- cent of the SAT I questions measured elementary the skills they measure, only the SAT I and GSE included items that Test Format Context Graphs Diagrams required students to gen— MC QC GR OE C 3 RO P 3 RC P ‘2 " wn r 3) ns s. 61 ":6 the“ O €81. O 6 ACT 100 O O 0 22 5 2 0 13 0 0 The GS]: open-ended CSU 100 0 0 0 24 0 0 0 16 0 0 questions, however, were GSE (Algebra) 95 0 0 5 15 0 5 0 10 0 0 much more extensive than GSE (Geometry) 95 O 0 5 10 O 0 5 75 0 O . i i ( GSE (HS Math) 92 0 0 8 33 O 5 0 23 0 5 the SAT 1 1t€mS~ SATI 58 25 17 0 25 7 o o 18 0 o SUCCCSSfill solution Of 21 SAT ll-Level IC 100 0 O 0 18 8 0 0 26 0 0 s w « SAT II—Level “C 100 O 0 O 12 12 2 O 2 0 (:SI: open—ended problem 0 _ . Stanford 9 100 0 o o 58 21 4 0 42 0 0 generally required multi— ple steps, and students were asked to justify or explain their solutions Legend: —frequently with diagrams or charts. In contrast, the SAT I open—ended items Format Context did not necessarily call for multiple strategies, and could sometimes be solved with algorithmic proce— dures. Furthermore, the SAT open—ended items were constrained, as the responses could not take on negative values. Thus, although the two tests MC : multiple—choice items QC : quantitative comparison items GR 2 fill—in—the—grid items OE : open—ended items Formulas M : formula needs to be memorized G : formula is provided C = contextualized items RO : graph/diagram within response options P = graph/diagram needs to be produced Content PA = prealgebra EA : elementary algebra IA 2 intermediate algebra CG 2 coordinate geometry PG 2 plane geometry TR : trigonometry SP 2 statistics and probability MISC = miscellaneous topics (:RL'CIAL ISSL‘ES 1x CALIFORNIA FDL'cA'I‘lox 1000 Table 3. Percent of Items Falling in each Category: Mathematics Alignment Among Secondary and Post—Secondary Assessments in California algebra knowledge, whereas 40 percent of the Stanford 9 items focused on statistics. For col- lege admissions exams such as the SAT 11 Level HC and ACT, relatively greater empha— sis was given to trigonometry, a topic that was absent from the both the GSE Algebra and SAT I exams. The misalignments among the measures go beyond content sampling, and extend to the reasoning requirements elicited by each test. Although none of the assessments focused heavily on problem-solving items, there were some differences with respect to the emphasis given to domain knowledge. Ninety—eight per— cent of the CSU items entailed straightforward application of declarative and procedural knowledge. In a similar vein, the vast majority of questions on the ACT, Stanford 9, and SAT II Level IC tests were also solvable via heuristics and algorithms. The SAT II Level HC, which was intended for S = graph/diagram within item-stem Cognitive Requirements CU : conceptual understanding PK = procedural knowledge P8 = problem—solving Cognitive Formulas Content Requirements examinees enrolled in more advanced M G PA EA 'A CG PG TR SP M'SC CU PK PS college preparatory math courses, 15 0 17 22 5 15 25 8 3 5 40 53 7 placed the most emphaSIS on problem- 18 0 6 32 8 16 14 2 22 o 28 70 2 solving ability (20 percent of its ques— 10 0 0 52 0 19 14 0 10 5 19 76 5 thDS). 25 0 0 0 O 5 86 10 O 0 52 38 10 P l h . f h . ‘ 15 0 23 15 0 23 23 0 15 0 62 23 15 er iaps t e soutce o t e inconsis— 1 8 13 37 2 5 1g 0 13 11 32 53 15 tenc1es can be traced to variations in the 12 0 2 30 10 12 28 4 8 6 34 58 8 purposes of the assessments and in the 10 O 2 14 22 12 14 18 6 12 26 54 2O fram V' k th‘lt 0. .1 ltl . Cl . l 6 6 0 13 2 19 19 4 40 4 63 31 6 e \01‘ s , buit et ieir exe op ment. The GSE and CSU were designed to be aligned with state—adopt— ed content standards, which have clearly prescribed guidelines that shape the content of the assessments. The Stanford 9 also employs an external Graphs/Diagrams framework, the Nation/II Comm?! 011 Teachers afl [zit/.m/mfirx Sf/IIId/I/‘a'x, but this set of guidelines encompasses stan— dards that cut across state lines. Because they do not follow any explicit frame— work, the college admissions exams that assess knowledge in particular subjects (i.e., SAT l1 and ACT) have more loosely defined standards, and draw upon COI‘E COI‘lCCptS taught within IllOSt mathematics courses. The SAT 1, on the PACE I 191 192 other hand, is independent of any specific cur- riculum or course, and is intended to assess general mathematical reasoning proficiency developed over years of schooling. Several of the misalignments discussed earli- er should probably not be considered problem- atic, as some of the differences emerge from appropriate efforts to adapt a test to serve a particular purpose. For instance, although both the SAT Level IIC and Stanford 9 included topics from a wide variety of courses, the SAT Level IIC drew upon trigonometry, whereas the Stanford 9 rarely included such material. The broad content sampling found on both of these assessments can be further contrasted with the topics on the GSE Geometry test, which reflected the curriculum of a specific course. In this particular case, the Stanford 9, SAT Level IIC, and GSE Geometry exam have disparate purposes, which call for differing lev— els of mathematical sophistication and varying extent of domain sampling. They are also tar— geted toward somewhat different examinee populations. Because the SAT Level IIC is typi— cally used to select among higher—achieving students for entrance into universities and col— leges, the test needs to include many complex problems with advanced content in order to distinguish among the examinees and rank order them consistently. The Stanford 9, on the other hand, is used to monitor K—IZ student achievement, and therefore require items of more moderate difficulty that can be attempted by students with a wider range of proficiency levels and course—taking histories. In a similar vein, the GSE Geometry test, unlike the SAT Level IIC or the Stanford 9, is not a measure of general math ability, but a measure of achieve— ment in a particular course. Consequently, it is CRL'CIAL IssL'Iis IN CALIFORNIA EDUCATION more appropriate for this assessment to limit its content to a narrow area of math than to sam- ple extensively from the entire mathematics domain. Thus, when making decisions con- cerning whether misalignments pose a potential problem, it is important to consider the use of the test. For the measures discussed above, the discrepancies most likely arise from variations in their purposes, and are therefore acceptable instances of misalignment. However, discrepancies among exams with similar purposes are also evident. Consider the SAT II Level IC and GSE High School Math exams.” Although both are intended to assess the proficiency of students who have taken three years of college preparatory math cours- es, they differ in their structural and cognitive features. The GSE contained a higher propor- tion of contextualized items (33 percent com— pared to 18 percent), whereas the SAT II included more graphs (8 percent compared to none). The iSE High School Math test also placed a greater emphasis on problem-solving items. Finally, there were vast differences in content sampling; the GSE was more likely to draw upon pre—algebra (23 percent compared to 2 percent), whereas the SAT II included more elementary algebra items (30 percent compared to 15 percent). In this particular case, the inconsistencies among the two sets of test— ing materials may send mixed messages to stu— dents regarding the emphases placed on various topics and skills. Implications of the Misalignments The nusalignments among the exam materials can create a confusing set of signals pertaining to how students should prepare for the assess- ments. For example, the ACT and SAT I are 2000 Alignment Among Secondary and Post-Secondary Assessments in California often used interchangeably for college admis— sions, yet require students to demonstrate sub— stantially different skills and knowledge. The ACT requires examinees to memorize formulas and identities, and includes numerous text— book—like problems that can be solved Via sim- ple application of procedural and declarative knowledge. In contrast, the SAT I provides stu— dents with formulas and mathematical identi— ties, and places relatively more emphasis on abstract reasoning. Ostensibly, students prepar- ing for the SAT I should not spend their time memorizing formulas, and should instead focus their efforts on furthering their inferential rea— soning skills. On the other hand, examinees studying for the ACT might attempt to review formulas or practice algorithmic problems. Particularly for high-stakes exams, it is crucial that students prepare in appropriate ways, as differences in preparation efforts can greatly influence performance. Inconsistencies among the exams are not the only potential source of confusion, as discrepan— cies between a framework and a test can send contradictory messages. The il/Iothewmtics Content Stmzdordxfor California Public Schools, Kindergmten Through Grade 12, the National Council on Teachers of Mathematics (NCTM) Standards,13 and the Statement on C‘o711petc12cic.s‘ in iWclthematics Expected ofEnteting Col/ego Students indicate that desired outcomes of math— ematics instruction include an increase in math— ematical reasoning and communication, as well as a greater appreciation for the role that math— ematics plays in everyday life. Of the three exams that resulted from these frameworks (the GSEs, Stanford 9 and CSU, respectively), the content of two of the tests did not appear to address these particular outcomes. Fewer than 10 percent of the items on the Stanford 9 and the CSU assessed problem-solving ability, and none required students to communicate mathe- matically. Furthermore, the plethora of abstract questions on the CSU exam, and the limited practical applications of the Stanford 9 contex- tualized items, may suggest to students that mathematics is not useful or relevant to real— world problems. The multiple—choice format favored by the Stanford 9 and CSU can also send negative messages regarding the impor- tance of reasoning skills. Although items in any format can be designed to measure a variety of abilities, multiple—choice items are popularly believed to be less adequate than free-response questions at measuring higher-order thinking. Additionally, multiple—choice items are solution— oriented, as students who select the correct option receive full credit, regardless of the logic or reasoning underlying the given response. The signals stemming from the Stanford 9 or CSU can be contrasted with those from the GSE. The GSE open—ended items were well— contextualized and process-oriented. The latter factor was clearly evident in the scoring rubric, which awarded different scores to two students who had the same set of calculations but who varied in their justifications of their work. In essence, scores were strongly affected by the degree to which students communicated their responses. However, the GSE test instructions were vague as to how elaborate the students’ explanations should be, and in some instances the failure to receive the maximum number of points might have stemmed from a mismatch between the item stem and the scoring guide— lines. For instance, one item presented students with data relating the amount of compression with the height of a ball shot upwards, and PACE I 103 194 asked students to “make a graph of this infor- mation.” Students choosing a bar graph received only partial credit because the bar graph was not the most appropriate manner in which to represent the data. Perhaps if the instructions were more specific in their require- ments and prompted students to consider the most suitable manner of data representation (as opposed to any mode of representation), these students might have chosen a different type of graph. Especially for free-response items, the standards that will be used for judging respons- es must be clearly and adequately conveyed to examinees. Results for English/Language Arts (ELA) In this section we present the results of our analysis of alignment among tests used to assess students’ skills in reading and writing. The tests’ names were varied, but they all focused on reading and/or writing in the English lan— guage. Table 2, discussed briefly above, lists the tests along with basic details. Framework The ELA framework covers three types of items: reading, objective writing (mainly multi— ple—choice items), and essay writing. Many of the tests we examined included two or all three of these item types, whereas others focused on a single type. In contrast to mathematics, there were no clear content areas that could be used to categorize items. Instead, the ELA analysis focuses more on structural characteristics and cognitive demands. In addition, many of the tests include short passages followed by sets of CRUCIAL ISSUES IN CALIFORNIA EDUCATION items, so it was necessary to categorize both the passage and the individual item. There was extensive overlap among the frameworks for reading, objective writing, and essay writing. As with math, we identified sub- categories to sharpen the distinctions among the main categories, but we coded using only the main categories. The structural dimensions, described in further detail in Table 4a, included three categories. The topic category captured the subject matter of the passage, and consisted of five areas—fiction, humanities, natural sci— ence, social science, and personal accounts. The type category identified the author’s writing style as narrative, descriptive, persuasive, or informative. The stimulus category referred to the presentation of the passage, such as a letter, essay, poem, or stOIy. Raters used all three cate- gories when coding the reading and objective writing items, but used only the topic category when coding the essay writing questions. The cognitive framework for both the read— ing and objective writing measures consisted of a single cognitive demand dimension. Raters coded questions as assessing ability to recall information, make inferences, or evaluate an item’s style. In reading, questions that could be answered via direct reference to the passage were coded as recall items, whereas questions that required the examinees to interpret the material were coded as inference items. Questions that pertained to the development of ideas or improved upon the presentation of the reading passages were coded as evaluating style. For the objective writing measures, items that entailed application of grammatical rules were considered recall items. Typically, most of these questions concerned mechanics or usage errors. ZOOO Alignment Among Secondary and Post-Secondary Assessments in California Description or Example Type of Writing Natural sciences Social sciences Personal Stimulus materials Letters Essays Poems Stories Attempt to influence others to take some action or to influence someone’s attitudes or ideas Share knowledge: convey messages, provide information on a topic, instructions for Narrative Stories, personal essays, personal anecdotes Descriptive Describes person, place, or thing Persuasive Informative performing a task Topic Fiction story, poem Humanities e.g,, artwork of Vincent Van Gogh e.g., the reproductive process of fish e.g., one man, one vote; cost effectiveness of heart transplants e.g., diary account of death of a parent Table 4a. Description of the ELA Structural Dimension Coding Scheme Inference items were those that required exami— nees to identify cause—and—effect relationships, and “evaluating style” items asked students to display rhetorical skills including an understand— ing of sentence organization, clarity, and other stylistic features of written work. Table 4b gives more details of the cognitive coding systems. The above framework was not applicable to the essay writing items, since all of the essay tests prompted students to establish and sup— port a thesis. Students could use recalled knowledge as well as make inferences, and were asked to construct a clear presentation (see Table 4b). For the essay writing questions, raters focused on the scoring criteria, which highlight the emphasis given to mechanics, word choice, organization, style, and insight. Aspects of Alignment mid ZWisalignment in Elzglisb/Lm'IgWage Arts \Ve analyzed the degree of alignment among the different assessments by comparing the structural and content dimensions for each pas— sage and each item. All of the ELA exams with reading sections used a passage as an item prompt, and virtually all of the studied tests included a set of multiple-choice items (the UC placement test was the exception). Perhaps indicative of the loosely defined nature of the subject matter, differences among PACF I 195 the exams were much more prevalent than in math. Some assessments did not involve a writ- ten composition (ACT, SAT 1, SAT II Literature, and Stanford 9) whereas others required two or three essays (AP, GSE Reading/Literature, and GSE Written Composition). There were also vast differences in the amount of time students were permitted to write their essays; the UC system allotted two hours for a single essay, whereas the SAT II W’riting exam and the Santa Barbara City College exam each allowed only 20 minutes for essay completion. The differences were not limited to the administrative characteristics of each exam, but were also apparent with respect to the structur— al features. In reading, all of the passages on the SAT II Literature test were narrative, and 63 percent were on fictional literary topics (see Table 5a). In contrast, the SAT I passages tend- ed to be informative (60 percent), and were much more likely to draw from humanities (40 percent). The essay was the most predominant presentation mode, with all of the passages on the AP, CSU, GSE Reading/Literature, and Santa Barbara City College exams presented in this manner. The majority of the passages on the SAT I and ACT were also essays (80 per- cent and 75 percent, respectively), but the SAT II Literature and the Stanford 9 varied the stimuli in which the reading passages were pre- sented. The Stanford 9 included a letter and a flyer, whereas the SAT II Literature test was the only reading exam that included poems as a stimulus. These formats were not found on the other reading exams. On measures of objective writing, the ACT, CSU, GSE Reading/Literature, SAT II Description or Example Used for Used for Used for Reading Object Writing Essay Writing Cognitive Demands Recall Answer can be found directly in the text, or by using X X the definitions of words or literary devices, or by applying grammatical rules lnfer lnterpret what is already written X X Evaluate style Improve the way the material is written X X Scoring Criteria X Mechanics Grammar, punctuation, capitalization X Word choice Use of language, vocabulary, sentence structure X Organization Logical presentation, development of ideas, use of X appropriate supporting examples Style Voice, attention to audience X Insight Analytic proficiency, accurate understanding of stimu— X lus passage Table 4b. Description of the ELA Cognitive Dimension Coding Scheme 196 CRUCIAL ISSUES IN CALIFORNIA EDL‘CATon ZOOO Alignment Among Secondary and Post-Secondary Assessments in California Writing, and Santa Barbara City College assessments included passages as item prompts, whereas the SAT I did not (see Table 5b). Vlrtually all the passages were presented as essays, although the Santa Barbara City College exam did include stories as a stimulus. There was some variation in the types of passages, as the GSE Written Composition passages were narrative, whereas the CSU passages were informative. Passages on the ACT, SAT II Writing, and Santa Barbara City College exams were approximately equally divided between narrative and informative. In a similar manner, the topics of the objective writing passages var— ied greatly from one test to the next; the ACT and SAT II Writing items tended to include themes from humanities (60 percent and 100 percent, respectively) whereas the CSU test focused on issues in social science. In contrast, the GSE Written Composition included per— sonal accounts. For the extended essay writing assessments, all the measures but the CSU and Santa Barbara City College exams included a topic from humanities (see Table 5c). Personal accounts were also commonly chosen prompts, found on such assessments as the AP, GSE Written Composition, GSE Reading/ Literature, UC Subject A, and Santa Barbara City College tests. ()f the forms that we stud— ied, only the GSE Written Composition and CSU exams selected a social science theme, and only the UC Subject A test included a topic from natural science. None of the prompts drew from fictional material. Inconsistencies among the exams were par— ticularly evident with respect to the cognitive demands of each test. Of the reading assess— ments, only the AP test required students to analyze a literary excerpt via a written composi— tion. The remaining exams assessed knowledge and understanding of a passage solely with mul- tiple—choice items. The cognitive complexity of the multiple—choice questions varied greatly among each of the measures. In reading, for instance, the SAT I and SAT II Literature tests placed great emphasis on analytical ability, with 83 percent and 80 percent of their items, respectively, assessing inferential skills (see Table 5d). Tests such as the AP and CSU also emphasized inferential skills, although not as heavily as the SAT I or SAT II Literature exams (77 percent and 66 percent, respectively). In contrast, measures such as the ACT, Stanford 9, and GSE Reading/Literature focused on straightforward recollection of information (58 percent, 71 percent, and 86 percent of their questions, respectively). There was also great variation in cognitive complexity on the objective writing assessments (see Table Se). Of the six measures, only the SAT I and the Santa Barbara City College included a significant proportion of items assessing inferential skills (100 percent and 57 percent of their items, respectively). Such ques— tions comprised less than 5 percent of the items on the ACT and SAT II \Vriting exams, and were completely absent from the GSE \Vritten Composition test. The CSU focused on evalu— ating writing style (64 percent), whereas GSE emphasized recall items (67 percent). Tests such as the ACT and SAT II \Vriting exams were more balanced in the kinds of skills they assessed; the items on these tests were mainly divided among recollection of information and evaluation of style. There was much more consistency with respect to the kinds of cognitive demands PACE 1‘)? Test TYPe TOPiC Narrative Descriptive Persuasive Informative Fiction Humanities Natural Social Personal Science Science ACT 50 O 0 50 25 25 25 25 0 AP 75 O O 25 O 25 25 O 50 CSU 100 O O 0 0 100 0 0 o GSE Reading/Literature 100 O 0 O O O O O 100 Santa Barbara City College 0 0 O 100 O 43 43 14 0 SAT I 40 O O 60 2O 4O 20 20 0 SAT II Literature 100 0 O 0 63 0 0 13 25 Stanford 9 50 O 17 33 17 33 33 O 17 Table 5a. Percent of Reading Passages Falling into Each Category Test Type Topic Narrative Descriptive Persuasive Informative Fiction Humanities Natural Social Personal Science Science ACT 40 O O 60 0 60 20 O 20 CSU O O O 100 O O O 100 0 GSE Written Composition 100 0 O 0 0 0 O 0 100 Santa Barbara City College 50 O O 50 O 100 0 O 0 SAT I 0 0 O 0 O 0 0 0 0 SAT ll Writing 50 O O 50 0 100 O 0 O Table 5b. Percent of Objective Writing Passages Falling into Each Category Test Fiction Humanities NaturE’lpSlfzience Social Science Personal Essay AP X X CSU GSE Reading/Literature X X GSE Written Composition X X SAT ll Writing X Santa Barbara City College X Table 5c. Topic Contents of Essay Writing Prompts 198 CRUCIAL ISSUES IN CALIFORNIA 1C 1) L'(1.-\'l‘l 0 N 2000 Alignment Among Secondary and Post-Secondary Assessments in California Stimulus Letter Essay Poem Story 0 75 0 25 0 100 O 0 0 100 0 0 O 100 0 0 0 100 0 0 0 80 O 20 13 25 50 13 17 33 0 50 Stimulus Letter Essay Poem Story 0 100 0 0 0 100 O 0 0 100 0 0 0 50 O 50 0 0 O 0 0 1 00 O O required by measures of writing ability (see Table 5 f) Skills such as mechanics, word choice, style, organization, and insight were identified as important factors in virtually all of the tests we studied. However, the GSE Reading/Literature test downplayed the importance of mechanics, word choice, and style, and the SAT ll Writing test did not identify did insight as part of its scoring criteria. The implications of these omis— sions will be discussed later. As was the case with math, two verbal tests may have the same construct label, yet make vastly different cognitive demands. The USP. Reading/Literature, AP Literature and Composition, and SAT II Literature test are all measures of reading proficiency, but differ in the kinds of skills assessed. The GSE Reading/Literature items typically entailed rec- ollection of facts directly from a given passage, and usually did not ask students to judge the mood or tone of the piece. Both the AP and SAT 11 Literature assessments, on the other hand, required deeper analysis of the reading passage, oftentimes asking students to deter- mine the effect of a given line or infer the intentions of the author. The AP exam, in par— ticular, required students to apply their knowl- edge of literary devices. The AP test included many items asking students to identify exam— ples of hyperboles, alliterations, and the like, but such questions were not found on either the GSE Reading/Literature or the SAT 11 Literature exams. Discrepancies between the curricular stan— dards and the tests were also apparent. For instance, the ability to learn the meaning of a word from context is perceived to be an integral aspect of English, yet most of the tests did not address this skill. Instead, many of the vocabu— lary items assessed students’ recall ability rather than their inferential skills. The ACT, AP, GSE Readirig/Literature, SAT 11 Literature, and Stanford 9 assessments typically framed a vocabulary item as follows: “In lines XX, the word ‘panacea’ is best understood to mean...”. Although the question is phrased to indicate that the meaning relies on context, it can be construed as a recall question, as (I prion knowl— edge of the definition is sufficient for a correct answer, since the context oflines XX did not affect the standard definition of “panacea.””‘ Two tests that did ask examinees to derive meaning from context were the CSU and the SAT I. The CSU contained a section in which P A (I F. E 19‘) Test Recall Infer Evaluate Style ACT 58 42 3 AP 23 77 0 CSU 33 66 0 GSE Reading/Literature 86 14 0 Santa Barbara City College 54 46 0 SAT l 18 83 0 SAT II Literature 13 80 7 Stanford 9 71 29 O Table 5d. Percent of Reading Items Falling into Each Category Test Recall Infer Evaluate Style ACT 48 4 48 CSU 14 21 64 GSE Written Composition 67 O 33 Santa Barbara City College 16 57 27 SAT I O 100 0 SAT II Writing 50 3 47 Table 5e. Percent of Objective Writing Items Falling into Each Category Scoring Criteria Factors Test Mechanics Word Choice Organization Style Insight AP X X X X X CSU X X X X X GSE Reading/Literature X X GSE Written Composition X X X X X SAT II Writing X X X X UC Subject A X X X X X Table 5f. Factors Identified in the Scoring Criteria of Each Test 200 CRUCIAL ISSUES IN CALIFORNIA EDUCATION 2000 Alignment Among Secondary and Post-Secondary Assessments in California a nonsense word was used in a sentence, and students were asked to decipher the meaning of the nonsense word. Unlike the other vocabu— lary tasks described earlier, students must infer the meaning of the word based on how it is used, and cannot rely on prior knowledge to answer the item. Similarly, the SAT I contained questions assessing analytical and inference ability. The SAT I included an analogy section that required students to analyze the relationships between a pair of words, and choose another pair of words whose relationship was most sim- ilar to the original pair. The SAT I also con— tained an additional section in which a sentence with omitted words was presented. Examinees were then asked to choose which set of words, when inserted into the sentence, would make the sentence most meaningful. A unique feature of some of the SAT I items was that they addressed not only the primary meaning of a word, but the secondary and tertiary meanings as well. Implications of the Misalignments As with math, the misalignments among the ELA assessments can send confusing messages. There appeared to be little consistency among the exams, thereby rendering it difficult to counsel students on the best preparation meth— ods. Measures that include only multiple— choice items would be approached in vastly dif— ferent ways than exams that require a sample of the examinees’ writing proficiency. Moreover, even when two tests require a written composi— tion, the variations in the administrative condi— tions and scoring criteria call for different kinds of strategies. For instance, teachers sometimes instruct students to organize their thoughts with a detailed outline. This technique may be appropriate for a two—hour UC essay, but it is less feasible for a 20—minute SAT II Writing task. Again, it is important to acknowledge that some of these inconsistencies may be more problematic than others, given the diverse pur— poses and examinees populations of these test- ing programs. The inconsistency in the scoring rubrics, particularly the omission of mechanics, word choice, and style from the GSE Reading/ Literature scoring rubrics and of insight from the SAT II Writing scoring guidelines, give rise to several concerns. First, these skills are part of the scoring criteria in most English courses and for the other assessments we examined. This means that the GSE Reading/Literature and SAT II Writing standards are incongruent with those that are typically expressed. Additionally, it is highly unlikely that the raters would be unconcerned with these factors when scoring the test, as mechanics, word choice, style, and insight are inherently part of what constitutes good writing ability. If raters are indeed including these skills as part of the scor- ing criteria, then students have been misin- formed about the standards on which they are judged. In light of the kinds of signals the scor- ing rubrics send, developers of the GSE Reading/Literature and SAT II Writing assess— ments may wish to reconsider the current guidelines, and be more explicit about their scoring criteria. Finally, there are concerns about the incon— sistencies among the scoring standards across different measures of writing ability. The requirements for a model essay under the GSE Written Composition or CSU guidelines are less rigorous than those found for the AP exam. PACE I 201 202 For the two former tests, maximum scores were awarded to sample essays that had diction errors, usage and mechanics lapses, and under— developed paragraphs. Under the AP guide- lines, such compositions might receive ade— quate scores, but would not be viewed as exem— plary; only essays that demonstrate exceptional rhetorical and stylistic techniques, with sub— stantial evidence to support a position, would receive a maximum score under the AP scoring rubrics. Because the GSE Written Composition, CSU, and AP exams are intended for different student ability levels and serve dif— ferent purposes, misalignments among their scoring criteria are inevitable. Nevertheless, such discrepancies may send mixed messages to students and school personnel regarding the standards of what is considered an excellent composition. Discussion In general, many of the studied tests were not well-aligned with respect to structure or con— tent. However, whether the inconsistencies are a source for concern needs to be interpreted in light of the purpose to which the assessments are intended. The misaligmnents may not pose a problem if they represent legitimate differ— ences stemming from diverse uses of the mea— sures. Indeed, different test purposes will neces— sitate different kinds of formats, administrative conditions, and item content. As was discussed earlier, variations in the content and difficulty level of the SAT Level IIC, Stanford 9, and GSE math tests should not be considered prob— lematic, as the exams have different test uses, and it is virtually impossible to create one test that can simultaneously serve those different purposes. However, when the IIIG‘JSUI‘CS SGI'VE CRL‘CIAL ISSUES IN CALIFORNIA EDL'CATION similar purposes and examinee populations, yet differ substantially in terms of content and cog- nitive demands (as appears to be the case for the GSE High School Math and SAT Level IC assessments, for example), there may be valid concerns regarding the misalignments. Regardless of whether or not the discrepan- cies are warranted, the inconsistencies can translate to a perceived testing overload by the examinees. Consider, for instance, the students applying for entrance to the University of California system. They are required to take the SAT I or ACT, SAT II, and possibly a place- ment exam. They are also encouraged to take the GSE and AP exams. The overabundance of exams students are required to take can foster a perception that the various measures are redun- dant. Although many of the tests have distinct uses and are therefore not interchangeable, it is likely that many students will not recognize the reasons underlying the need for multiple assess— ments, and may view the exams as unnecessary, time—consuming, and stressful. The nus-alignments can also send inconsis- tent signals with respect to preparation efforts. Although all of the testing preparation materi— als claimed that a challenging and rigorous aca— demic program was the best way to prepare for their exams, structural and content variations among the tests dictated differences in the most appropriate preparation strategies. It is likely that instructors confronted with preparing stu— dents for the entny—level CSU placement exam would most likely approach this task in a differ— ent manner than if they were to prepare their students for the more rigorous college—entrance assessments. Perhaps the most important sig— naling function of the tests relates to the mes— sages they send to students about what kinds of 2000 Alignment Among Secondary and Post-Secondary Assessments in California skills are valued. It has been shown that large- scale assessments, particularly those with direct consequences for students or teachers, often influence the kinds of skills and knowledge that are developed.H That is, both students and teachers are likely to focus their attention on the content that is tested. For this reason, there have been efforts from various educational reform movements and professional develop- ment organizations to increase the emphasis given to problem-solving items that are framed in real—world contexts. However, there remains a disassociation between the skills that are considered valuable and the skills that are actually assessed. In math, the majority of the items on the studied assessments involved heuristics using procedur— al or declarative knowledge. Moreover, as few items had meaningful applications to the real world, these tests do not convey the importance of math beyond the classroom or testing con— text. It appears that despite efforts to the con— trary, students may be receiving messages that mathematics is a sequence of algorithms to be memorized and applied, with little connection to real life problems. Similarly, on the ELA assessments students are not given clear signals as to which skills are valued. Arguably, the ability to make inferences or to evaluate the style of a given piece is as valuable as the ability to remember informa— tion, but this message is probably not adequate— ly conveyed by exams such as the Stanford 9 or the GSE Reading/Literature test. Such tests encourage students to direct their efforts toward recollection of facts and details, as opposed to deeper analysis of the given passage. NIoreover, the emphasis given to recall skills, particularly with respect to the assessment of vocabulary, can lead some students to learn the definitions of words through rote methods, such as memorization. Although this may lead to an increase in scores, it is not the ideal way of acquiring meaning, as nuances are not learned as adequately as if the word had been encountered in context. Perhaps the most problematic signal arises from the exams administered at the high school level, including the Stanford 9 and the exams required for college admissions, because the majority of these do not require examinees to demonstrate their writing skills. The SAT 11 Writing test, which does include an essay item, does not require multiple writing samples, nor does it allow an extended period of time for students to develop their ideas fully in a single essay. This may serve to communicate to stu- dents that writing is not an essential skill for college—level courses. In reality, however, most university—level classes require students to write extensively. Thus, the kinds of skills and knowl— edge valued in universities can differ substan— tially from students’ expectations. Again, if the measures are to send signals that writing ability is a desired skill, then the current tests need to be modified to reflect that message. Limitations of the Alignment Analysis and Recommendations for Future Work The use of expert judgments is a fairly common approach to studying alignment as well as con— tent validity.“ The evidence gathered through this study will be useful in evaluating the validi— ty of currently used tests for the purposes for which they were designed. However, this study PACE I 203 304 does not provide a complete picture of these assessments, and other analytic approaches might lead to somewhat different conclusions. Observations and interviews with students as they take the tests, an approach that is some— times used during the test development process, would undoubtedly result in somewhat different interpretations of a tests reasoning require- ments. Empirical data are also needed to quan— tify the consistency of student performance across various kinds of tests. It is important to evaluate the likelihood that students who per— form well on one kind of assessment will do so on another, as large discrepancies in perfor— mance can send confusing signals regarding the actual proficiency level of a student. Particularly for examinees attempting to prepare for a high— stakes measure, it is essential that they receive accurate and consistent information about their strengths and weaknesses. Finally, increasing the number of forms studied for each assessment would enhance the generality of our findings. The studied tests represent a sample of skills from a single testing occasion, and forms from other occasions will certainly vary somewhat. This is especially true when we analyze align— ment among ELA topics, where there is a limit— ed sample on any given test form (e.g., there may be only one essay). Studying multiple forms could increase the stability of our results. The study would also be improved if we increased the number of expert raters and refined our analysis of agreement levels among these raters. An ideal study would bring in a larger number of expert judges, selected to represent a range of experience in both the K— 13 and higher education sectors. It would also involve a more systematic analysis of differ- ences in coding, with perhaps some quantifica— (:RL'ClAL ISSL'ES IN CALIFORNIA l‘lDL'CA'l‘lON tion of commonalties and differences among tests. Because we looked at a large number of tests across several states," it was not feasible to conduct a more thorough study. However, as we argued earlier, alignment is a more critical consideration for some sets of tests than for others. Therefore there may be great benefit in conducting a more comprehensive align— ment study on the few tests for which align- ment really matters, allowing resources to be targeted rather than spread across a large num— ber of tests. In California, for example, it would be worth conducting a study in which the Stanford 9 test is compared with other measures of high school math and reading achievement, such as the SATII exams. Comparisons with the SATI are arguably less relevant. In any case, it is clear that students, parents, educators, and policymakers all could benefit from attention to the messages and sig- nals that tests are sending students. An additional problem stems from the lack of availability of full test forms for some of the testing programs. Inspection of actual forms would provide more accurate information about the distributions of items across our vari- ous categories. 011 the other hand, because this study is focused on the signaling function of tests, the use of publicly released materials rather than actual forms may actually be prefer- able. It is unlikely that students remember many details of the items they took on a single testing day. In contrast, the preparation materi— als, including sample items and sample test forms, probably have a greater influence on students preparation behaviors and their inter- pretation of what the test measures. Finally, many of the interpretations we make above depend on assumptions about stu— lOOO Alignment Among Secondary and Post-Secondary Assessments in California dents interpretatlons of the slgnals sent by heavily influenced by these tests than are oth— tests. It would be extremely valuable to inter— ers. For example, the group of students who View students, educators, and other school and engage in extensive SAT preparation activities college personnel to assess their views on is undoubtedly different from those who take these various testing programs and to find out the SAT with little prior preparation. Data how the tests influence their teaching and collected as part of the Stanford Bridge learning. It is also important to discover Project will provide useful information to sup- whether some groups of students are more plement this alignment study. Notes i We did not include the results for the AP Calculus AB exam because it was markedly different from the other studied tests. For example, it did not include material from any other mathematical content area except calculus. and was the only measure that necessitated a graphing calculator. Moreover, it was intended to assess the proficiencv level of a very select group of high-ability students. Given that the AP shared few commonalities with the other assessments, it was excluded from the following discussion. ii Again, for the GSE we did not examine an actual test form, but instead use the set of released items given to teachers and students. Thus the percentages discussed here do not represent percentages of items that examinees take, but instead indicate the relatlve emphases given to various topics on the materials that students use to prepare for the tests. iii As discussed earlier, whether an item assesses inferential skills or recall ability depends upon a students proficien— cv level. iv Although this report includes only California, we performed similar analyses for five additional states. 1. .llaking Smndmrlv .llnmv‘ I 997: .«In ."ilillllfll Fifty—Shire Report on Efforts to Raise Academic Smmlzmlx. (“Itshingtom DC.: American Federation of Teachers, 1997). 2. See, for example, AIAV. Kirst, Improving and Aligning [(—16 Smurf/nth, Adi/lirxionx, (1nd F/‘erlinmn Plate/nan POM‘fl’3‘. (Stanford, CA: National Center for Postsecondary Improvement, 1998); A. Venexia, “Connecting Californias R—ll and Higher Education Systems: Challenges and Opportunities,” this volume. 3. R. Edgerton, I fig/JCT Education IVb/‘n’ Paper: (The Pew Charitable Trusts: 1997). 4. M. Smith, AL and J. ()I)ay, “Systemic School Reform, in The Politics ofCurriculum and Testing.” cds. S. H. Fuhrman 8: B. Malen (Bristol, PA: The Falmer Press, 1991), 233—368. 5. Nmionnl Edna/[non Snnnnir Briefing Book (Achieve, Inc, 1999), available at http://\vww.achieve.org. 6. NL. \Vebb, Alignment ofSt/ena’ nnzl .l Indie/11min Stand/mix [Ind .{tress/nemit in Four anz‘cr. Research Monograph No. 18 (Aladison: National Institute for Science Education, 1999). 7. Not Good Enough: .4 Confem‘ .4llzlfi'.\‘i.\‘ (gf’li’nrln'l‘ I ire/lying [in]Innznfionr (Education Trust, Spring 1999). 8. L. Burstein, D. Koretz, R. Linn, B. Sugrue,]. Novak, E. Baker. and EL Harris, Describing Performance Standards: Validity of the 1993 National Assessment of Educational Progress Achievement level Descriptors as Characterizations ofMathematics Performance, lfr/nrnz‘ionnl .»I.\‘.\‘e\:r//n'11r 3 (1995/1996): 9—51; Setting Performance Standards for Student Achievement: A Report of the National Academy of Education Panel on the Evaluation of the Trial State Assessment: An Evaluation of the 1992 Achievement Levels (Stanford, CA: The National Acadenrv of Education, 1993). PACE I 205 206 9. HM. Stecher, and SI Barron, Quadrennial Milepost Accountability Testing in Kentucky, CSE Report 505 (Los Angeles: Center for Research on Evaluation, Standards, and Student Testing, 1999). 10. Kenny and Silver, 1999. 1. Ihid. ,_. 12. National Assessment Governing Board, A/Itif/M/mr/m Ii‘I'zmlvwor/cfor r/ac I 996 Nor/mm! {IA‘A‘l’J‘JV/lt’llI‘ ofEd/u‘orio/I/ll Progress: NAEP AlzIr/Jeflzoricx C'onxcnmv Project, GPO ()65-000—00772-0 (US. Department of Education, 1995 ), available at http://www.nagl).org). 13. Curriculum and Eva/(union Srondonhfbr School AI/IthI/{Irim‘ (Reston, VA: National Council of 'I‘eachers of Mathematics, 1989). 14. G.F. Madaus, “The Influence of Testing on the Curriculum, in Critical Issues in Curriculum,” ed. L. Tanner (Chicago: University of Chicago Press 1988), 83—121; T. Kellaghan, and Madaus, “National Testing: Lessons for America from Europe,” Educational Leadership 49 (1991): 87—93; Stecher and Barron, 1999. 15. S.G. Sireci, “Gathering and Analyzing Content Validity Data,” Edi/airfonol A.§‘.§’L’.\'.\‘7Ilt’lif, 5 (1998): 299-321. CRUCIAL ISSUES IN CALIFORNIA EDUCATIoN 2000 About the Authors Elizabeth Burr is Project Director for a child— care provider training and retention initiative at Policy Analysis for California Education (PACE). Her work focuses on early education policy research and analysis, with a focus on issues of access and equity. Recently she co— authored a Child Care Needs Assessment for Los Angeles County’s Department of Public Social Services. She holds a BA. degree from Brown University and an lVIPA degree from Columbia University Bruce Fuller is Associate Professor of Public Policy and Education at UC Berkeley. His cur— rent work focuses on family poverty and early education policy. As co—director of a Berkeley— Yale initiative called Growing Up in Poverty, Fuller is investigating how young children’s lives are being affected by welfare reform. Fuller also writes in the area of decentralization and education policy, including school choice and charter schools. Before joining the Berkeley faculty, Fuller taught at Harvard University. He has also worked on education and family policy issues at the World Bank and in the California legislature. He received his Ph.D. in Sociology and Education from Stanford University. Since 1969, ilez'r/Jrlel Kim? has been a Professor of Education and Business Administration at Stanford University. Before joining the Stanford faculty, Kirst held several positions with the federal government, including Staff Director of the U.S. Senate Subcommittee on lVlanpower, Employment and Poverty, and Director of Program Planning and Evaluation for the Bureau of Elementary and Secondary Education in the U.S. Office of Education (now the U.S. Department of Education). Kirst was a member of the California State Board of Education from 1975—1981 and its president from 1977—1981. He received his BA. in Economics from Dartmouth College, his NLRA. in Government and Economics from Harvard University, and his Ph.D. in Political Economy and Government from Harvard. As director of PACE’s Sacramento office, Gem/11' Hayward works primarily on issues in higher education, class size reduction, account— ability and school finance. Until recently, Hayward served as Deputy Director of the National Center for Research in Vocational Education at UC Berkeley. He is also a found- ing partner of Alanagement, Analysis and Planning, an educational consulting firm. From PAC 1“, I 307 208 1980—1985 Hayward served as Chancellor of the California Community Colleges, and prior to that served for a decade as Principal Consultant to the California State Senate Committees on Education and Finance. Hayward is a former teacher and administrator in California’s public schools. He received his BA. in Political Science from UC Berkeley and a master’s degree in Education Administration from San Francisco State University. Richard S. Brown obtained his Ph.D. in Social Research Methodology with a focus on advanced quantitative data analysis and mea— surement from the University of California, Los Angeles. His earlier research at the National Center for Research on Evaluation, Standards, and Student Testing (CRESST) involved developing assessments of teamwork skills in collaborative group settings using net- worked computers and investigations into the differential effects of question formats on meta- cognitive processes. His current research includes a project to develop an indicator sys— tem for California schools and an investigation into various methodological effects involved in setting performance standards on complex per— formance tasks using multi—level modeling, latent class analysis, and other advanced quanti— tative techniques. The C enter for the Furmr of Warning and Learning is comprised of education profession— als, scholars, and public policy experts who care deeply about improving education for California’s children. The Center was founded in 1995 as a public, nonprofit organization with the purpose of strengthening the capacity of California’s teachers for delivering rigorous, well-rounded curriculum, and ensuring the continuing intellectual, ethical, and social development of children. Nc/Il D. Fin/eelrz‘cin is an education policy researcher, based in California. From 1997-99, he served as a senior program officer with the National Research Council’s Committee on Education Finance which wrote iM/I/cz'ng Afaney iM/Ifier: Financing Ania/it‘d} Schools. Prior to his work with the committee, Dr. Finkelstein was the assistant director of PACE and a research associate with the National Center for Research in Vocational Education, both located on the University of California, Berkeley cam— pus. He has conducted research on numerous education policy issues, including public school finance, school governance, school-to—work programs and early childhood education. He holds a Ph.D. in Education Policy from UC Berkeley. LII/SA. Hnw'm is a research associate at PACE. His research focuses on issues of decentraliza- tion related to school reform and school choice, as well as the impact of school finance inequities on school reform. He is a contribut— ing author to a PACE report titled School Choice: Abundant Hopes, Scare Evidence of Results (1999), and is also a contributing author to an upcoming book published by Harvard University Press, titled Inside Charter Schools: The Paradox of Radical Decentralization (Bruce Fuller, editor). Huerta received his mas— ter’s degree in Education, and is presently a doctoral student at UC Berkeley. Before returning to graduate school, he served as a public school teacher for six years. CRL‘CIAL IsSL'Es IN CALIFORNIA E1)L‘(Z;\'l‘l().\‘ zooo William S. Funny is an education consultant, engaged in policy research, political analysis, and strategic advocacy. Furry has held positions with The RAND Corporation, the California State Assembly, and the Governor’s Office. Pan‘ieia Ga’ndara is Professor of Education at University of California at Davis and Associate Director of the University of California Linguistic Minority Research Institute. Among her recent publications are Tbe Dimensions of Time and Me Cballenge ofSebool Reform (1999) SUNY Press, and Bilingual education programs: A (TOSS national penpeez‘iz’e, co—authored with Fred Genesee, forthcoming in the Journal of Social Issues. Lama 3. Hamilton is an Associate Behavioral Scientist at RAND where she conducts research on educational assessment and the effectiveness of educational programs. Her areas of interest include the validity of large— scale achievement tests, the effects of high— stakes testing and accountability systems, and the relationships between classroom practices and student achievement in math and science. She received a AIS. in Statistics and a Ph.D. in Educational Psychology, both from Stanford University. joan Herman is associate director of the UCLA Center for the Study of Evaluation and co— director of the National Center for Research on Evaluation, Standards, and Student Testing (CRESST). Her research has explored the effects of testing on schools and the design of information systems to support school planning and instructional improvement. Her recent work emphasizes the validity and utility of per— About the Authors formance assessment and the measurement of students’ opportunity to learn. She also has wide experience as an evaluator of school reform. Vi—Nbuan Le is an Associate Behavioral Scientist at RAND where she conducts research on educational assessment. She is interested in group differences in achievement and exploring alternatives to multiple—choice tests. She earned a MS in Statistics and a Ph.D. in Educational Psychology, both at Stanford University. Abby Robyn is an Associate Social Scientist at RAND. Her research focuses on the imple- mentation of educational innovations. Her recent work includes studies of the relationship between teacher instructional practices and stu— dent achievement in math and science, a national conference on large-scale student assessment, and an evaluation of a high school reform initiative to improve the performance of students at risk of dropping out. She received a master’s degree in English from UCLA. Russell IV Ru'uibeiger is Professor of Education at the University of California, Santa Barbara and Director of the University of California Linguistic Minority Research Institute. He received is Ph.D. from Stanford University in 1978. A faculty member at UCSB since 1987, Rumherger has published widely in several areas of education: education and work; the education of disadvantaged students, particular— ly school dropouts; and education policy. He is currently working on a book on school dropouts and a study on the effects of school segregation 011 student lethVCIIICI‘lt. 1’16 SCI'VBS P A C E I 309 210 on the editorial board of Teachers College Record, Economics of Education Review, and the American Education Research Journal. Andre/1 Venezin is researcher at the Stanford Institute for Higher Education Research and the director of the Bridge Project: Strengthening K—16 Transition Policies. Venezia’s work focuses on education policy research and analysis, particularly as related to the transition from K—12 to postsecondary education, with an emphasis on issues of CRL‘CHL Isst‘iis IN CALIFORNIA EDL'CA'I‘IOX access, equity, and policy coherence. She has worked for a variety of state, federal, and not- for—profit education organizations including the National Education Goals Panel, the Texas Higher Education Coordinating Board, and the American Institutes for Research. Venezia earned a Ph.D. in Public Policy from the Lyndon B. Johnson School of Public Affairs at the University of Texas at Austin and a master’s degree in Administration and Policy Analysis in Higher Education from Stanford University. 2000 U..C BE REK LEY LIBRARIES 1.0700062 Policy Analysis for California Educafion School of Education 3653 Tolman Hall Berkeley, CA 94720-1670 510-642-7223 URL: http://pace.berkeley.edu