Policy Paper No. PP 89-9-1 M, California Partnership Academies: 1987—88 Evaluation Report Charles Dayton Alan Weisberg David Stern "\M, 65.": September 1989 Directors James w. Guthrie University of California Berkeley Michael w. Kirst Stantord University Policy Paper No. PP 89-9-1 V California Partnership Academies: 1987—88 Evaluation Report Charles Dayton Alan Weisberg David Stern qup‘v ‘ 2~.;>!ilUTE OF GOVERNMENT“. STUDlES USRARY Se tember 1989 n- p OUT * 1- 799:}, UNEVELRSIW or CA“ " re a A Charles Dayton is a policy analyst with PACE. ”FOLK!“ Alan Weisberg is a policy analyst with PACE. David Stem is an associate professor of education at the University of California, Berkeley. This paper was sponsored and published by Policy Analysis for California Education, PACE. PACE is funded by the William and Flora Hewlett Foundation and directed jointly by James W. Guthrie and Michael W. Kirst. The analyses and conclusions in this paper are those of the authors and are not necessarily endorsed by the Hewlett Foundation. This report was funded with special grants from the Edna McConnell Clark Foundation and the William and Flora Hewlett Foundation. The authors wish to acknowledge the assistance in data collection and analyses of Estelle Barber, Laurie Harrison, Dana Hollister, Il- Woo Paik, and Susan Sather; Robert Rowe of the California State Department of Education; in the report’s design, review, and production of James Guthrie, John Evans, Julie Koppich, Michael Menchaca, Norma Needham, and Judy Snow of PACE; and all academy site managers, who provided assistance in gathering data and gave helpful feedback on the case studies. Additional copies of this paper, PP89—9—l, are available by sending $10 per copy, plus $1.50 postage/handling and applicable California sales tax, to: PACE School of Education University of California Berkeley, California 94720 CHECKS PAYABLE TO THE REGENTS OF THE UNIVERSITY OF CALIFORNIA Policy Paper No. PP89-9-I Policy Analysis for California Education (PACE) Berkeley, California September 1989 ii Executive Summary Context This report presents findings from the third annual evaluation of the Partnership Academy Programs in California. These are high school-based state funded programs, selected through a grants competition conducted by the state Department of Education. They are based on SB 605 passed during the 1987 legislative session. During the 1987—88 school year there were twelve academies operating in the state: 0 The two original Peninsula Academies, in the Sequoia Union High School District in Redwood City, begun in the fall of 1981 Eight replications begun in the fall of 1985, four of which are in the Bay Area, three in or near Sacramento, and one in Bakersfield Two replications begun in the fall of 1987, one in Santa Barbara and one in Inglewood An academy is a modified high school program with a number of specific elements: A student selection process designed to enroll students with academic potential, but whose past performance indicates they are in danger of dropping out of school A school-within-a-school administrative structure, such that academy students take three core academic subjects as a group in grades 10—11, with selected teachers A technical course in grades 10—12 designed to provide students with basic job skills in a promising labor-market field in their geographical area Support from local businesses, including curriculum development, guest speakers, field trips, mentors, and work experience positions Both high school and district support for the program, providing the necessary teacher preparation time, facilities, equipment, curriculum development, and counseling support Findings The evaluation entails two broad components—process and outcomes. The process evaluation addresses the quality of program implementation and the degree to which programs follow the academy model. Each site is rated in terms of twenty-seven elements which together comprise the full model. While implementation improved somewhat over the previous year, it continued to span a broad range among the eleven sites. In 1986—87 the ratings ranged from 7.5 to 23, with a mean of 18.0; in 1987—88 they ranged from 14 to 26.5, with a mean of 20.7. Converted to “numerical grades,” the mean implementation grade increased in 1987-88 from 72 percent to 81 percent. The outcomes evaluation addresses the degree to which student performance changes as a result of the academies. A comparison group design is used for this part of the evaluation, in which a group of nonacademy students similar to those in each academy are selected and tracked along with the academy students. Comparisons are made in terms of retention in school, attendance, credits earned, courses failed, and grade-point averages. A regression model is used to test for differences between program and comparison groups on these dimensions; this model corrects for differences in prior school performance, gender, race or ethnicity, and date of birth. This year’s results were combined with those of the past two years to provide the fullest possible picture of academy performance. Of 212 tests of differences between academy and comparison group students on their performance during these three years, 61 were statistically significant in favor of academy students and 11 in favor of comparison groups. These differences were spread about equally across four variables: attendance, credits earned, courses failed, and grade-point averages. The fifth dimension, retention in school, showed few statistically significant differences, largely due to the small numbers involved in any individual site and the limitations of the statistical test. Statewide, for the first cohort, the only one to complete the full three years of the program, the dropout rate among academy students across three years was 7.3 percent and among comparison group students 14.6 percent. The transfer rate among academy students was 25.6 percent, and among comparison students, 33.1 percent. These figures suggest the academics are having some success in reducing transiency between schools and substandal success in reducing drOpouts. In the eight academies that enrolled seniors in 1987—88, an estimated twenty-nine students were saved from dropping out. The net benefit to society of saving those twenty-nine dropouts is estimated to be more than one million dollars. These results were not spread evenly across sites or years, however. Three sites have strongly positive results. The remaining sites range from several where most differences favor academy students but generally fail to reach statistical significance, to others where there is little pattern to the differences. By year, the academies seem to exert a strong positive influence on participants during grade 10, the first program year. This effect weakens during the second year (grade 11), and disappears altogether in the third year (grade 12). iv The evaluation also revealed a correlation between the quality of implementation and student outcomes. The five sites with implementation ratings of 90 percent or higher have: ' Sixty-seven differences favoring academy students, of which thirty-one are statistically significant, and seventeen favoring the comparison students , of which two are significant 0 An average attrition difference of 9.4 percent favoring academy students Contrastingly, the remaining sites, which have lower implementation ratings, have: - Seventy—seven differences favoring academy students, of which just fifteen are statistically significant, and forty-two favoring the comparison students, of which seven are significant - An average attrition difference of 5.4 percent in favor of academy students This indicates that the first group of academies is having a strong positive effect on its participants, with few examples of negative effects. The second group, while having a net positive effect, is doing so less strongly, and with more frequent instances of negative effects. Thus while full implementation of the academy model does not guarantee success, it notably improves its chances, while failure to implement the model faithfully substantially weakens the likelihood of positive impact. Feedback from student questionnaires shows that most students in the academies like the academy equipment and materials they work with, see a clear connection between their academy studies and post— graduate plans, and like the academy better than their regular high school program. Relatively few students are developing career plans through the academies; most plan to attend some form of college upon graduation. A significant proportion of students report more positive feelings toward their class work after being in the academy. Recommendations 1. During the 1988—89 school year there were eighteen academics in operation, with five more in the planning stage. The governor has recommended that there be thirty-three academies in place by the 1990—91 school year. This evaluation provides evidence that such expansion is warranted. However, the quality with which new replications are implemented is critical to their success. 2. Accordingly, sufficient training and technical assistance should be provided to each new academy replication to ensure the best possible chance of success. 3. The legislation that provides a basis for the academics (SB 605) includes a performance-based funding formula. The terms of this bill should be consistently implemented to ensure that those academies performing well are continued and those that are not are encouraged to cease operations. 4. Although SB 605 calls for an evaluation of the academies, this one, conducted from 1985 to 1988, was entirely supported through private foundation support, and is completed with this report. It seems in the best interests of the state to support some form of evaluation of these programs, if it views them as a model for potentially wider use in California. vi Contents Executive Summary ...................................................................... iii Foreword .................................................................................. ix Policy Analysis for California Education ............................................ xi Chapter 1: Introduction .................................................................... 1 Chapter 2: Academy Case Studies ........................................... 7 The Health Careers Academy, Bakersfield ............................................ 9 The Johnson Corporate Academy, Sacramento ....................................... 21 The East Side Electronics Academies, San Jose ...................................... 33 The Peninsula Academies, Redwood City ............................................ 53 Computech Academy, Mountain View ................................................ 71 The Foothill Computer Academy, El Dorado Hills .................................. 85 The Health Academy, Oakland ......................................................... 97 The Rio Cazadero Academy, Elk Grove .............................................. 107 The Computer Academy, Santa Barbara .............................................. 119 Chapter 3: Cross-Site Analyses and Conclusions ................................ 129 Academy Implementation ............................................................... 129 Student Outcomes ........................................................................ l 3 6 Benefit Cost Analysis .................................................................... l4 8 Conclusions and Recommendations ................................................... 15 3 Appendix A: Multivariate Analysis of Academy Program Effects ................. 159 Appendix B: Academies Bill SB 605 .................................................. 163 vii R a {‘Ms‘}: we 3” i m. .L- , , v m w . J #1 ‘ V.» 'l ‘7; '4 ‘ 1% _.;.i« Foreword Education is a field often guided more by tradition and inertia than by innovation. When new educational programs are implemented, only infrequently are they subjected to rigorous, methodologically sound evaluations. This report, by Charles Dayton, Alan Weisberg, and David Stern, represents an important continuing effort to understand and assess the outcomes of a particular educational innovation. Dayton and his colleagues report findings from the third annual evaluation of Califomia’s Partnership Academies Program. Partnership Academies are designed to give disadvantaged high school students employment skills and prevent them from dropping out of school. The researchers employed a longitudinal, quasi-experimental design, using control and comparison groups, to assess the effectiveness of individual academy programs and then to conduct cross-site analyses. Their findings underscore the potential significance of the academy model, in reducing the number of young people who leave school prior to high school graduation. The careful design of this evaluation renders study results unusually credible. Continuing to build in collected data will better reform the deliberations of policymakers, educators, and researchers as they strive to develop ever more productive programs to meet emerging educational challenges. James W. Guthrie Professor of Education and Co—director, PACE University of California, Berkeley Julia E. Koppich Associate Director, PACE University of California, Berkeley i"? i; n; 1:. mm‘ V? r Policy Analysis for California Education Policy Analysis for California Education, PACE, is a university-based research center focusing on issues of state educational policy and practice. PACE is located in the Schools of Education at the University of California, Berkeley and Stanford University. It is funded by the William and Flora Hewlett Foundation and directed jointly by James W. Guthrie and Michael W. Kirst. PACE operates satellite centers in Sacramento and Southern California. These are directed by Gerald C. Hayward (Sacramento) and Allan R. Odden (University of Southern California). PACE efforts center on five tasks: (1) collecting and distributing objective information about the conditions of education in California, (2) analyzing state educational policy issues and the policy environment, (3) evaluating school reforms and state educational practices, (4) providing technical support to policy makers, and (5) facilitating discussion of educational issues. The PACE research agenda is developed in consultation with public officials and staff. In this way, PACE endeavors to address policy issues of immediate concern and to fill the short- term needs of decision makers for information and analysis. PACE publications include Policy Papers, which report research findings; the Policy Forum, which presents views of notable individuals; and Update, an annotated list of all PACE papers completed and in progress. Advisory Board Joseph Alibrandi A. Alan Post Chief Executive Officer California Legislative Analyst, The Whittaker Corporation Retired Mario Camara Sharon Schuster Partner President, Castle & Nicholson American Association of University Women Gerald C. Hayward Floraline Stevens Deputy Director Director National Center for Research Program Evaluation and Assessment in Vocational Education Los Angeles Unified School District Robert Maynard Eugene Webb Editor and President Professor, Graduate School of Business The Oakland Tribune Stanford University Aaron Wildavsky Professor of Political Science University of California Berkeley xi gain "" 21f}, ”34.1mm, It 3“”me "’r‘ 1t- ‘ ‘* 1111mwmm ;_r)“a 441"” $11k figgwm; 1117153 $1711 1%“ ‘S »‘~ M. 1'” m:- ,_ m. x : a fif’f’fi’ 9w 1: )1! 7w . . K «m; _ m m m 5.1m i u ‘9 \ -1 32m: _ W32 ' “ ~ ' ' 3. A S ,g . 1‘: a y (f f : IV ‘ “Hm _> A r 5 E . ) hr: H ‘H 3” .‘1-1Wfiafiiu a} .‘gg'mwm V i - , u A. Y H > > H . . w‘l'r‘ « ‘3‘ ‘i chapter 1 Introduction During the 1987—88 school year, twelve academy programs were operated in California under state sponsorship. Two of these are the Peninsula Academies operated since 1981 by the Sequoia Union High School District in Redwood City. The remaining ten are replications of these, now called Partnership Academies. The academies are directed at reducing dr0pouts among “at-risk” high school youth. They combine a modified high school curriculum and structure with exposure to local companies and jobs, with the intent of improving students’ motivation to perform well in school, graduate, and make the transition from school to work. The twelve academies received state funding from Assembly Bill 3104, passed during the 1984 legislative session. That bill also included a requirement to “evaluate the Peninsula Academies model program and report to the Legislature regarding issues which shall include, but not be limited to, the cost effectiveness and educational quality of the program.” This report and study address that requirement. It was performed by Policy Analysis for California Education (PACE), with joint support from the Edna McConnell Clark and William and Flora Hewlett Foundations. Academy History The two Peninsula Academies were begun in the fall of 1981 by the Sequoia Union High School District. They were designed from extensive research and modifications of model programs begun elsewhere. Their purpose was to reduce the high dropout rate among at— risk students in that district. Three years later, the performance of those two programs was found compelling enough for the California Department of Education to declare them a model program, and AB 3104 was passed to replicate them in ten other high schools around the state. What is an academy? It involves a modified high school program with a number of specific elements: 0 A student selection process designed to enroll students with potential, but whose past performance indicate they are in danger of dropping out - A school-within-a-school administrative structure, such that academy students take three core academic subjects as a group in grades 10—12, with selected teachers 2 PENINSULA ACADEMIES 1987—88 EVALUATION 0 Along with the academic classes, participants have a technical course in grades 10—12 designed to provide them with basic job skills in a promising labor market field in their geographical area ° Strong support from local businesses, including curriculum input, speakers, field trip sites, mentors, and work experience positions 0 Both high school and district support for the program, providing the necessary teacher coordination time, facilities, equipment, curriculum development, and counseling support 2 The academies address a significant problem. The high school attrition rate in California is approximately 30 percent. Among urban schools and minority youth this figure often surpasses 50 percent; 55 percent of the participants in academies are black or Hispanic. The loss to young people who drop out is enormous, both in terms of their self esteem and financial future. At the same time, due to the “baby bust” there is a declining number of graduates and an estimated 40 percent drop in the number of youth who will be in the work force by the year 2000. In many areas, a shortage of trained entry-level employees is occurring already. Thus, on the one hand there are youths leaving school with no diploma and limited job skills; on the other hand, the economy is suffering from a lack of well-prepared entry-level workers. Given today’s intensely competitive international marketplace, high school performance and students’ transition to work are issues important not just for California youth or local businesses, but for the nation as a whole. The academies represent three-way partnerships among the state, local school districts, and supporting companies. The state provides a grant to each district, which must be matched in direct or in-kind support by both the receiving district and local business community. Thus the funding mechanism is designed to encourage cooperation among school districts and the private sector companies in preventing dropouts. For the past three years the state grant to academy districts has been $50,000 per year. During the 1987 legislative session, a bill (SB 605) was passed to expand the replications by up to another fifteen sites per year for each of the next four years. During the 1988—89 school year, eighteen academies were in operation under state sponsorship, with a total of thirty-three planned by the fall of 1990. SB 605 changes the system by which academies are funded. Rather than a $50,000 grant provided at the beginning of the school year, districts will receive an amount based on the number of students they keep in school and who are making normal progress toward graduation, as determined at the end of the school year. The amount can be as high as INTRODUCTION 3 $67,500. Thus the programs will survive if they are successful and will be phased out if they are not. The required matching support from the receiving districts and local business communities continues. A copy of SB 605 is included as Appendix B. Site Descriptions The twelve academy programs in operation during the 1987—88 school year are located in ten school districts. Six are in San Francisco Bay Area high schools, three in or near Sacramento, and one each in Bakersfield, Santa Barbara, and Inglewood. They were all selected through a grants competition conducted b§ the state Department of Education. They are at three stages of development: - The two Peninsula Academies were in their seventh year of operation during the 1987—88 school year - Eight of the replications which began in the fall of 1985 were in their third year of operation in 1987—88 - The academies in Santa Barbara and Inglewood, replacements for two dropouts from the first round of replications, were in their first year of operation in 1987—88 The “Academy School Profile” chart on the next page provides information on these 12 high schools, including name, enrollment, percent of LEP and AFDC students, number of graduates, and student ethnicity. As it shows, 44 percent of students in these schools were black or Hispanic, and over 50 percent were minority. The Evaluation Design The evaluation includes both process and outcome components. The process evaluation examines successes and problems regarding implementation. This information derives from site visits, interviews with school staff, students, and business representatives, and questionnaires completed at the end of the school year by all involved parties. The process findings are presented in the individual case studies in Chapter 2, and summarized across sites in Chapter 3. The outcome evaluation employs a comparison-group design. This means that a group of students not participating in the academy program was selected at each high school and matched with academy students, and the performance of both groups was tracked. The matching was done in terms of descriptive indicators (age, grade level, gender, and Academy School Profiles ‘ . 86—87 STUDENT SCHOOL / COUNTY % % SES I987 Amerlcan Paclllc 13mm AFDC LEP fink. East Bakersfield lllgh School 2.176 11.3 0.0 29 Kern County lllram Johnson [“311 School 2.775 41.0 5.1 36 86 87 436 1.4 18.7 .0 2.5 14.9 18.1 44.3 Sacramento County Mountaln Vlew ngh School 1.299 1.9 5.0 93 65 89 283 .1 13.3 .5 5.3 8.7 6.0 66.1 Santa Clara County Oakland Technlcal ngh Schmll 1.607 35.8 16.6 27 14 45 216 .2 16.6 .2 1.4 3.0 72.7 5.8 Alameda County Oak Rldge lllgh School 931 2.0 0.0 89 14 76 160 .0 .4 .4 .1 2.0 .2 96.8 Eldorado County Rlo Cazadero Illgh School 233 NotAvaihble 17 .9 .4 .9 1.7 14.6 24.5 57.1 Sacramento County Independence "1311 School 4.063 9.7 29.2 52 28 76 766 1.2 24.1 .2 12.6 29.4 7.7 24.8 Santa ClaraCounty Sllver Creek 11131) School 2.420 15.8 22.9 34 28 84 361 1.7 23.4 .7 8.4 29.6 12.3 24.0 Santa Clara County Menlo Atherton ngh School 1.681 4.1 3.0 93 86 90 327 .1 4.7 2.2 1.4 14.3 15.7 61.6 San Mateo County Sequola lllgh School 1.710 2.6 0.0 60 87 99 314 .1 3.8 2.3 1.5 31.9 .9 55.6 San Mateo County San Marcos lllgh School 1.698 0.6 0.6 85 71 84 369 .4 4.1 .2 .4 26.6 2.4 65.8 Santa Barbara County Inglewood "1311 School 2.294 15.1 2.7 34 11 4 419 .0 1.5 .5 .2 32.0 65.0 .8 Los Angeles County WEIGIITEI) MEANS 14.0 10.0 52 .7 12.0 6 4.2 24.0 20.0 39.0 NOILVH'IVAEI 88-L861 salwaavav V’IDSNINEIcI INTRODUCTION 5 ethnicity) and school records (attendance, credits, courses failed, and grade point average) from the year prior to the students’ entry into the program. Both academy and comparison- group students were then tracked on these same outcome variables, as well as on retention in school. The statistical procedures used to test for differences between academy and comparison- group students is described in Appendix A. In addition, the outcome evaluation includes information on academy students from two additional sources. The first was a student questionnaire that collected information on students’ attitudes toward careers, school, and self, and tracked changes from year to year. Questionnaires were administered before students participated in the program and at the end of each school year. A series of items related to students’ socioeconomic status was included in these questionnaires. A standardized achievement test was also administered to academy students the past two years, tracking their progress on this dimension. Report Overview Chapter 1 of the report is this Introduction. Chapter 2 includes each of the individual site case studies. There are nine of these. The two Peninsula Academies in the Sequoia High School District, and the two electronics academics in the East Side San Jose School District, are jointly administered and identically structured, and so in each case are treated together. Although Inglewood was officially listed as an academy site, in fact it was never provided state funding nor was it operative during 1987-88, and therefore is omitted here. Each case study contains several sections: . A program description, which includes the setting, a student profile, a program management description, and a sketch of program resources 0 A summary of statistical tests showing differences between academy and comparison-group students on attendance, credits earned, grades, courses failed, and dropouts - Questionnaire feedback from students, school personnel, parents, private sector representatives, and mentors - A summary with conclusions A briefer version of this format was used in the San Marcos (Santa Barbara) case study, where the academy was at an initial stage of development in 1987—88. 6 PENINSULA ACADEMIES 1987—88 EVALUATION Chapter 3 provides a summary and conclusions. It presents a cross-site summary of the findings with respect to program implementation, and student outcomes and questionnaire feedback. It includes a calculation of the net economic benefit to society from the academies’ dropout-prevention program. It also offers some conclusions about what is needed to implement a successful academy. Finally there is a brief section of recommendations. Chapter 2 Academy Case Studies . The Health Careers Academy, Bakersfield - The Johnson Corporate Academy, Sacramento - The East Side Electronics Academies, San Jose - The Peninsula Academies, Redwood City - Computech Academy, Mountain View ' The Foothill Computer Academy, El Dorado Hills 0 The Health Academy, Oakland - The Rio Cazadero Academy, Elk Grove 0 The Computer Academy, Santa Barbara . ‘ t '. . ‘ 8:. 7‘ a": £["‘,~i an .ASV-’.r_ £5 xxx-“51 ' The Health Careers Academy East Bakersfield High School Kern Union High School District Bakersfield, California The Health Careers Academy continued much as it had the previous year, operating effectively at the high school and struggling with private sector and community support. The two teachers who provided a strong nucleus of staff for the program continued in this role, and with a year of experience working together became even more effective. Curriculum continued to develop, with seniors enrolling in a three-period ROP health course. The high school’s Dean of Students took a more active role in supporting the program, helping to secure resources from the school and district, to organize an Advisory Board, and to develop jobs. While there was an active program of speakers and field trips, and some mentors and jobs were made available to students, private sector involvement continued to be the program’s greatest weakness. Setting Bakersfield is located in the California central valley, north of Los Angeles and south of Fresno. Its population is approximately 100,000, and its economy is based largely on agriculture and oil. East Bakersfield High School, one of ten high schools in the district, was built in the late 19303; it underwent extensive modernization during the last year. It is located on the east side of the city in a low- to middle-income neighborhood. Its 2,200 students are 2 percent Asian, 8 percent black, 45 percent Hispanic, and 45 percent white. Its socio-economic status percentile rank in the state (based on CAP testing) is 29, and its 1986—87 CAP percentile scores are 71 (reading) and 94 (math). Student Profile As the program has grown over the past two years, it has achieved an increasingly established image in the school and has developed an effective system of recruitment, based on student applications and referrals from other teachers and counselors. Students are selected using a number of criteria: 1 0 PENINSULA ACADEMIES 1987—88 EVALUATIONS - Aptitude in science, health, and the ability to get along with people 0 Patterns of underachievement in 9th grade, in terms of attendance, grades, and credits 0 Emotional and family problems, including economic disadvantage, in order to include those with such problems as long as they are not too severe for the program to handle - Involvement in Gifted and Talented Education (G ATE), which eliminates students from consideration 0 Other indicators of at-risk status The table below provides a profile of the 1987-88 academy classes, and their matched comparison groups. TABLE 1. Student Profile, Bakersfield Academv Comparison Group Class (N) Sophomores 43 47 Juniors 23 47 Seniors 13 31 Race/Ethnicity (%) White 33 36 Black 11 8 Hispanic 54 55 Asian 0 0 Other 1 1 Gender (%) Male 39 57 Female 61 43 THE HEALTH CAREERS ACADEMY 1 1 Program Management The program is managed jointly by the two teachers whose time is fully dedicated to the program—the English and science/lab teachers. In addition, another part-time lab teacher joined the program’s staff, and will be more involved next year. And the seniors this year were enrolled in a three-period health course that covered a wide variety of health-related topics. Perhaps the biggest change in the program at the school level was the active involvement of the Dean of Students, who is responsible at East Bakersfield High for all retention programs. He not only provided administrative support at the school, but headed an effort to organize an Advisory Board with active private sector support, and helped to secure summer jobs for juniors and positions for graduating seniors. The program follows the school-within-a-school structure, with one exception—the absence of math. Sophomores and juniors have English, science, and the lab as part of the program, while seniors have the lab and an ROP “hospital health” course. Attempts were made the first year to incorporate math into the program, and encountered the problem that students were at so many different levels in their ability that there was no one math course into which they could sensibly fit. However, plans for next year include an academy physical education course at both the sophomore and junior levels, and a one-semester “computers/Spanish for medicine” course for sophomores (nine weeks on each topic). Substantial efforts are made to incorporate health topics into the sophomore and junior English and science classes. There is as yet no Advisory Board for the program, or private sector involvement in its management. Efforts to develop this were frustrated, although these efforts are continuing and appear to be headed for success. There is also no systematic private sector input to the curriculum, which will be an early order of business for the Advisory Board if and when it does materialize. Other features of private sector involvement fared variously: 0 There were many speakers, including a dental assistant, transplant expert, surgical nurse, nurse practitioner, representative of Charter Hospital, and of the epilepsy society, a speaker on personal integrity and self image, another on anorexia, another on contraceptives. - There were field trips to a dialysis center, Charter and Mercy Hospitals, the Kern Medical Center, a wildlife preserve, and a trip was planned for late spring to the 1 2 PENINSULA ACADEMIES 1987-88 EVALUATIONS Martin Luther King Jr. Hospital and the Charles Drew Medical School in Los Angeles. 0 While a few students were provided with mentors this year, neither the teachers nor students view this part of the program as successful; plans are being made to improve both the recruitment and training of mentors for next year. 0 Eight of the juniors had summer jobs last summer, and it appears all who qualify will this summer, most through the local JT PA SYETP; those who fail to qualify for this will be placed through a Chevron summer jobs program. 0 About half of the seniors had part-time school-year jobs, and it appears that all will be provided with positions upon graduation, although most are planning to go to Bakersfield Community College at least part time. Conversations with seniors also revealed a close sense of family and comradeship among these students. Clearly the academy is working as intended in this sense. Students had helped to raise money for the program, and liked the identity it provided them. They had shared in various social activities during the year, and were eagerly looking forward to the awards/ graduation ceremony planned at a local restaurant to honor them. The high school went through an extensive remodeling during the school year, and looked considerably modernized, given its 50-year-old status. Related to this, the program requested and received another classroom, a fully equipped science lab, and will move the English classes to the present lab room, giving it two “home bases.” Parental contact continued this year, with calls made whenever students were absent two days in a row. Teachers continue to view this as an important part of the program’s student support network. Three computers that were hoped for did not materialize, although academy students will be able to use the high school’s computer lab next year. Teachers feel more support for staff development would be useful, particularly visits to other academies. THE HEALTH CAREERS ACADEMY 1 3 Program Resources/Costs District Support The academy’s 2.4 full-time equivalent teachers represent an increase of .9 additional teachers over what would be required without the smaller class sizes and extra released period provided for the program’s lead teacher. The program enjoyed the services of a 40% time instructional assistant. The Dean of Students contributed 20% of his time to the program this year. No specific staff development activities occurred, although the academy teachers did make presentations at several faculty meetings. Private Sector Support An estimated 19 days of time were contributed by representatives of local employers: eight days supervising students on the job, three days speaking to groups of students, three days organizing and leading field trips, three days arranging job placements; and two days serving as mentors. Other categories of support, which were contributed to by a mixture of state, district, and private sector support, were: - Facilities. Approximately $35,000, partly state and partly district, was spent on preparing a new science/biology/health careers lab for use in the fall of 1988. . Equipment. Approximately $150 for dissecting tools. Approximately $7,000 in district expenditures is planned for the new lab in 1988—89. . Supplies. Approximately $520 was spent on dissection animals, and regular and medical dictionaries. Again, plans for 1988—89 disuict expenditures are much larger here—approximately $5,000. . T ranspggatign. A total of about $500 in school funds was spent on field trips. - Rewards/social events. Approximately $1,100 was spent in this category, most from student-raised and district funds. Activities supported included a fall picnic, honor banquet for seniors and mentors, and scholarships for graduating seniors. 14 PENINSULA ACADEMIES 1987—88 EVALUATIONS Student Outcome Data The most objective data by which to judge the program’s effectiveness are measurable indicators of student performance. These include attendance, credits earned, grade point average, courses failed, and retention in school (the opposite of dropouts). While these are not the only indicators of the program’s effect on students, they provide the most statistically convincing measures. There are now data across three years by which to judge the program’s performance: that from 1987-88 for all three cohorts; from 1986—87 for cohorts one and two; and 1985—86 for cohort one. Table 2 on the following page presents differences in outcomes between each cohort of academy students and their matched comparison groups. These differences have been adjusted to account for differences between the two groups in age, gender, race or ethnicity, and academic performance in the preceding year. In this table, positive numbers for “attendance,” “credits,” and “GPA” mean that academy students compiled better records than the comparison group. Negative numbers for “courses failed” and “dropout probability” also mean that academy students performed better than the comparison group. Only when an asterisk appears next to the value is the difference large enough to be statistically significant. Of the 29 differences shown in this table, 19 favor the academy students and 10 the comparison groups. Of those favoring the academy students, one is statistically significant; of those favoring the comparison students, four are statistically significant. The first cohort (seniors) of academy students significantly outperformed the comparison group in attendance its first year. The second cohort of comparison students (juniors) significantly outperformed the academy students in attendance, credits, courses failed, and grade point average its second year. However, there were problems in forming this second cohort comparison group, and it is not as well matched as the first and third, suggesting that these results should be viewed cautiously. With respect to dropouts, because the actual number (as opposed to the percent) of students who drop out is relatively small when analyzed by individual cohort, few statistical tests of differences between academy and comparison groups will prove to be significant, even when differences in the attrition between the two groups is substantial. For this reason, the following table presents the actual differences between the two groups. THE HEALTH CAREERS ACADEMY 1 5 TABLE 2. Adjusted Differences Between Academy and Comparison Groups, by Class, Bakersfield Health Careers Academy. Atten- Courses Dropout dance Credits GPA Failed Probabilitv Cohort entering fall 1985 (seniors): 1985-86 Outcomes 55* 0.7 0.1 —0.2 —0.03 1986—87 Outcomes -3.7 1.9 0.1 -0.3 ~0.08 1987—88 Outcomes 4.3 5.5 0.1 0.1 —O.11 Cohort entering fall 1986 (juniors): 1986—87 Outcomes 2.6 1.4 0.1 -0.2 i 1987—88 Outcomes —-—3.7#* —9.4* —O.6* 1.4* 0.08 Cohort entering fall 1987 (sophs): 1987—88 Outcomes ~0.1 —0.2 0.2 0.1 0.16 # Estimated from separate regressions for academy and comparison groups i Insufficient number of dropouts for analysis * Statistically significant The overall “attrition” (what is usually meant when the term “dropout” is erroneously used) is broken out into its three components: known school dropouts (official withdrawals), transfers to other high schools, and probable dropouts (students lacking records as of June 1988 who neither officially withdrew nor transferred). Table 3 presents the figures by cohort. For the first cohort, seniors in 1987—88, the figures represent the cumulative rates across the three years of the program; for the second cohort, juniors, they represent the cumulative rates over two years; and for the third cohort, sophomores, they represent the rates for just the last year. 16 PENINSULA ACADEMIES 1987—88 EVALUATIONS TABLE 3. Known Dropouts, Probable Dropouts, and Transfers, by Cohort, for Academy and Comparison Group Students Academv Comparison Group Known Prob. Trans- Known Prob. Trans- Drons Droos fers Drops Droos fers Cohort 1 9% 7% 27% 7% 2% 19% Cohort 2 3% 6% 14% 2% 2% 17% Cohort 3 7% 6% 21% 5% O 5% This table suggests that the academy is not reducing attrition, although the particular at- risk group served in the program here and the problems in forming a well-matched comparison group for the second cohort make it difficult to draw clear conclusions. Student Questionnaires Academy students complete questionnaires at the point they enter the program, at the beginning of their sophomore year, and again at the end of each school year. Some of the questions—on career-related plans, and attitudes related to career, school, and self—appear in parallel form on the “pre” and “post” versions of the questionnaires, allowing assessment of changes over the course of the program. Another set of questions appears only on the “post” version, and asks for feedback on various program activities, and ratings of certain program components. The sections below present feedback from these questionnaires. Responses have been averaged across cohorts to simplify their presentation, except where clear differences appear between cohorts. Thirty-seven students are represented in these data: six seniors, 10 juniors, and 21 sophomores. o Proorg am Activities. Average number of academy classes reported by students was 2.0 for sophomores, 2.6 for juniors, and 1.0 for seniors. One-fourth of juniors and one-third of seniors reported having had mentors. Four juniors reported THE HEALTH CAREERS ACADEMY l 7 having had a job. Sophomores reported having had 10 speakers, juniors and seniors five. Seniors went on two field trips, juniors five, and sophomores none. 0 Proorg am Ratings. Three-fourths of sophomores rated the equipment, materials and schedule highly; one-third of juniors and seniors did so. Ninety percent of academy students saw a close connection between what they were studying in school and their post— graduate plans. Ninety percent also liked the academy better than the regular program. - Career-Related Plans. Junior data is missing in this and the next section. One-third of participants reported having a definite post-graduate plan. Of these, 85 percent were planning on college (up substantially from pre-program responses), and 60 percent were planning on a job. One~third of seniors had a long-term career goal; 80 percent of sophomores did. - Attitudes Related to gamer, School, and Self. Ninety percent of academy students view it as important to make career plans and gain job skills while in high school. All seniors found their classwork interesting during the last year, as did 60 percent of sophomores; in both cases these were substantial increases from pre-program responses. All seniors reported that they “feel good about themselves,” as did 90 percent of sophomores (a 20 percent increase over pre-program responses). School Personnel Responses Four staff from East Bakersfield High School completed a questionnaire at the end of the school year—three teachers and one administrator. Responses from these questionnaires brought out the following: - The equipment and facilities were rated “poor,” the instructional materials “average,” and the curriculum “excellent.” - Non-academy teachers were rated as “somewhat supportive,” while counselors and administrators were rated as”very supportive.” - Program planning was rated “good,” with several comments that it was improving over time. - Cooperation and support from the private sector was rated from “not very effective” to “non-existent.” 1 3 PENINSULA ACADEMIES 1987-88 EVALUATIONs 0 Other aspects of the program were rated variously. Student selection, motivational activities, and individual student attention were all given high ratings; counseling was seen to be “inadequate,” and parents were rated “somewhat supportive and interested.” ' On a seven point scale (1 = very weak; 7 = very strong), the academy received an average rating of 5.3 “in concept” and 5.5 “as implemented.” 0 The greatest weakness was seen to be the need for more community support, more mentors, an academy counselor, a larger science facility (provided in 1988-89), and “new rules concerning attendance, failures, and senior graduation.” 0 The greatest program strength was seen to be the quality of the teachers and other staff involved in the program, and the curriculum. Parent Responses Three parents completed questionnaires at the end of the school year. Their responses included the following: o All three rated the program as “excellent.” 0 Two saw their children as “much more interested in school and learning” since joining the academy; the third parent saw no change. 0 Parents had been involved with the program in several ways: attending school events, helping their child with homework, meeting with teachers and counselors, and helping with a program activity. ' Asked how well they had been kept informed of their children’s progress in school, all three said “very well.” 0 The features of the academy that parents liked best were the teachers’ attitudes and support of students, and the closeness between teachers and students. - The only suggestion offered for improvements in the program was to provide teachers with aides. THE HEALTH CAREERS ACADEMY 1 9 Private Sector Representatives Responses One private sector representative, an employee of a local hospital it was hoped would provide support for the program, completed a questionnaire at the end of the school year. Her responses: - Asked whether the academy’s curriculum would prepare students for jobs locally, she responded “don’t know.” - She was unable to rate any students as employees. ' Asked about communications with the program, she rated these as “good.” 0 The effectiveness of the Steering Committee was rated as “good” (although none actually existed). . On a seven-point scale (1 = very weak, 7 = very strong), the academy was rated 5 both “in concept” and “as implemented.” - The respondent felt unable to comment on program weaknesses with the limited contact she had had. She felt the program was well-designed, preparing goal— oriented youth for work in the field. Mentor Responses Two mentor questionnaires were received. Their responses showed that: . On average, during the school year mentors got together with their proteges twice in person, and talked by telephone twice. - Neither mentor visited their student at school; both brought the student to their place of work. . Asked whether the academy’s curriculum would prepare students for jobs in local companies, both said “don’t know.” - Asked to rate the mentor program, one said “somewhat useful,” the other OK” 2 0 PENINSULA ACADEMIES 1987-88 EVALUATIONS; 0 The program’s biggest problem was seen to be matching students’ and mentors’ schedules. 0 The greatest strengths of the program were seen to be “giving students a broader awareness of what their career choices entail,” and “showing career choices in a non—threatenin g atmosphere.” Summary The student outcomes analysis for the 1987—88 school year shows one disturbing finding: the second cohort of academy students (juniors) performed very poorly, across the board, on the measures examined (attendance, credits, courses failed, and grade point average), in comparison with their matched comparison group. The reason for this is unclear, but needs to be explored and addressed. For other student outcomes measures, both other years and other cohorts, the pattern of differences favors the academy students 19 to 6, although only one of these differences reaches statistical significance. The other information presented here is on balance positive and encouraging. The program is staffed with strong teachers who provide it with real leadership. Support from the school administration, in the person of the Dean of Students, is now also strong. Student attitudinal feedback is largely positive. While juniors and seniors rated the equipment and materials as inadequate (as did the teachers), an entire new science lab has been provided the academy by the high school for the 1988—89 school year. Feedback from teachers and parents is also generally very positive. Private sector involvement is clearly still a weakness, but efforts are being made to remedy this problem as well; in the summer and fall of 1988 a mentor program involving almost the entire population of the academy was put in place. This is a program that began with many weaknesses. It has seen a clear and steady improvement over the past three years. If the trend continues, it could well become one of the most effective academies in California. The Corporate Academy Hiram Johnson High School Sacramento City Unified School District Sacramento, California The Corporate Academy at Hiram Johnson High School continued to function fairly well this year. It offered a full complement of classes supplemented by an active program of speakers and field trips. The strong teaching staff remained intact from the previous year, including an English, math, social studies, and computer teacher, and this team continued to work together effectively. Leadership continued to be provided by the high school’s outreach consultant and principal. Weaknesses a year ago included lack of district and private sector support for the program. Some progress was made on both fronts. The district provided support for some above-ratio staffing, to allow smaller class sizes in the academy. Next year two of the teachers will also have an extra preparation period. The mentor program improved, largely through the efforts of the Sacramento East Area Rotary Club. Some progress was also made in securing computer-related jobs, although program staff recognize this as an area that still needs much work. Private sector involvement in general continues to be an area for improvement. Setting Sacramento, the state capitol, is the largest California city in the central valley, with a population of about 300,000. It is the fastest growing metropolitan area in California, with a healthy, diverse economy, including agriculture, state government, and services. Hiram Johnson is one of ten high schools in the Sacramento City Unified School District (the city includes other districts). It is a relatively modern facility on the city’s south side, in an inner—city neighborhood. Its approximately 2,800 students are 21 percent Asian, 18 percent black, 15 percent Hispanic, 44 percent white, and 2 percent other. Its statewide socio—economic status percentile is 36 (based on CAP testing), and its 1986—87 CAP percentile rankings are 86 (reading) and 87 (math). 21 2 2 PENINSULA ACADEMIES 1987—88 EVALUATIONS Student Profile The program uses a well-defined selection system. Ninth grade students are told about the program, and those who express interest apply. Participants are then selected using several criteria: 0 Standardized test scores that are low, but not more than three years behind grade level (fifth grade level when tested in eighth grade) 0 Poor attendance, specifically more than 20 percent absences 0 Credit deficiencies of 10 credits or more (of 60 expected in ninth grade) Interest is a particularly important criterion, as teachers feel they can best serve students who genuinely want to be in the program. Since most students in the school are “at-risk,” they feel this is the best way to distinguish appropriate candidates. For those students who apply and meet all the above criteria, parents are asked to attend an information meeting and sign a permission and support form. Table 1 on the following page provides a profile of the 1987—88 academy classes, and their matched comparison groups. Program Management The program is directed by Hiram Johnson’s Outreach Consultant, with strong support from the school’s principal. The director works out of a “Corporate Academy” office, next to two academy classrooms. One of these is the computer lab, which is well supplied with computers, software, and peripherals. The director does not teach any academy classes, but does coordinate efforts among the teachers and manages the many features of the program, both within the school and those involving the private sector. This year the teachers also played a stronger role in coordinating various components of the program. For example, the social studies teacher was primarily responsible for contacting parents, and made over 100 calls during the year, primarily for attendance problems. She is planning to play a larger role in helping seniors make post-graduate plans next year, when she will have an extra preparation period. THE CORPORATE ACADEMY 2 3 TABLE 1. Student Profile, Hiram Johnson Academv Comparison Group Class (N) Sophomores 27 87 Juniors 31 56 Seniors 42 24 Race/Ethnicity (%) White 37 36 Black 30 28 Hispanic 20 22 Asian 9 10 Other , 3 4 Gender (%) Male 50 59 Female 50 41 The math and English teachers took charge of the mentor program, organizing a series of panel discussions, each of which included a group of Rotarians who talked about their work and background, answering students’ questions. The matching process was reorganized to be more effective. Approximately 20 juniors had mentors this year, a fewer number than wanted them. But teachers felt this program was more successful than it was previously, as mentors were better prepared and more active than in the past. The program of business speakers overlapped with the mentor program, via the panel discussions described above. In addition to the Rotarians, who represented a broad cross section of professions, the program brought in a mortician, bondsperson, retired death row warden, and representatives from a fashion institute and the League of Women Voters. Student response to these speakers was very positive. Meanwhile the social studies teacher took charge of field trips. Three trips were made to the Bay Area, and included stops at Alcatraz, the United States Mint, the Exploratorium, and the Steinhart Aquarium. Seniors also visited the Sacramento Courthouse. Field trips 2 4 PENINSULA ACADEMIES 1987-88 EVALUATIONS included all students who met grade and citizenship requirements. English classes included research in preparation for each trip as well as written papers following them. A high school counselor was assigned to the program this year and provided various kinds of support. In addition to monitoring academy student detention and billing, she talked individually with each student about their credits and progress toward graduation. The director also helps with counseling, particularly related to career plans and job placement. Perhaps least progress was made on the job front. While about half of the seniors had part-time jobs this year, most got them on their own, and few were related to the program’s training. Approximately ten data entry jobs had been located for the summer for juniors, and this seems to be a fertile direction in which to develop jobs in the future. The majority of jobs for academy students were still being located by students themselves, however, and many were in fast foods or other fields unrelated to the program’s training. Related to this weakness was the fact that the Advisory Board was again inactive. And again there was little private sector input to the program’s technical curriculum. These are areas for attention next year. The district showed some interest in the program. A district committee looking into programs to reduce dropouts visited the academy, talked to teachers and students, and received what was felt to be a positive picture of the program as a model for wider use in the district. And some support for the extra preparation periods, as well as some release time for teachers’ extra responsibilities, will be made available next year. The fact that the district and teachers’ union have been at impasse over the teachers’ contract has lent a generally adversarial tone to relations between the two, which has not helped the program. Support at the high school itself is strong, and the program will regain its block scheduling next year. Students and teachers particularly like the family atmosphere, greater teacher coordination, and the fuller sense of caring that students experience. The sense of trust that has developed between students and teachers is symbolized by the fact that one teacher has loaned students up to $50 in “crisis” situations during the year, and has been paid back every cent. Recruitment of new students went well this spring, and the program expects to take in approximately 70 sophomores in the fall. THE CORPORATE ACADEMY 2 5 Program Resources/Costs District Support The academy’s four full-time-equivalent teachers represent an increase of approximately two additional teachers over what would be required without the smaller class sizes and extra released periods for the teachers. In addition, the high school’s Outreach Counselor contributes a portion of his time as the program’s administrator. Teachers spent eight days this year in staff development activities, making program plans. Private Sector Support An estimated 112 days of time were contributed by representatives of local employers: 40 speaking to groups of students—35 in advisory committee meetings, 22 serving as mentors, 10 organizing field trips, four arranging job placements; and one supervising students on the job. Other categories of resources: - Facilities. A regular school classroom is used as the computer lab; this required no additional resources this year. - Eguipment. Approximately $40,000 was spent from the state grant the first year of the program for computers. No new expenditures were made this year. - Supplies. The grant covered $5,000 in software and computer supplies, and $1,000 for miscellaneous school supplies. 0 Transportation, A total of about $5,000 in grant funds was spent on field trips. . Rewards/social events. Approximately $500 was spent from the grant in this category on awards and certificates. 2 6 PENINSULA ACADEMIES 1987—88 EVALUATIONS Student Outcome Data The most objective data by which to judge the program’s effectiveness are measurable indicators of student performance. These include attendance, credits earned, grade point average, courses failed, and retention in school (the opposite of drOpouts). While these are not the only indicators of the program’s effect on students, they provide the most statistically convincing measures. There are now data across three years by which to judge the program’s performance: that from 1987—88 for all three cohorts; from 1986—87 for cohorts one and two; and 1985—86 for cohort one. Table 2 presents differences in outcomes between each cohort of academy students and their matched comparison groups. These differences have been adjusted to account for differences between the two groups in age, gender, race or ethnicity, and academic performance in the preceding year. In this table, positive numbers for “attendance,” “credits,” and “GPA” mean that academy students compiled better records than the comparison group. Negative numbers for “courses failed” and “dropout probability” also mean that academy students performed better than the comparison group. Only when an asterisk appears next to the value is the difference large enough to be statistically significant. Of the 25 differences shown in this table between academy students and their comparison groups, 13 favor the academy students and 12 the comparison students. Of those favoring academy students, four are statistically significant; of those favoring comparison students. none is. The first cohort (seniors) of academy students outperformed its comparison group on credits and courses failed its first year. The second cohort (juniors) of academy students outperformed its comparison group on attendance and grade point average its first year. With respect to dropouts, because the actual number (as opposed to the percent) of students who drop out is relatively small when analyzed by individual cohort, few statistical tests of differences between academy and comparison groups will prove to be significant, even when differences in the attrition between the two groups is substantial. For this reason, Table 3 presents the actual differences between the two groups. The overall attrition (what is usually meant when the term “dropout” is erroneously used) is broken out into its three components: known school dropouts (official withdrawals), transfers to other high schools, and probable dropouts (students lacking records as of June 1988 who neither officially withdrew nor transferred). Table 3 presents the figures by cohort. For the first cohort, seniors in 1987-88, the figures represent the THE CORPORATE ACADEMY 2 7 TABLE 2. Adjusted Differences Between Academy and Comparison Groups, by Class, Hiram Johnson Corporate Academy Atten- Courses Dropout dance Credits GPA Failed Probabilitv Cohort entering fall 1985 (seniors): 1985.86 Outcomes NA 7.6* 0.3 -1.1* i 1986—87 Outcomes 1.3 -3.0 —-0.3 0.5 0.08 1987—88 Outcomes 2.5 -8.3 -0.1 0.3 i Cohort entering fall 1986 (juniors): 1986—87 Outcomes 7.1* 5.3 0.4’* —1.1 i 1987—88 Outcomes '—3.7# —4.6# —0.2 0.1 0.06 Cohort entering fall 1987 (sophs): 1987—88 Outcomes 1.9 1.2 —0.1 —0.3 i # Estimated from separate regressions for academy and comparison groups i Insufficient number of dropouts for analysis * Statistically significant cumulative rates across the three years of the program; for the second cohort, juniors, the cumulative rates over two years; and for the third cohort, sophomores, the rates for just the last year. These figures suggest that the academy is having a substantial influence in reducing attrition among its participants. 2 8 PENINSULA ACADEMIES 1987—88 EVALUATIONS TABLE 3. Known Dropouts, Probable Dropouts, and Transfers, by Cohort, for Academy and Comparison Group Students Aca dem V W Known Prob. Trans- Known Prob. Trans- Droos Drops fers Drops Droos fers Cohort 1 9% O 24% 35% 8% 31% Cohort 2 5% 15% 13% 24% 9% 17% Cohort 3 5% 0 5% 9% 0 9% Student Questionnaires Academy students complete questionnaires at the point they enter the program, at the beginning of their sophomore year, and again at the end of each school year. Some of the questions—on career-related plans, and attitudes related to career, school, and self—appear in parallel form on the “pre” and “post” version of the questionnaires, allowing assessment of changes over the course of the program. Another set of questions appears only on the “post” version, and asks for feedback on various program activities, and ratings of certain program components. The sections below present feedback from these questionnaires. Responses have been averaged across cohorts to simplify their presentation, except where clear differences appear between cohorts. Twenty-six students are represented in these data: 14 juniors and 12 sophomores. Senior data are missing. - Procram Activities. Average number of classes reported in the academy was three. About 15 percent of sophomores and juniors had mentors. Four reported having jobs. Participants had five speakers during the year. Juniors went on two field trips, sophomores one. 0 Program Ratings. About 80 percent of students rated the equipment and materials highly. Two-thirds of sophomores liked the academy schedule; 30 percent of juniors did so. Two-thirds of SOphomores saw a close connection between what they were studying in school and their postgraduate plans; 20 percent of juniors did THE CORPORATE ACADEMY 2 9 so. Ninety percent of sophomores liked the academy better than the regular program; 45 percent of juniors did so. - career-Related Plans. One-fourth of participants reported having a definite post- graduate plan; of these, three-fourths were planning on college (up somewhat from pre-program responses); one—third planned on a job (down sharply). About half have a long-term career goal (down slightly from pre-program responses). . A i R l r r h l n if. Three-fourths of academy students view it as important to make career plans and gain job skills while in high school. Three-fourths of sophomores found their classwork interesting during the last year; 20 percent of juniors did so. All sophomores “feel good about themselves” (a 14 percent increase over pre-program responses); half of juniors do so. School Personnel Responses Four staff from Hiram Johnson High School completed a questionnaire at the end of the school year—two teachers and two administrators. Responses from these questionnaires: - The equipment was rated as “excellent,” and the facilities, instructional materials, and curriculum as “good.” ~ Non-academy teachers and counselors were rated as “somewhat supportive,” while administrators were rated as “very supportive.” . Program planning was rated as “good.” - Cooperation and support from the private sector was rated “somewhat effective.” 0 Student selection, motivational activities, individual student attention, counseling, and parental support were all rated good. 0 On a seven point scale (1 = very weak; 7 = very strong), the academy received an average rating of 6 “in concept” and 5.5 “as implemented.” - The greatest weaknesses were seen to be district support, business involvement in job placement, the lack of block-scheduled classes, and the need to expand the program. 3O 0 PENINSULA ACADEMIES 1987-88 EVALUATION: The greatest program strengths were seen to be teacher cooperation and positive attitudes, small class size, student-teacher comraderie, and the sense of belonging provided the students. Parent Responses Four parents completed questionnaires at the end of the school year. Their responses: One rated the program as “excellent,” two as “good,” and one as “poor.” One saw their child as “much more interested in school and learning” since joining the academy; the other three saw no change or were unsure. Parents had been involved with the program in several ways: attending school events, helping their child with homework, meeting with teachers and counselors, and helping with a program activity. Asked how well they had been kept informed of their child’s progress in school, all four said “moderately well.” Strengths of the program were seen to be the close contact between students and teachers, small classes, field trips, weekly reports to parents, and staff working together toward a common goal. Suggestions for improvements included more parental involvement, more speakers, more career-oriented field trips, more summer jobs, and clearer guidelines for mentors. No “Private Sector Representative Questionnaires” were received here. Mentor Responses Three mentor questionnaires were received. Their responses showed that: On average, during the school year mentors got together with their proteges twice in person, and talked by telephone once. THE CORPORATE ACADEMY 3 1 0 All three mentors visited their student at school; two brought the student to their place of work. 0 Asked whether the academy’s curriculum would prepare students for jobs in local companies, one said “yes,” one “no,” and one “don’t know.” 0 Asked to rate the mentor program, one said “somewhat useful,” the other two “O'K.” - The program’s biggest problems were seen to be the need for more mentors, more contact between school staff and mentors, and for more discussion sessions with students. . The program was seen to be a valuable resource for students, particularly the discussion sessions with groups of mentors and students involved. Summary The student outcomes analysis shows some evidence of a first-year academy effect, which appeared in 1985—86 for credits and courses failed, and in 1986—87 for attendance and grades. None of the 1987—88 tests showed statistically significant differences between the academy and comparison groups. However, the data on dropouts do suggest a substantial program effect in reducing these. Questionnaire feedback from sophomores was quite positive; from juniors it was lukewarm (senior feedback was missing). Teacher feedback was generally positive. That from parents and mentors was mixed. This academy continues to demonstrate considerable potential, but also to encounter substantial problems. Both district and private sector support continue to be less than needed to allow full implementation of the academy model. One of the strengths of the program has been its four main teachers; two of these have been lost for the 1988—89 school year (one to promotion, one to retirement). The high school itself encountered a serious asbestos problem in the fall of 1989, losing its freshman and sophomore classes and further complicating things. At the same time, there continues to be strong support at the high school, where both the principal and outreach counselor provide leadership. The academy is well-implemented in terms of its high school structure and activities. Its teachers have been involved and dedicated. Private sector involvement seems to be increasing. If the strengths of the 3 2 PENINSULA ACADEMIES 1987—88 EVALUATIONE program can be built on and the weaknesses eliminated, its potential for having a positive impact on Hiram Johnson’s at-risk students is considerable. The East Side Electronics Academies Silver Creek and Independence High Schools East Side Union High School District San Jose, California In the 1987—88 school year the two academies in the East Side Union High School District completed their transition towards three~year, grades 10—12 programs, and henceforth will follow the general academy progression. The electronics academies at Independence and Silver Creek High Schools each began in the 1985-86 school year with more than one hundred 9th grade students. The original plan was to move each group of one hundred through graduation in the summer of 1989 without adding additional new classes. But a combination of attrition in the original class and the realization that the grades 10—12 model followed elsewhere would work best at Eastside moved the district towards adding a new sophomore class each of the past two years. As a result, at the beginning of the 1988—89 school year, each school had an enrollment of about 115, including over 50 of the original class who in June, 1989 will be the first district academy graduates. Taken together, they will comprise a very significant proportion of all academy graduates statewide. The East Side Academies are now well established and are clearly among the best implemented academies programs in the state. Both district and private sector support are broad and deep. There have been very few recent changes in teaching personnel and most of the teachers in the two academies have three years of experience and a clear commitment to their students and the academics. The addition in 1987—88 of a full time “loaned executive” from Hewlett Packard to coordinate all aspects of private sector involvement is unique in the state and by all accounts is a tremendous plus for students at both schools. And as will be reviewed below, the outcome measures at both schools strongly favor academy students. With the departure of school superintendent Harry Reynolds, the high level of district support the academics have enjoyed for the past three years may be not be as automatic in future years. Reynolds was always a strong supporter, having initiated the program in the East Side District after helping found the original Peninsula Academies while superintendent in the Sequoia District. 3 4 PENINSULA ACADEMIES 1987—88 EVALUATION; The question of whether business will continue to support the loaned executive position which helped the mentor and summer jobs programs achieve such success in 1987—88 now appears to be resolved. District administrator Keith Bush, who oversees both academies, reports that the program’s business partners will come through again in future years. He is also confident that district support under new superintendent Joe Coto will continue to be strong. Coto is familiar with the academics, having supported the creation of the Health Academy at Oakland Tech and academics at three other Oakland high schools during his tenure as superintendent in that city. Bush also points to the consistent support from the East Side Board of Education, and their understanding of the success achieved by the two academies. The summer of 1988 marked the first time the two academies placed students into summer jobs. Virtually every “eligible” junior—about half of all juniors at the two schools—was placed in a job. Total summer placement was 29 juniors at Independence and 18 at Silver Creek. Most of the jobs were structured for academy students’ talents and needs, and paid well over $6 per hour. Some students were asked to stay in part-time positions in the fall, and others are likely candidates for positions when they graduate in the summer of 1989. Setting The East Side Union High School District has 11 high schools and 22,500 students, making it one of the ten largest districts in the state. Covering a large part of San Jose and parts of neighboring communities, the district is surrounded by “Silicon Valley,” the inspiration for the electronics theme of the academies. Unemployment is generally below state and national levels. The school district’s enrollment is very diverse ,as is the enrollment at the two schools. Whites are in a minority at both schools, making up fewer than 25 percent of the population and are roughly equal in numbers with Asians, the fastest growing minority. Hispanics comprise nearly 30 percent of each school and Filipinos make up 13 percent of the Independence student population and 8 percent of Silver Creek’s. Of the two schools. Silver Creek serves a lower—income population with an AFDC rate of 16 percent versus Independence’s rate of 10 percent. Their respective statewide socio- economic status percentiles are 52 and 34 (based on CAP testing), and their respective THE EAST SIDE ELECTRONICS ACADEMIES 3 5 1986—87 CAP percentile scores are 28 (reading) and 76 (math) for Independence and 28 (reading) and 74 (math) for Silver Creek. Silver Creek’s architecture is fairly typical of California’s suburban schools, with a sprawling one-story campus. Independence High is part of a larger educational and government “park” setting, which includes a planetarium, a major branch of the public libraries, impressive athletic facilities, and other amenities which can be justified for a school of its size. Its current enrollment is just over 4,000, making it one of the largest schools in the state. Silver Creek’s enrollment is about 2,400. Student Profile The two schools used similar recruiting mechanisms in the 1984—85 school year when they signed up the original 100 students. Visits were made to the various feeder junior high schools. Presentations were made in 8th grade classes. Meetings were held with parents and students to explain the program and its goals. The recruiting focused on “at risk” students—those with potential but with poor achievement and attendance records as well as those identified by teachers and counselors. But Silver Creek tended to take a lower achieving group in its original class than Independence. Similar criteria were used in adding sophomore classes the last two years. Student records and counselor/teacher recommendations were used to identify potential students. Parent approval was required for admission to the program. Recruiting has not been difficult because the two programs have generally positive standing in the respective schools. 3 6 PENINSULA ACADEMIES 1987—88 EVALUATION TABLE 1. Student Profile, Independence Academv Comparison Group Class (N) Sophomores 26 44 Juniors 72 82 Race/Ethnicity (%) White 37 23 Black 13 10 Hispanic 36 36 Asian 10 28 Other 4 3 Gender (%) Male 64 59 Female - 36 41 TABLE 2. Student Profile, Silver Creek Academy Comparison Group Class (N) Sophomores 41 44 Juniors 52 36 Race/Ethnicity (%) White 26 18 Black 25 24 Hispanic 42 43 Asian 5 9 Other 2 6 Gender (%) Male 56 64 Female 44 36 THE EAST SIDE ELECTRONICS ACADEMIES 3 7 Program Management As already mentioned, district support for the academies has been consistently strong. The district administration plays as active a role in the program’s management as any academy in the state. Keith Bush, Administrator of Career Services for the district, continues to serve as the program manager for both academies. He is supported by a clerical assistant. In February of 1987, Myra Tincher, the Hewlett Packard executive on loan, began working as business liaison in space provided by the district. Each school also has a part-time site manager. At Silver Creek, the English teacher for the program serves in this role; at Independence, it is a counselor. Both programs also have an aide serving the academy. In 1987—88, the time available to the counselor at Independence was reduced, and even he wonders whether coordination wouldn’t be better achieved by a teacher. Business support for the two schools is centralized through three committees: (l) A Board of Directors which meets once a year and includes the superintendent of schools and high—level business executives. (2) An operating committee, whose members are equally divided between school personnel and mid-level business executives and which meets monthly and is chaired by a business person. Its major responsibility is to coordinate business involvement, but general policy questions about the direction of the academics are also discussed at these meetings. (3) A curriculum advisory committee serving both schools which meets regularly to help ensure that the course offerings are consistent with the demands students will face in either further education or jobs in the electronics industry. Space arrangements at both programs are good, with an office set aside for use by the aide and teachers, and most classrooms clustered close together. At Independence, one of the largest high schools in the nation, Academy students are clustered in one of the three “villas” by which the school is broken up into smaller units, and the intimacy thus provided stands in contrast to the bigness of the school as a whole. At both schools, students study English, math, science and electronics in the academy in the morning, and take other non-academy classes the rest of the day. Some students are specially programmed, particularly in math courses, to meet their particular needs. There have been numerous field trips over the course of the three years, and many guest speakers as well. A number of academy students have been able to take training classes offered by businesses to their employees at no cost. Every student wanting a mentor was assigned one in 1987—88. Between the two schools, 93 students had mentors in 1987—88, with mentors coming from 20 different companies. As noted in the 1986—87 3 8 PENINSULA ACADEMIES 1987-88 EVALUATION: evaluation and confirmed by Myra Tincher, the mentor program still needs work to achieve the potential teachers and industry supporters envision. Finding the right mentors and providing apropriate support to them and their mentees is a constant challenge. Class size for academy classes is generally kept at 25 or fewer. In 1987—88, a new electronics teacher was hired at Silver Creek, one of the few staff changes at either school over the past three years. Both district and Silver Creek staff feel strongly that the change has been very positive for students. Silver Creek has also brought its classes closer together on campus, also with positive effects. Program Resources/Costs All resource figures listed below are for the two academies combined. District Support. An additional 2.2 full-time-equivalent teachers are employed by the academy. The total number of FTE academy teachers is 8.4. There are also two full-time aides who work with teachers, students, and parents, and assist in maintaining relationships with industry participants. Approximately .3 FTE administrative time is required for the program. This time is divided among the director of supplemental education, and the career services administrator. Eight staff days over the summer were devoted to planning. Another staff development day was devoted to coordinating site activities. Private Sector Support Employer representatives donated approximately 553 days to the program during the year. The bulk of these (279) came while serving as mentors to individual students, and 68 days were devoted to mentor coordination. Seventy-five days were spent supervising students on the job and another 75 arranging job placements, 40 attending advisory committee meetings, 15 speaking to student groups, and 10 days organizing field trips and advising on the curriculum. THE EAST SIDE ELECTRONICS ACADEMIES 3 9 The Academy made use of the following additional resources: 0 Equipment. Over $34,000 worth of equipment was donated to the academy. This included $28,404 in test equipment from Hewlett Packard, $1,685 from ROLM- MilSpec for furniture, equipment, and electronic components, $750 from F. L. Jennings for furniture, a $500 donation from William H. Shallenberger, and a $2,000 donation for Unisys. 0 Supplies. $7,000 from SB 605 funds was spent on instructional supplies to support the educational program. . Transpgnatign. $4,000 from SB 605 funds was allocated for all field trips to companies and related sites. - RewardsZSocial Events. Numerous social and reward events are planned and funded 60 percent .by the state grant and 40 percent by private sources. Student awards are presented two to three times a year at each site. Mentors and participating companies receive awards annually. Student Outcome Data The most objective data by which to judge the program’s effectiveness are measurable indicators of student performance. These include attendance, credits earned, grade point average, courses failed, and retention in school (the opposite of dropouts). While these are not the only indicators of the program’s effect on students, they provide the most statistically convincing measures. There are now data across three years by which to judge the program’s performance: that from 1985-88 for the first cohort; and from 1987—88 for the second cohort. Tables 3 and 4 present differences in outcomes between each cohort of academy students and their matched comparison groups. These differences have been adjusted to account for differences between the two groups in age, gender, race or ethnicity, and academic performance in the preceding year. In this table, positive numbers for “attendance,” “credits,” and “GPA” mean that academy students compiled better records than the comparison group. Negative numbers for “courses failed” and “dropout probability” also mean that academy students performed better than the comparison group. Only when an asterisk appears next to the value is the difference large enough to be statistically significant. 4 o PENINSULA ACADEMIES 1987-88 EVALUATION TABLE 3. Adjusted Differences Between Academy and Comparison Groups, by Class, Independence High School Atten- Courses Dropout dance Credits GPA Failed Probability Cohort entering fall 1985 (juniors): 1985—86 Outcomes —1 .1 7.4* 0.1 —1.4* i 1986—87 Outcomes 3.3* 6.9* 0.3* —1.4* i 1987—88 Outcomes 2.9# -—O. 1# 1.1# 0.1# i Cohort entering fall 1987 (sophs): 1987—88 Outcomes 1.6 10.2* 0.4* —-1.2 i # Estimated from separate regressions for academy and comparison groups i Insufficient number of dropouts for analysis * Statistically significant Of the 16 differences between academy students and comparison groups shown in this table (Table 3), 13 favor the academy students and three the comparison groups. Of those favoring the academy groups, eight are statistically significant; of those favoring the comparison students, none is. The first cohort (juniors) of academy students outperformed its comparison group in credits and courses failed its first year; and in attendance, credits, grade point average, and courses failed its second year. The second cohort of academy students (sophomores) outperformed its comparison group in credits and grade point average its first year. Of the 16 differences between academy students and comparison groups shown in the following table (Table 4), nine favor the academy students and seven the comparison groups. Of those favoring the academy groups, six are statistically significant; of those favoring the comparison students, one is. The first cohort (juniors) of academy students outperformed its comparison group in attendance and credits its third year; its comparison group outperformed the academy students in courses failed the first year. The second cohort of academy students (sophomores) outperformed its comparison group in attendance, credits, grade point average, and courses failed its first year. THE EAST SIDE ELECTRONICS ACADEMIES 4 1 TABLE 4. Adjusted Differences Between Academy and Comparison Groups, by Class, Silver Creek Atten- Courses Dropout dance Credits GPA Failed Probabilitv Cohort entering fall 1985 (juniors): 1985-86 Outcomes -l.5 —4.9 -0.2 1.7* i 1986—87 Outcomes -2.3 —2.5 0.1 0.4 i 1987—88 Outcomes 12.2* 6.4* 0.1 ~0.5 i Cohort entering fall 1987 (sophs): 1987—88 Outcomes 6.8* 10.5* 0.46* —2.0* i # Estimated from separate regressions for academy and comparison groups i Insufficient number of dropouts for analysis * Statistically significant With respect to dropouts, because the actual number (as opposed to the percent) of students who drop out is relatively small when analyzed by individual cohort, few statistical tests of differences between academy and comparison groups will prove to be significant, even when differences in the attrition between the two groups is substantial. For this reason, the following table presents the actual differences between the two groups. The overall “attrition” (what is usually meant when the term “dropout” is used) is broken out into its three components: known school dropouts (official withdrawals), transfers to other high schools, and probable dropouts (students lacking records as of June 1988 who neither officially withdrew nor transferred). Table 5 presents the figures by cohort. For the first cohort, juniors in 1987—88, the figures represent the cumulative rates across the three years of the program; for the second cohort, sophomores, the rates for just the last year. Figures for both Independence and Silver Creek are included. 4 2 PENINSULA ACADEMIES 1987—88 EVALUATION TABLE 5. Known Dropouts, Probable Dropouts, and Transfers, by Cohort, for Academy and Comparison Group Students Academv Comparison Group Known Prob. Trans- Known Prob. Trans- Drops Drons fe rs Drops Drops fers Cohort 1 Independence 3% 0 16% 5% 0 21% Silver Creek 6% O 37% 7% 2% 30% Cohort 2 Independence O O 4% 2% O 2% Silver Creek 0 O 3% 0 O 5% This table suggests that the academies have had a slight influence in reducing dropouts, but mixed results in reducing transfers from one school to another. Student Questionnaires Academy students complete questionnaires at the point they enter the program, at the beginning of their sophomore year, and again at the end of each school year. Some of the questions—on career-related plans, and attitudes related to career, school, and self—appear in parallel form on the “pre” and “post” version of the questionnaires, allowing assessment of changes over the course of the program. Another set of questions appears only on the “post” version, and asks for feedback on various program activities, and ratings of certain program components. The sections below present feedback from these questionnaires. Responses have been averaged across the two cohorts to simplify their presentation, except where clear differences appear between cohorts. THE EAST SIDE ELECTRONICS ACADEMIES 4 3 W. Seventy-seven students are represented in these data: 54 juniors and 23 sophomores. W Average number of classes in the academy was 3.5. Three- fourths of juniors had mentors. Two juniors reported having jobs. There were 15 outside speakers during the year. Students went on an average of 6.5 field trips. Program Ratings. Ninety percent of students rated the equipment highly, and 80 percent the materials. One-third of participants liked the academy schedule. Two-thirds saw a strong connection between what they were studying in school and their post-graduate plans. And two-thirds also liked the academy better than their regular high school program. Career-Related Plans. Forty percent of participants reported having a definite post- graduate plan; of these 90 percent were planning on college (up 10 percent since entering the academy), and 35 percent on a job (no change). Two~thirds reported having a long-term career goal (also up 10 percent). Attitudes Related to Career School and Self. Eighty percent of academy students view it as important to make career plans and gain job skills while in high school. Half found their Classwork interesting during the last year, a sharp rise from their pre-program responses. Three-fourths “feel good about themselves.” Silvgr gzrggk. Sixty students are represented in these data, 20 juniors and 40 sophomores. Procram Activities. Average number of classes in the academy was three for juniors and four for sophomores. All juniors had mentors; 5 percent of sophomores did so. No juniors reported having jobs; two sophomores did. There were seven outside speakers during the year. Juniors went on six field trips, sophomores on four. Program Ratings. Seventy percent of students rated the equipment highly. All juniors rated the materials highly; 53 percent of sophomores did so. Half the participants liked the academy schedule. Three-fourths of the juniors saw a strong connection between what they were studying in school and their post- graduate plans; half of sophomores did so. Three-fourths of participants liked the academy better than their regular high school program. 4 4 PENINSULA ACADEMIES 1987-88 EVALUATION: ° Career-Related Plans. Forty-seven percent of juniors reported having a definite post-graduate plan (up 21 percent since entering the program); 23 percent of sophomores did so (no change). For both classes, of those with plans, three- fourths were aiming for college (up 18 percent since entering the academy for juniors, and 7 percent for sophomores), and 45 percent on a job (no change). Eighty-seven percent of juniors reported having a long-term career goal (up 47 percent); 45 percent of sophomores did so. 0 Attitudes Related to glareer, School, and Self. Seventy-seven percent of academy juniors view it as important to make career plans (up 18 percent since entering the program); 63 percent of sophomores do so (down 21 percent). Three-fourths of both groups see gaining job skills while in high school as important (down 15 percent from their program entry). Forty percent found their classwork interesting during the last year (up slightly from their pre-program responses). Eighty-eight percent of juniors “feel good about themselves” (up 23 percent); 66 percent of sophomores do so (down 6 percent). Responses from Students Who Dropped Out of the Program Eight students who dropped from the program completed questionnaires. Their responses included the following: ' Reasons for leaving were “lack of control over my classes,” “teachers expect too little,” “no interest in electronics,” “didn’t get along with one of my teachers,” and “want to experience the real school and meet more people and teachers.” 0 Students liked “working in cooperation with the companies.” “field trips,” “good math program,” and “how well the program was run.” 0 Suggestions for changes to the program included “teachers’ attitudes,” “getting together with other academies so that you could transfer out of a class like regular high school.” THE EAST SIDE ELECTRONICS ACADEMIES 4 5 School Personnel Responses, Independence High School At the end of the school year, six staff completed questionnaires, providing responses to the program. These included one administrator, four teachers, and one counselor. Responses from these questionnaires: 0 Interest in school and learning was seen to have increased in 69 percent of the students, decreased in 6 percent of the students, with no change seen in 25 percent. - Parents of students were rated “very supportive and interested.” . Facilities, equipment, and learning materials in the academy were all rated “excellent”. - Non-academy teachers were rated “somewhat” to “slightly supportive,” while counselors and administrators were rated “very” to “slightly supportive.” ° Cooperation and interaction between the private sector and the academy was rated “very” to “somewhat effective.” - On a seven-point scale (1 = very weak; 7 = very strong), the academy received an average rating of 6.6 “in concept” and 4.5 “as implemented.” - The greatest weaknesses of the program were seen to be teacher commitment, inconsistency in behavior standards, lack of block of time to visit industry, students having the same teachers three or four years, students unmotivated, getting the right teachers, and cooperation between subjects. . The greatest program strengths were seen to be the improvement in attendance and self-esteem, industry involvement, close monitoring of students, teachers working together, excellent equipment, and more administrative support. 0 When asked if satisfied with the procedures and criteria for selecting and dropping students from the program, the staff gave equal “yes” and “no” responses. 0 When rating the counseling program, three staff rated it “fair,” two “good,” and one “excellent.” - Quality of the curriculum in the technical class received four “poor” ratings, one “excellent,” and one “average.” Quality of curriculum in the academic class 4 6 PENINSULA ACADEMIES 1987-88 EVALUATION. received four “average,” one “good,” and one “excellent.” Suggestions were made to update and implement electronics, and to include more grammar, writing, reading and verbal skills, and practical mathematics. School Personnel Responses, Silver Creek High School At the end of the year seven staff including one clerk, one counselor, one administrator, and four teachers completed a questionnaire. Their responses included: 0 Interest in school and learning was seen to have increased in 87 percent of the students, decreased in 2 percent of the students and no change was seen in 13 percent of the students. 0 Self-esteem was perceived to have increased in 89 percent of the students and no change was seen for 11 percent. 0 Parents were reported to be “very supportive and interested” in the program. Teachers, counselors, and administrators at the school were also rated “very supportive.” ° Counselling for academy students was rated “excellent.” - Orientation/staff inservice for the new academy teachers received six “excellent” ratings and one “poor” rating. - Facilities were rated “excellent,” with suggestions for “clustering classes for group lectures/business meetings,” “being in one building,” and “having a phoneline for students to call mentors.” Equipment was rated “excellent.” Books and learning materials also received “excellent” ratings. - Quality of the curriculum in the technical class was rated “excellent” to “good.” Quality of the curriculum in the academic class received four “excellent” ratings and two “good” ratings with a suggestion for more “cross-curriculum activities.” 0 Staff were satisfied with the criteria and procedures for selecting and dropping students. 0 On a seven-point scale (1 = very weak; 7 = very strong), the academy received an average rating of 6.9 “in concept” and 6.4 “as implemented.” THE EAST SIDE ELECTRONICS ACADEMIES 4 7 . The greatest weaknesses of the program were seen to be “behavior problems”, “not being able to select a matched comparison group for evaluation,” “need for a social worker,” “permanent liaison,” and “social and emotional problems of students.” 0 The greatest strengths of the program were seen as “staff, administration and liaison,” “teamwor , teacher/student relationship, small class,” “caring/giving/dedicated teachers,” “cohesive student body,” and “outstanding program.” 9’ ‘6 ’9 ‘6 Parent Responses, Independence High At the end of the year, four parents completed questionnaires about the program. Their responses included: 0 The overall quality of the program was rated “excellent” by three and “good” by one parent. ° Best features of the program being small classes, electronics, guidance, recognition for work well done, industry field trips, and individual attention. The weakness noted was “not enough homework and reports.” A suggestion was made to have regular progress reports. 0 All four stated that their children showed more interest in school and learning and were making good progress in planning and preparing for what to do after graduating from high school. 0 Parents have been involved in the program by attending open house, helping children with homework, meeting with teachers and counselors, and helping with the newsletter. 0 The academy staff keeps the parents “very” to “moderately” well informed on their child’s performance in school and “very well” informed on the program’s special activities. 4 8 PENINSULA ACADEMIES 1987-88 EVALUATION Parent Responses, Silver Creek At the end of the year, five parents completed questionnaires about the program. Their responses included: - The overall quality of the program received three “excellent” ratings and one “good” rating. - The best features of the program were seen as guidance, curriculum, frequent communications, positive reinforcement for student efforts, field trips, and progress reports. The weakness noted was that some students continued to have a negative attitude. Some suggestions for improvement of the program were group electronics project, more publicity in the business community for summer jobs, and progress reports. 0 Four parents felt that their children were “much more interested” in school and learning while one parent felt her child “a little more interested.” All felt their children were making good progress in planning and preparing for what to do after graduation. 0 Parents have been involved in the program by attending open house, helping children with homework, and meeting with teachers and counselors. Private Sector Representatives Responses Six private sector representatives completed a questionnaire at the end of the school year. Their responses included the following: - The companies were involved in the program by providing classes on hardware repair and equipment operation, software, field trips, factory tours, summer jobs, mentors, and donations of equipment. The representatives were involved as mentor coordinator, curriculum advisor, and member of Board of Directors. 0 Program curricula was seen to provide students with the skills needed by local private business. THE EAST SIDE ELECTRONICS ACADEMIES 4 9 ° The effectiveness of the Steering Committee and quality of communications between academy staff and private sector supporters were rated “excellent” to 6‘good‘,’ - The greatest weaknesses of the program were seen to be inadequate teacher involvement, the lack of broad—based industry support, and inadequate small to medium company support. - The greatest strengths noted were the dedication of teachers, the personal attention received by students, the strength of industry participation, mentors, staff support and help, and the students themselves. 0 On a seven point scale (1 = very weak; 7 = very strong), the academy was rated 6.6 “in concept,” and 6.0 “as implemented.” 0 Suggestions were made to link all academy programs in the area to leverage media and industry support, provide communication between groups where activities overlap, and maintain industry momentum through a long-term PR program. Mentor Responses Twenty-two mentors completed questionnaires at the end of the year. Their responses included the following: - When asked if the academy program is serving the appropriate target population of students, thirteen responded “yes,” two “no,” and seven “not sure.” Cements included: “low-income families need the extra guidance for their child to succeed” and “the extra structure helps them focus on life after high school.” 0 Asked if they believed the program provided students with job skills needed by companies in the area, fifteen responded “yes,” one “no,” and five “don’t know.” 0 Most successful mentor/mentee activities were job fair, communication session, opening of new TV studio, company tour, work on career packet, and visiting college campuses. - Twenty mentors had visited their mentees at school and eighteen had had mentees visit them at their places of work. 5 O PENINSULA ACADEMIES 1987—88 EVALUATION ' The mentor program was rated “highly useful” by seventeen mentors and “somewhat useful” by three. The effectiveness of planning and management of the program received ratings of “excellent” to “average.” 0 The greatest weaknesses of the program were seen as “no follow-up on student/mentors,” “mentors don’t know if they are being useful,” and “communication with teachers.” 0 The greatest strengths of the program were seen as “good teachers,” “follow- through on commitments,” “mentor-mentee bond,” “interaction with industry,” “class structure,” and “volunteers.” ° Suggestions for improving the academy were “graduates meeting with current students to pass on success stories,” “newsletters to mentors,” “mentor/student/teacher meetings,” and “more publicity on the program.” SUMMARY In last year’s report, Silver Creek presented a dilemma. While the program appeared to be as well implemented as that at Independence and thus as well implemented as any academy in the state, the student outcome measures did not show positive results. In the 1987—88 school year, Silver Creek’s outcome measures indicate a remarkable tumabout. At Silver Creek, all eight measures for the 1987—88 school year are positive, and six of the eight are statistically significant. All four measures for the sophomore class are positive and statistically significant. Silver Creek’s record for 1987—88 is thus as positive as any academy in the state. Looking again only at 1987—88 outcome data, Independence’s record is also positive. Six of the eight measures favor academy students. Two of those favoring academy students are statistically significant—both credits and GPA for the new sophomore class. The two negative figures—both for the original cohort of students—are not significant. Though these numbers are positive, they are less so than in the previous year when the original cohort outperformed the comparison group on every measure, and each was statistically significant. The very positive outcome measure performance of Silver Creek was confirmed both by interviews with staff and in staff questionnaires. In their questionnaire responses, THE EAST SIDE ELECTRONICS ACADEMIES 5 1 program teachers were pleased with the program and its impact on students and aware of many improvements that have been made in the program. At Independence, there appear to be unresolved problems. Although the electronics curriculum used in the laboratory class is the same as at Silver Creek, some staff members at Independence, including the electronics teacher, feel that a broader technology approach might be more effective than the current narrower focus on electronics. Coordination with business involvement may also need improvement at Independence. As mentioned earlier, the counselor who is the nominal leader of the academy, has less time available to play this role, and the program may be suffering accordingly. Apparently, some action to correct this has already been taken. Still, overall the Eastside Union High School District can be proud of its two academies. Both are well implemented. District support is very strong. The level of business involvement is as great as at any academy in the state. And over the three years of the program, academy students have outperformed comparison students on most outcome measures. , gym“ The Peninsula Academies Menlo-Atherton High School Sequoia High School Sequoia Union High School District Redwood City, California The Peninsula Academies were in their seventh year of operation during the 1987-88 school year. Begun in the fall of 1981, they provide the model for the other academies funded under the state replication legislation. They are operated jointly by the Sequoia Union High School District, and will be included together in this one case study. They were evaluated through December of 1986 by the American Institutes for Research in Palo Alto, at which time that evaluation ended. Given their importance to the academies in California, and at their request, we agreed to include them in this evaluation for the 1987—88 school year. Both the Menlo—Atherton Computer Academy and Sequoia Electronics Academy are mature programs that follow the full academy model, and indeed helped to define it. They have both enjoyed stable teaching staffs for many years. They are administered through the district Special Projects Director. In the fall of 1987 the lead teacher in the Sequoia Electronics Academy was appointed coordinator for the programs, working under the Special Projects Director; she was given an additional release period for this role. This was another successful year for these academies. The steering committee, which had not been active the previous two years, was re-established and became quite active, with several new corporate members. Both schools continued the established patterns of speakers, field trips, and other motivational activities. All juniors had mentors, as did many seniors and even a few sophomores. All eligible juniors had summer jobs, and many seniors had part-time after-school jobs. Parental contacts were frequent. Interest among freshmen was high, with over a hundred applicants for each academy. As strong as these programs appear, even they exhibit some problems. One concern was a feeling expressed by a number of teachers in both schools of friction between the district and teachers. A difficult year of contract negotiations (now resolved) no doubt contributed to this sense. Some teachers questioned the degree of district support for the academies, at least in terms of concrete day-to-day help. There is also some tension between the two academies, partly over competition for district support. The coordinator role, which went to the Sequoia lead teacher, was questioned by the Menlo-Atherton staff 53 5 4 PENINSULA ACADEMIES 1987—88 EVALUATION as serving their needs equally. A pattern seems to be developing of the two academies operating as separate programs. Setting The Sequoia Union High School District is located on the peninsula south of San Francisco, just north of Palo Alto. This is a largely urbanized area, although the district serves a bipolar distribution of students, who come from some of the wealthiest communities in California (such as Woodside and Portola Valley) and some of the poorest (including East Palo Alto). Menlo-Atherton High School’s 1,700 students are 8 percent Asian, 16 percent black, 14 percent Hispanic, and 62 percent white. Sequoia High School’s 1,700 students are 8 percent Asian, 5 percent black, 32 percent Hispanic, and 55 percent white. Their respective statewide socio-econornic status percentiles are 93 and 60 (based on CAP testing), and their respective 1986—87 CAP percentile scores are 86 (reading) and 90 (math) for Menlo-Atherton, and 87 (reading) and 99 (math) for Sequoia. Student Profile The Peninsula Academies use a well-defined and fairly elaborate system for selecting students. This begins with brief announcements in ninth grade classes about the program, followed by more in-depth presentations for those students interested. Teacher and counselor recommendations also contribute to the pool of potential participants. Applications are then completed by interested students. Applicants’ academic backgrounds are checked to see if they fit the program’s intended student profile, namely underperforming students with unmet potential. Once a reasonably sized group of potential participants is reached, teachers interview these students to determine their level of interest and appropriateness. Then parents are asked to attend an evening meeting to learn about the program, and sign an agreement to support their child through the program, if they agree it is well suited to them. Final selections are then made, and students are notified of their selection. The tables below provide a profile of the 1987—88 academy classes, and their matched comparison groups, in the two schools. THE PENINSULA ACADEMIES 5 5 TABLE 1. Academy Student Profile, Menlo Atherton Academv Comparison Group Class (N) Sophomores 35 48 Juniors 30 49 Seniors 39 45 Race/Ethnicity (%) White 19 21 Black 45 48 Hispanic 27 25 Asian 5 4 Other , 4 2 Gender (%) Male 56 49 Female 44 51 5 5 PENINSULA ACADEMIES 1987—88 EVALUATIOlN TABLE 2. Academy Student Profile, Sequoia Academy Comparison Group Class (N) Sophomores 43 48 Juniors 37 56 Seniors 44 28 Race/Ethnicity (%) White 46 46 Black 11 10 Hispanic 39 42 Asian 3 1 Other 1 1 Gender (%) Male 54 63 Female 46 37 Program Management The academies are administered under the office of the Special Projects Director. He in turn appointed the Sequoia Electronics Academy lead teacher as the program coordinator, giving her an additional released period for this role (one is allocated for being a teacher in the academy, and one for the role of department chair). He then handled interactions with other district staff, while she handled most contacts with teachers and private sector supporters. The lead teacher and computer lab teacher at Menlo-Atherton continued to handle most of the private sector contacts for that academy. Both academies followed the school-within—a—school model, with techncial/lab classes in grades 10—12, and three academic classes at grades 10—11. These classes are in English, math, and science. At Sequoia, the seniors attend the “Lenkurt Lab” for their technical class, a high-prestige facility formerly available primarily to high-achieving students. A new English teacher began this year for the sophomores, to provide the lead teacher with THE PENINSULA ACADEMIES 5 7 the additional release time. He also began one section of senior English, and plans to provide this for all seniors next year. At Menlo-Atherton, students take computer lab for three years, along with English and math at grades 10 and 11. The full-time lab teacher handles grades 11 and 12, while a part- time computer teacher handles grade 10. While the academy Students also take science, this is in mixed classes with non-academy students because of the students’ varying ability levels and space limitations for teaching science. The lab, English, and math teachers’ schedules are fully dedicated to the academy at both schools. At Sequoia this is also true in science, while at Menlo-Atherton the science teacher, who is also the lead teacher and very involved in other ways, includes non-academy students in his classes to meet class size requirements. Both programs enjoy the services of a full—time aide. At both schools teachers have an extra preparation period daily, and meet weekly to plan program activities and discuss methods for dealing with problem students. Teachers also have occasional staff development days or activities. Telephone contacts and meetings with parents are frequent, usually resulting in contracts among students, parents, and teachers designed to correct student behavior problems. Monthly evaluations of all academy students are prepared and sent home, modeled on employee evaluations. They include ratings for attendance/punctuality, attitude, behavior, and performance in each academy class. Both programs bring in speakers to talk to students, often identified through the mentor program. They also take students on approximately one field trip per quarter. This year these trips included the tide pools, Hewlett Packard, Heald College, San Jose State University, the College of San Mateo, the Lawrence Hall of Sciences, and an ACT play. The mentor program was active this year; all juniors had a mentor, as did many seniors and a few sophomores. Companies supplying mentors provided strong support, organizing common activities such as shadowing of students at school, day-long visits to the company, practice in job interviewing, a self-esteem workshop, monthly luncheons, and attendance at sports events. Job placements continued as usual, although teachers began to feel competition with the other academics in this respect, with Lockheed’s jobs divided among the five academies on the peninsula. One response to this was to form a “Bay Area” coalition among these academies, which proved helpful in approaching companies and coordinating efforts. A higher proportion of juniors were required to attend summer school this year, due in part to the increasing requirements for graduation in California, somewhat reducing the number of placements. However, about two-thirds of juniors had summer jobs—all who qualified. Almost all were in computer or electronics companies, although not all the jobs themselves 5 g PENINSULA ACADEMIES 1987—88 EVALUATIor were technical. Most seniors continued to have part-time school-year jobs, often extensions of their summer work, although time conflicts with course requirements caused increasing problems in this respect also. Program Resources/Costs Menlo-Atherton Computer Academy District Support. The four full-time-equivalent academy teachers represent an increase of 1.4 teachers over what would be required without the smaller class sizes and extra preparation period in the academy. In addition, the department chair is given two hours additional compensation above his teaching load for administrative duties, and an academy aide is employed six hours/day. Additional informal administrative support is supplied by the principal and instructional vice-principal. Teachers spent a total of seven days over the summer in reimbursed staff development activities, planning for the next year. Private Sector Suppprt. Local employers’ representatives contributed a total of 139 full-day equivalents to the academy: 24 for committee meetings, 5 speaking to groups of students, 5 organizing field trips, 80 serving as mentors, 10 arranging job placements, and 15 supervising students on the job. Other categories of support, which came variously from state, district, and private sector support: - Facilities. A 20 x 40-foot classroom and adjoining patio were prepared by the school for use as the academy computer center. - Equipment. Industry originally donated 10 PIP-85 and four PIP-86 computers. District funds have been used to purchase two IBM, one Macintosh, and five Apple 11 computers. - Supplies. About $1,500 of state funds are used each year for instructional supplies. ' Transportation. The district contributes approximately $900 per year in transportation funds. THE PENINSULA ACADEMIES 5 9 . Rewardslsppial events. Students raised about $1,500 for awards and prizes, and various social events. Sequoia Electronics Academy District Support. The five full-time-equivalent academy teachers represent an increase of 2.2 additional full-time-equivalent teachers over what would be required without the smaller class sizes and extra released period. The program also employs one full-time aide, and releases the department chair from an additional class for administrative tasks. The school administrators provide strong support to the program, although it is hard to precisely enumerate this. Academy staff were involved in approximately 70 staff development days during the 1987—88 school year. Most of this was spent working with the San Jose Eastside staff over the summer, helping to develop curriculum there. It also involved planning for the subsequent year at Sequoia, attending workshops and conferences, and making presentations. Private Sector Suppgtt. An estimated 363 days of time were contributed by the representatives of local employers: 20 for advisory committee meetings, 8 for speaking to groups of students, 25 organizing or leading field trips, approximately 200 serving as mentors, 10 arranging job placements, and approximately 100 supervising students on the job. Other categories of support, which were contributed to by a mixture of state, district, and private sector support: . Facilities. The district continued to improve several classrooms and the electronics lab. Industry contributed the use of 20 conference rooms for committee meetings, field trips, workshops, and the graduation ceremony. . Equipment. Various kinds of equipment were either purchased by the district or donated by industry: ocsilloscopes, meters, power supplies, service lab equipment, and four Apple II computers. 0 Supplies. A total of approximately $3,500 was spent on electronics kits, science lab supplies, and other miscellaneous supplies; much of this came from the state grant. . Transportation. A total of $1,500 was spent on 10 field trips. 6 o PENINSULA ACADEMIES 1987-88 EVALUATIOP ' 3W. Students raised $2,500 for mentor and student social events, and student-of-the-month T-shirts. Private sector and teacher donations contributed another $2,550 for graduation awards and other prizes. Student Outcome Data The most objective data by which to judge the program’s effectiveness are measurable indicators of student performance. These include attendance, credits earned, grade point average, courses failed, and retention in school (the opposite of dropouts). While these are not the only indicators of the program’s effect on students, they provide the most statistically convincing measures. Tables 3 and 4 present differences in outcomes between each cohort of academy students and their matched comparison groups. These differences have been adjusted to account for differences between the two groups in age, gender, race or ethnicity, and academic performance in the preceding year. In this table, positive numbers for “attendance,” “credits,” and “GPA” mean that academy students compiled better records than the comparison group. Negative numbers for “courses failed” and “dropout probability” also mean that academy students performed better than the comparison group. Only when an asterisk appears next to the value is the difference large enough to be statistically significant. THE PENINSULA ACADEMIES 6 1 TABLE 3. Adjusted Differences Between Academy and Comparison Groups, by Class, Menlo-Atherton Computer Academy Atten- Courses Dropout dance Credits GPA Failed Probability Cohort entering fall 1985 (seniors): 1987—88 Outcomes 2.7# 3.2 —O.3 0.6 —O.13 Cohort entering fall 1986 (juniors): 1987-88 Outcomes 0.9 5.0 0.1 0.2 i Cohort entering fall 1987 (sophs): 1987—88 Outcomes 0.9 7.1* 0.1 —0.3# i # Estimated from separate regressions for academy and comparison groups i Insufficient number of dropouts for analysis * Statistically significant All of the data in Table 3 above are from the 1987—88 school year. Of the 13 differences shown between academy and comparison groups, 10 favor the academy students and three the comparison group. The third cohort (sophomores) of academy students outperformed its comparison group on credits its first year. 6 2 PENINSULA ACADEMIES 1987—88 EVALUATION: TABLE 4. Adjusted Differences Between Academy and Comparison Groups, by Class, Sequoia Electronics Academy Atten— Courses Dropout dance Credits GPA Failed Probabilitv Cohort entering fall 1985 (seniors): 1987—88 Outcomes 1.8# 2.1# 0.2# -0.5# -0.24 Cohort entering fall 1986 (juniors): 1987—88 Outcomes 3.1 10.2* 0.2 -0.5# -0.05 Cohort entering fall 1987 (sophs): 1987-88 Outcomes 0.3 5.5 0.04 -O.1 i # Estimated from separate regressions for academy and comparison groups i Insufficient number of dropouts for analysis * Statistically significant Data in Table 4 above are all from the 1987—88 school year. All 14 of the differences shown in this table favor academy students over their matched comparison group counterparts. One of these differences is statisitically significant, that for the second cohort (juniors), for credits, during their second year. With respect to dropouts, because the actual number (as opposed to the percent) of students who drop out is relatively small when analyzed by individual cohort, few statistical tests of differences between academy and comparison groups will prove to be significant, even when differences in the attrition between the two groups is substantial. For this reason, the following table presents the actual differences between the two groups. The overall “attrition” (what is usually meant when the term “dropout” is erroneously used) is broken out into its three components: known school dropouts (official withdrawals), transfers to other high schools, and probable dropouts (students lacking records as of June 1988 who neither officially withdrew nor transferred). Table 5 presents the figures by cohort. For the first cohort, seniors in 1987—88, the figures represent the THE PENINSULA ACADEMIES 6 3 cumulative rates across the three years of the program; for the second cohort, juniors, the cumulative rates over two years; and for the third cohort, sophomores, the rates for just the last year. Figures for both Menlo-Atherton and Sequoia are included. TABLE 5. Known Dropouts, Probable Dropouts, and Transfers, by Cohort, for Academy and Comparison Group Students Aca dem v r' n r Known Prob. Trans- Known Prob. Trans- Drons Droos fers Drons Drops fers Cohort 1 Menlo-Atherton 1 1% O 19% 1 1% 0 52% Sequoia 14% 0 7% 21% O 4% Cohort 2 Menlo-Atherton O 0 7 % 7% 0 9% Sequoia 8% 0 5% 16% 0 15% Cohort 3 Menlo-Atherton 0 0 0 O 0 0 Sequoia 3% O 5% 4% 2% 8% These figures suggest that the academies are having a fairly significant effect on reducing attrition, particularly with respect to transfers to other high schools. Student Questionnaires Academy sophomores completed questionnaires at the point they entered the program (since this was the first year this evaluation had been conducted here, this is not true for juniors and seniors). All academy students completed a questionnaire at the end of the school year. Some of the questions—on career-related plans, and attitudes related to career, school, and self—appear in parallel form on the “pre” and “post” version of the questionnaires, allowing assessment of changes over the course of time. Another set of questions appears only on the “post” version, and asks for feedback on various program activities, and ratings of certain program components. 64 PENINSULA ACADEMIES 1987—88 EVALUATION: The sections below present feedback from these questionnaires. Responses have been averaged across the three cohorts to simplify their presentation, except where clear differences appear between cohorts. It should be emphasized that the data in this section reflect student perceptions. Menlo-Atherton Sixty-seven students are represented in these data: 23 seniors, 20 juniors, and 25 sophomores. Prgggam Activities. Average number of classes in the academy was three for sophomores and juniors, one for seniors. Ninety-five percent of juniors had a mentor, as did 35 percent of seniors; no sophomores did so. Three-fourths of seniors reported having jobs, as did one-fourth of juniors. There were an average of three outside speakers during the year. Juniors and seniors went on four field trips, sophomores on two. figgram Ratings. Eighty percent of sophomores and juniors rated the equipment highly; 60 percent of seniors did so. Seventy-five percent of sophomores rated materials highly; 60 percent of juniors and 30 percent of seniors did so, . Fifty- seven percent of sophomores liked the academy schedule; 40 percent of juniors and seniors did so. Eighty percent of sophomores and juniors saw a strong connection between what they were studying in school and their post— graduate plans; 60 percent of seniors did so. Ninety percent of sophomores liked the academy better than their regular high school program, as did 75 percent of juniors and 65 percent of seniors. Career-Related Plans. Forty percent of sophomores reported having a definite post- graduate plan, as did 50 percent of juniors and 73 percent of seniors. Of these, 85 percent were planning on college and 45 percent on a job (up 31 percent over the year for sophomores). Slightly over half reported having a long—term career goal. Attitudes Related to Career. School. and Self. Eighty percent of academy students view it as important to make career plans and gain job skills while in high school. Two-thirds found their classwork interesting during the last year (a rise of 24 percent for sophomores from their pre-program responses). Eighty percent “feel good about themselves” (up 10 percent for sophomores). THE PENINSULA ACADEMIES 6 5 Sequoia. Ninety-one students are represented in these data: 27 seniors, 30 juniors, and 34 sophomores. O Prggam Activities. Average number of classes in the academy was three for sophomores and juniors, two for seniors. Ninety-seven percent of juniors had a mentor, as did 59 percent of seniors and 21 percent of sophomores. Sixty percent of seniors reported having jobs, as did 7 percent of juniors. There were four outside speakers for seniors, nine for juniors, and eight for sophomores during the year. Seniors went on two field trips, juniors on nine, and sophomores on seven. Progr_am Ratings. Eighty-five percent of sophomores and juniors rated the equipment and materials highly; 50 percent of seniors did so. Seventy-five percent of sophomores rated materials highly; 60 percent of juniors did so, and 30 percent of seniors. Half the sophomores liked the academy schedule; 45 percent of juniors did so, and 19 percent of seniors. Three-fourths of all three classes saw a strong connection between what they were studying in school and their post— graduate plans. Eighty percent of sophomores liked the academy better than their regular high school program, as did 85 percent of juniors and 65 percent of seniors. areer-R late Plans. A quarter of sophomores reported having a definite post- graduate plan, as did 50 percent of juniors and 70 percent of seniors. Of these, 60 percent of sophomores were planning on college (down 14 percent since their pre-program responses), as were 90 percent of juniors and seniors. Forty percent of sophomores were planning on a job (up 14 percent over the year), as were 55 percent of juniors and 70 percent of seniors. Sixty percent of sophomores (up 23 percent reported having a long-term career goal, as did 75 percent of juniors and seniors. Attitudes Related to Career. School. and Self. Eighty-five percent of academy students view it as important to make career plans and gain job skills while in high school (up 24 percent over the year for sophomores). Slightly over half found their classwork interesting during the last year (a rise of 14 percent for sophomores). Eighty-six percent of sophomores “feel good about themselves” (up 17 percent), as do 79 percent of juniors and 67 percent of seniors. 6 6 PENINSULA ACADEMIES 1987-88 EVALUATION School Personnel Responses Six school staff from Menlo-Atherton completed a questionnaire at the end of the school year, as did five from Sequoia. These were all teachers, except one aide; at both schools they included the program coordinator. ° At Menlo—Atherton, the equipment was rated “average,” the facilities and instructional materials “good,” and the curriculum “excellent.” At Sequoia, the equipment and facilities were rated “average,” and the instructional materials and curriculum “good.” 0 At Menlo-Atherton, non-academy teachers and administrators were rated as “very supportive,” while counselors were rated “somewhat supportive.” At Sequoia, non-academy teachers and counselors were rated “somewhat supportive,” while administrators were rated “very supportive.” 0 Program planning at Menlo-Atherton was rated “good,” while at Sequoia it was rated “excellent.” 0 Cooperation and support from the private sector was rated “very effective” at both schools, with very positive comments about mentors, job supervisors, and the steering committee. 0 Virtually all other aspects of the program questioned were seen as strong at both schools: student selection, motivational activities, individual student attention, counseling, and parental support. - On a seven point scale (1 = very weak, 7 = very strong), at Menlo—Atherton the academy received an average rating of 6.7 “in concept” and 6.2 “as implemented.” At Sequoia the ratings were 7.0 “in concept,” and 6.6 “as implemented.” 0 The greatest weakness at Menlo-Atherton was the need for more and better computers; at Sequoia “the budget” was listed as the main weakness. Both schools stressed the need for stronger district support. 0 Both schools saw the greatest program strength to be the quality of the teachers involved in the program and their sense of teamwork and leadership. THE PENINSULA ACADEMIES 6 7 Parent Responses Two parents from Menlo-Atherton and three from Sequoia completed questonnaires at the end of the school year. Their responses included the following: Three rated the program as “excellent,” two as “good.” Two saw their children as “much more interested in school and learning” since joining the academy, and two as “a little more.” Parents had been involved with the program in several ways: attending school events, helping their children with homework, meeting with teachers and counselors, and helping with a program activity. Asked how well they had been kept informed of their children’s progress in school, two said “very well” and three “moderately well.” The features of the program parents liked best included the teachers’ care, support, and training, and the small class size. Suggestions offered for improvement were to establish links to local colleges and technical schools, to have more clearly defined student goals and related rewards, to include social studies in the program, to not limit the program to low achievers, and to have more parental involvement. Private Sector Representatives Responses Three private sector representatives completed questionnaires at the end of the school year. These were all steering committee members, and so responses pertain to both academies. Asked whether the academy’s curriculum would prepare students for jobs locally, all three responded “yes”; two emphasized the need to maintain currency in the future. When asked to rate students as employees, only one individual responded. Ratings by this person on all characteristics listed were either “high” or “very high.” Asked about communications with the program, all three rated these as “excellent.” 6 8 PENINSULA ACADEMIES 1987—88 EVALUATIONsi o The effectiveness of the steering committee was rated as “good”; one commented that it was “growing in its effectiveness.” ° On a seven-point scale (1 =v ery weak, 7 = very strong), the academy received an average rating of 7 “in concept,” and 6 “as implemented.” 0 Program weaknesses included the need for more state support for replications, the need to keep up with state-of-the-art technology, and the need to coordinate industry requests through one person, even across academies. ° The program’s greatest strengths were seen to be its leadership and teachers, and the successful graduates. Mentor Responses, Twenty-one mentor questionnaires were received, nine from Menlo-Atherton and twelve from Sequoia. Their responses showed that: ° On average, during the school year mentors got together with their proteges eight times in person, and talked by telephone ten times. - Twenty of the 21 mentors visited their student at school; 14 brought the student to their place of work. o Asked whether the academy’s curriculum would prepare students for jobs in local companies, 11 said “yes,” and 10 said “don’t know.” - Asked to rate the mentor program, 13 responded “highly useful,” six “somewhat useful,” and two ‘0.”K - The program’s biggest problems were seen to be the need for more clarity in the mentor’s role, more and better organized mentor activities, the need for a serious commitment on the mentor’s part, student motivation, and parental support. . The greatest strengths of the program were seen to be the real-world experience given students, the personal attention and caring students experienced, the quality of the program’s staff, and student comraderie. THE PENINSULA ACADEMIES 6 9 Summary The precise student outcomes data for 1987—88 present two pictures. From a hard statistical standpoint, few of the tests of differences between academy students and their matched comparison groups reached significance. On the other hand, almost all of these differences were in a direction favoring academy student performance. Thus while the outcome evidence is not clear-cut in a hard statistical sense, it is very suggestive of positive program effect. Combined with the largely positive student attitudinal feedback, and the similar positive feedback from teachers, parents, mentors, and private sector representatives, the evidence from the 1987—88 school year must be viewed as positive. It should also be mentioned that these programs were selected by the National Alliance of Business for its “Distinguished Performance Award” among youth programs in the country. The Peninsula Academies continue to be well-run and effective programs, which serve usefully as the models for the growing number of academy replications in California. They continue to implement the full academy model, at least as well as any of the replications. They enjoy effective and stable teaching staffs—they are unusually rich in this respect. They also enjoy unusually strong private sector support. Their veteran status not only contributes to their own credibility, but provides a resource to others, and they accommodate many visitors from around the state and nation each year. They have become a force that extends far beyond the bounds of the Sequoia Union High School District. «(a “1;“ :6 fl'uful ,9”) 1 1“ mm? ”1% Computech Academy Mountain View High School _ Mountain View-Los Altos Union High School District Mountain View, California In the 1986—87 evaluation of the Computech Academy, several challenges to the program were noted. Student enrollment was low, staffing changes were hurting the program’s stability, private sector support needed improvement, and the program appeared to have a poor image within the school. The 1986—87 report of student outcome measures showed only marginal positive difference between academy students and the comparison group. Most of the problems identified in the 1986—87 report still exist, although progress is being made on some fronts. The greatest improvement came with the recruitment of a relatively large sophomore class for the current 1988-89 school year. Renewed efforts are being made to improve private sector participation. But as was the case in the previous year, 1987—88 outcome measures do not show significant positive results for the academy students. Setting The combined population of Mountain View and Los Altos is about 80,000 people. Both communities are located in Santa Clara County with its Silicon Valley economy. Unemployment countywide is under five percent. Although the electronics industry has its ups and downs, the overall trend is towards continuing growth, and with the growth in electronics has come a parallel growth in service industries and jobs. Mountain View High is a sprawling, one-story suburban campus, with an educational program stressing college admission. Computech is located in a separate spacious classroom at the fringe of the campus. School officials estimate that 90 percent of the school’s graduates go on to college. In 1986—87, black and Hispanic students made up about 15 percent of the 1,300 students enrolled. Mountain View High’s AFDC rate of 1.9 percent is the second lowest of all academy high Schools. Its socio—economic status percentile (based on CAP testing 71 7 2 PENINSULA ACADEMIES 1987-88 EVALUATION. rankings) is 93 percent—highest among all academies. Percentile scores on the CAP tests in 1986—87 were 65 percent in reading and 89 percent in math. Thus Computech exists in the context of a high school district with declining enrollment, relatively few at-risk students, and a strong orientation toward college preparation. Student Profile While it is true that academy students can be drawn from any of the three district high schools, including Los Altos High School and Shoreline Continuation School, in fact most of the students who have found their way into Computech come from Mountain View High. The number of students in the district who meet the general profile of an academy student is relatively small, and this in part explains the relatively low enrollments in the academy program. TABLEl. Student Profile, Mountain View Academy Comparison Group Class (N) Sophomores 20 42 Juniors 15 25 Seniors 9 25 Race/Ethnicity (%) White 68 70 Black 12 11 Hispanic 12 13 Asian 3 O Other 5 6 Gender (%) Male 67 56 Female 33 44 COMPUTECH ACADEMY 7 3 For recruitment purposes, ninth grade students are identified at all three schools— Mountain View, Los Altos, and the continuation school—who fit the academy profile of poor attendance and underachievement. They are contacted individually and in group presentations. Recruitment in 1987—88 resulted in 100 sophomore referrals for the 1988—89 school year, and an ultimate enrollment of 25. Program Management The Director of Pupil Personnel is ultimately in charge of Computech, though day-to-day operations are administered by a lead teacher who has one period a day of released time. The Pupil Personnel Director acts as liaison with the state Department of Education and also prepares the program’s budget. But he, of course, has many other district responsibilities, as his title implies. Within the high school there appears to be limited attention from administrative staff and non-academy teachers except for the assistant principal who was assigned to help oversee the program and who at one time taught part time in Computech. The relative lack of attention by the school administration has, according to some Computech teachers, reinforced the program’s isolation from the rest of the school. In 1987—88, new equipment was purchased for the computer lab, thus addressing a concern of staff in the previous year. Students are block programmed in the mornings. Sophomores take academy English, math, science and computer literacy. Juniors take academy English, US. history, science, and electronics. Seniors in 1987—88 took academy English and a two-period lab class focusing on electronics, computers, and computer repair. Other required and elective classes are taken in the afternoon outside of Computech. For the most part, the academy “model” has been closely followed at Computech. Students are block programmed, and all of the teachers in the program share a common planning period. Although teachers do not have an extra planning period, they are paid for three periods a week of extra planning and meeting time. Classes are small, although as already noted, smaller than planned because of low enrollment and attrition problems. Private sector involvement was organized in part in the first two years of Computech through assistance from the Stanford Mid Peninsula Urban Coalition. Both mentors and summer jobs were arranged by Urban Coalition staff. In 1987—88, private sector relations 7 4 PENINSULA ACADEMIES 1987—88 EVALUATION were managed by the district itself, using a part-time aide who arranged 25 mentorships and 11 summer jobs. Staff from neighboring academies in the Sequoia and Eastside Districts joined together in their summer 1988 job placement program with considerable success. They attempted to involve Computech in their coordinated effort but say that Computech staff chose not to be involved. 0 The program has a business/industry advisory committee, but participation has been limited to a handful of electronics and computer company people. Meetings of the committee were infrequent and not well attended in 1987-88. Program Resources/Costs District Support There are 2.4 FTE academy teachers. This represents an additional 1.55 FI'E teacher for the academy class. There are two pan-time aides who spent a combined total of .3 FTE working for Computech. One aide finds jobs for seniors and graduates, arranges field trips, and solicits funds, equipment, and supplies from industry. The second aide assists with classroom instruction and in the computer lab. Administrative time for the academy is .2 FTE. A total of 75 days of staff development were reported as devoted to the academy to plan and coordinate the program. Private Sector Support Local business representatives volunteered 123.5 days for the academy. The bulk of these (88 days) were spent serving as mentors to individual students. Additional time commitments included 12 days supervising students, 8.5 days speaking to groups of students, 7 days organizing field trips, 6 days arranging job placements, and 2 days attending advisory committee meetings. Non-personnel resources included the following: 0 Facilities. The district supplied two oversized classrooms for the program. They estimated this cost at $10,800. COMPUTECH ACADEMY 7 5 o Supplies. Instructional supplies and books were purchased for $2,965. - Transportation. Cost of the field trips came to $1,299. - RewardsZSpecial Events. Total expenses for mentor meetings, birthdays, overnight field trips, graduation, etc. were $900. In addition, the Mountain View Academy reports spending $626 on publications and $1,101 on other services. Student Outcome Data The most objective data by which to judge the program’s effectiveness are measurable indicators of student performance. These include attendance, credits earned, grade point average, courses failed, and retention in school (the opposite of dropouts). While these are not the only indicators of the program’s effect on students, they provide the most statistically convincing measures. There are now data across three years on these measures by which to judge the program’s performance: that from 1987—88 for all three cohorts, from 1986-87 for cohorts one and two, and 1985—86 for cohort one. Table 2 presents differences in outcomes between each cohort of academy students and their matched comparison groups. These differences have been adjusted to account for differences between the two groups in age, gender, race or ethnicity, and academic performance in the preceding year. In this table, positive numbers for “attendance,” “credits,” and “GPA” mean that academy students compiled better records than the comparison group. Negative numbers for “courses failed” and “dropout probability” also mean that academy students performed better than the comparison group. Only when an asterisk appears next to the value is the difference large enough to be statistically significant. Of the 25 differences between academy students and comparison groups shown in this table, 17 favor the academy students and eight the comparison groups. Of those favoring the academy groups, one is statistically significant; of those favoring the comparison students, one is also. The first cohort (seniors) of academy students outperformed its comparison group in grade point average its first year. The first cohort of comparison group students outperformed the academy students in attendance the third year. 7 6 PENINSULA ACADEMIES 1987—88 EVALUATION: TABLE 2. Adjusted Differences Between Academy and Comparison Groups, by Class, Mountain View Computech Academy Atten- Courses Dropout dance Credits GPA Failed Probabilitv Cohort entering fall 1985 (seniors): 1985-86 Outcomes NA 4.1 0.6* -0.4 I 1986-87 Outcomes 0.6 -O.8 0.1 —O.3 -0.01 1987-88 Outcomes -2.1* -6.5# 0.2# -0.2 I Cohort entering fall 1986 (juniors): 1986-87 Outcomes -0.4 -2.8 0.3 0.7 -0.89 1987-88 Outcomes .0.3 2.5 -O.1 -O.6 I Cohort entering fall 1987 (sophs): 1987-88 Outcomes 0.6 -7.2 0.3 “0.1 I # Estimated from separate regressions for academy and comparison groups i Insufficient number of dropouts for analysis * Statistically significant With respect to dropouts, because the actual number (as opposed to the percent) of students who drop out is relatively small when analyzed by individual cohort, few statistical tests of differences between academy and comparison groups will prove to be significant, even when differences in the attrition between the two groups is substantial. For this reason, the following table presents the actual differences between the two groups. The overall “attrition” (what is usually meant when the term “dropout” is erroneously used) is broken out into its three components: known school dropouts (official withdrawals), transfers to other high schools, and probable dropouts (students lacking records as of June 1988 who neither officially withdrew nor transferred). Table 3 presents the figures by cohort. For the first cohort, seniors in 1987—88, the figures represent the cumulative rates across the three years of the program; for the second cohort, juniors, the COMPUTECH ACADEMY 7 7 cumulative rates over two years; and for the third cohort, sophomores, the rates for just the last year. TABLE 3. Known Dropouts, Probable Dropouts, and Transfers, by Cohort, for Academy and Comparison Group Students Academv m ri n r Known Prob. Trans- Known Prob. Trans- Drons Drons fers Droos Drons fers Cohort 1 0 O 63% 6% 2% 42% Cohort 2 O 0 45% 3% 2% 54% Cohort 3 O 7% 7% 0 0 0 There is no clear pattern here, although the academy does seem to have slightly reduced dropouts among the first and second cohorts; dropouts for both academy and comparison students are relatively low. By contrast, the transfer rate is high in both cohorts for both academy and comparison group students; in cohort 1 it is higher for academy students, in cohort 2 for comparison students. It is premature as yet to draw conclusions about cohort 3. Student Questionnaires Academy students complete questionnaires at the point they enter the program, at the beginning of their sophomore year, and again at the end of each school year. Some of the questions—on career—related plans, and attitudes related to career, school, and self—appear in parallel form on the “pre” and “post” version of the questionnaires, allowing assessment of changes over the course of the program. Another set of questions appears only on the “post” version, and asks for feedback on various program activities, and ratings of certain program components. The sections below present feedback from these questionnaires. Responses have been averaged across cohorts to simplify their presentation, except where clear differences 78 PENINSULA ACADEMIES 1987-88 EVALUATION appear between cohorts. Twenty-two students are represented in these data: six seniors, two juniors, and 14 sophomores. Prgggm Activities. Average number of classes in the academy was 4 for sophomores and juniors, and 2.5 for seniors. All juniors reported having a mentor, as did 83 percent of seniors and 43 percent of sophomores. Three students reported having jobs, all seniors. Sophomores reported having seven speakers, juniors four, and seniors 15. Sophomores went on four field trips, juniors 12. Program Ratings. Data were missing in this section for seniors. Ninety percent of students rated the equipment highly. Two-thirds rated the materials and schedule highly. Three-fourths saw a close connection between what they were studying in school and their postgraduate plans, and 90 percent liked the academy better than the regular program. gamer-Related Plans. Thirty-one percent of sophomores reported having a definite post-graduate plan, as did 50 percent of juniors and 67 percent of seniors (all these were sizable increases over their pre-program responses). Of these, two-thirds of sophomores and 90 percent of juniors and seniors were planning on college (up sharply among all groups). A fourth of the sophomores were planning on a job, as were half the juniors and two-thirds of the seniors (these are also increases over pre-program responses). Attitudes Related to Career. School, and Self. Data is missing in this category for seniors. Three—fourths of the students view it as important to make career plans and gain job skills while in high school (a sizable increase from pre-program responses). Two-thirds of the students found their classwork interesting during the last year (also a large increase from pre-program responses). Ninety percent of students “feel good about themselves” (a 42 percent jump for sophomores over pre- program responses). School Personnel Responses At the end of the school year, five staff from Mountain View (including the program administrator and teacher coordinator) completed a questionnaire. Responses from these questionnaires: The quality of the facilities, equipment, books, and other learning materials were all rated “good.” COMPUTECH ACADEMY 7 9 - Responses about the orientation/staff inservice ranged from “excellent” to “terrible/non-existent” with a comment that they “could have been planned better.” 0 The curriculum in the technical and academic classes received “good” ratings with suggestions that more inservice for the technical training teachers is necessary. Attempts have been made to interrelate the academic and technical class curricula. 0 Program planning and management were rated “good” to “average.” Earlier recruitment with more help from the counselors and more planning were suggested. ° Asked how supportive of the academy program other teachers, counselors and administrators in the school are, other teachers and administrators were rated “very supportive” to “non-supportive,” and counselors “somewhat supportive” to “non- supportive.” - Parents were ratedt“somewhat” to “slightly supportive and interested” of their children in the program. ~ The level of cooperation and interaction between the private sector representatives and the academy was rated “not very effective.” The private sector representatives were seen as reluctant to give help and not always following through on summer jobs. This did not, however, affect the positive attitude which the staff held for the private sector before becoming involved with the academy program. 0 The motivational components, such as field trips, speakers, and movies, were “very useful.” 0 It was felt that student demand and need for individual instruction is satisfactorily being met and that counseling for school~related problems and future careers was “good” to “excellent.” - Three of the staff were satisfied with the current criteria and procedures for selecting students while two were not satisfied. Staff were generally satisfied with the current procedures for dropping students. 0 The greatest weaknesses were seen to be the need for more communication with business, summer jobs, recruitment time, inadequate teacher preparation time, and the need for greater support from staff. 8 O PENINSULA ACADEMIES 1987-88 EVALUATION 0 The greatest strengths were seen to be small classes, “year-long courses enabling a warm rapport between students and teachers,” and the caring of teachers. 0 On a seven point scale (1 = very weak, 7 = very strong), the academy received an average rating of 6.8 “in concept” and 5.2 “as implemented.” Parent Responses Twelve parents completed questionnaires at the end of the school year. Their responses included the following: ° Six rated the quality of the program “excellent” and six rated it “good.” 0 All saw a change in their children’s attitude toward school and learning. Nine found their children “much more interested” and three “a little more interested” in school. Nine also felt that their children were making progress in planning and preparing for post-graduation pursuits. One did not and two were not sure. - Parents have been involved with the program in several ways: attending open house, helping their children with homework, meeting with teachers or counselors to discuss their children’s progress, and setting up field trips. - Asked how well they had been informed of their children’s progress in school, seven responded “very well,” three “moderately well,” and one “poorly.” 0 The features of the academy program the parents liked best were small groups, personal attention, special events, and the encouragement given to the students. 0 Suggestions given to improve the program included more equipment and involvement from computer companies, opportunity for more students to enter the program, and to have only qualified teachers. Private Sector Representatives Responses Two private sector representatives completed the questionnaire at the end of the school year—a supervisor from local business and a microcomputer specialist. Their responses: COMPUTECH ACADEMY 8 1 0 The companies were involved in the program in many ways: donating computer equipment, providing a guest speaker, working on an advisory committee to establish a computer repair business operated by students, and joining an advisory committee in curriculum development. - Asked if the curricula being used in the academy prepared students with skills needed in local private business, one responded “yes” and one “no.” - The effectiveness of the steering committee was rated as “average.” 0 The quality of the communications between the academy staff and the private sector supporters of the program was rated “average.” 0 On a seven point scale (1 = very weak, 7 = very strong), the academy received an average rating of 5.5 “in concept” and 3.5 “as implemented.” . The greatest weaknesses of the academy program were seen to be “how the academy is seen by other students and teachers, msufficient funding,” and “lack of staff motivation.” - The strength of the academy was seen to be that it “provides a program for those who would not otherwise participate in school.” . The only suggestion given was that there should be “more industry/private sector participation and members.” Mentor Responses Eight mentor questionnaires were received. Their responses showed that: - During the school year the mentors got together with their mentees nine times in person and twelve times by telephone. Seven mentors had visited their mentees at school. Five mentors had taken their mentees to visit their places of work. The other three had not done so due to security restrictions. 0 Six mentors felt that the academy is providing students with job skills needed by companies in the local area and that the program is serving the appropriate target population of students. Two responded “don’t know.” 8 2 PENINSULA ACADEMIES 1987-88 EVALUATIOI‘ 0 The planning and management of the mentor program was rated “excellent” to “average.” The mentor program itself was rated “highly” to “somewhat useful.” 0 The greatest weaknesses of the mentor program were seen to be a curriculum “not tough enough,” “student apathy,” “not enough mentor/student get-togethers set up by the school,” and lack of employer participation. - The greatest strengths of the program were “providing the student with a direct line to educated, employed people who are willing to share their experiences,” “teacher cooperation,” “group functions,” “deserving students getting exposure in areas where they might never gain access on their own,” “exposing students to ‘reality’,” “field trips,” and “keeping kids in school and having them graduate.” 0 Suggestions for improving the program were: “more information about the mentees should be given so changes can be made if a mentor feels an incompatibility exists,” and “school and employer need to Spend more time together.” Summary Only two of the 25 outcome measures reported here show statistically significant differences—one favoring academy students, one favoring comparison group students. The first cohort of academy students achieved significantly better grades than their comparison group in 1985—86. But in 1987—88, that same group achieved significantly worse attendance. So the findings of the 1986—87 evaluation were repeated in 1987—88: based strictly on outcome measures, the program has not made a major difference in the performance of its students. But as was the case in last year’s report, students, parents, and staff believe strongly that Computech is an important alternative for the small number of youth in the district who do not fit into the mainstream of the school. Parent satisfaction with the program continues high. District and school-level staff are clearly concerned about the future of Computech. They recognize the major shortcomings of the program: staff turnover, under-enrollment, and insufficient private sector involvement, especially through the advisory committee. Computech may be at a crossroads. Recent changes have been made in the management of the program with a new lead teacher assigned to school-level program COMPUTECH ACADEMY 8 3 coordination. Efforts to strengthen the business/industry advisory committee are also being made. The curriculum focus is changing from computer repair to computer technology viewed more broadly. The problems of Computech are being faced head on by school officials at all levels. The program is seeking greater enrollments, stronger school-level management, and improved private sector participation. If these goals can be achieved, Computech may be able to evolve into a successful academy. The Foothill Computer Academy Oak Ridge High School El Dorado Union High School District Diamond Springs, California This was a program plagued with problems last year, and most of them continued this year. Many of the basic components of an academy have not yet been put in place, including virtually all of the private sector activities. However, the “acting” principal took a serious interest in the program and set in motion efforts to establish a full academy. Unfortunately he took a position in another district for 1988—89, leaving in doubt the future of this academy. Problems exist in almost every sphere at this academy: Most sophomores were assigned to the academy at the end of the first quarter, based on their failure in one or more courses, rather than through their being informed of the program in advance and choosing to enter it. There were one technical and two (rather than three) academic classes for sophomores and juniors, and no program for seniors. There was no block scheduling and little academic/vocational coordination. While there were a few speakers, most from Sacramento vocational schools, there were no field trips to businesses, no mentors or jobs, and there was almost no contact with the private sector. There were just two academy teachers, including one who was new this year. While she proved a strong addition, these two had no common preparation period and coordinated little. On the positive side, the students in the program reacted well to the small classes and family atmosphere. Parental contact was substantial and positive. The equipment and facilities were excellent. Selection procedures for next year were improved substantially, with new students being chosen through a voluntary system and scheduled into the program during the spring. The full complement of classes for next year is planned, and teachers have been selected for them. Time has been allotted for one of these teachers to 85 8 6 PENINSULA ACADEMIES 1987—88 EVALUATION develop private sector support. If the remaining pieces of the program can be established, there is a basis for a strong academy. Setting El Dorado Union High School District is located just east of Sacramento, and covers an area that ranges from suburban to rural. It contains several small cities, including El Dorado Hills and Diamond Springs. The economy is largely service-based, with almost no industry and few large employers. Oak Ridge High School is a modern facility located in a picturesque semi-rural location, with housing developments springing up around it. Its student population of approximately 950 is 97 percent shite, 2 percent Hispanic, and 1 percent Asian/Pacific Islander. Its statewide socio-economic status percentile rank is 89 (based on CAP testing), and its 1986—87 CAP percentile rankings are 14 (reading) and 76 (math). Student Selection A well-defined student selection process was developed during the planning phase for the program. This involves examining the records of ninth grade students, looking for underachieving students who fit the intended profile. This profile includes spotty attendance, poor grades, low numbers of credits, and standardized test scores high enough to permit students to succeed with the program’s curriculum. Students who fit the criteria are told of the program, and those interested are invited to apply. Parents of those who do so are invited to an evening information presentation and asked to provide their support through the three years of the program. However, in the past this system has not been consistently followed, and students with academic problems have at times been placed in the program during the school year to fill out enrollments, whether or not they knew about the program or expressed interest in it. While sincere efforts were made in the spring to follow the intended selection procedures, interest in the program was not high and it was unclear whether additions would be needed during the 1988—89 school year. THE FOOTHILL COMPUTER ACADEMY 8 7 TABLE 1. Student Profile, The Foothill Computer Academy Academv Comparison Group Class (N) Sophomores 14 30 Juniors 1 1 1 1 Seniors 14 9 Race/Ethnicity (%) White 100 95 Black 0 0 Hispanic 0 1 Asian 0 2 Other 0 2 Gender (%) Male 52 52 Female 48 48 Program Management One of the program’s central problems is the lack of a director or leader. Those involved in obtaining the grant have since left the school or district. While one teacher was designated as the lead teacher, she was new to the program last year and was assigned this position rather than volunteering for it. Further, she was assigned six rather than the usual five teaching sections, leaving her little time to coordinate the program or pursue private sector support. She has repeatedly sought evidence of support from the school and district administration, which until recently seemed largely lacking. This was partly due to the presence of a new “acting” principal, and district superintendent, both of whom had their hands full with other responsibilities. There were just two teachers in the program, one of whom handles the computer lab classes, and the other who teaches both English and social studies. Sophomores and juniors have these three classes, while there was no program at all this year for seniors. The two teachers did not have a common planning period, and in fact coordinated little with 8 8 PENINSULA ACADEMIES 1987—88 EVALUATION each other. While the school counselors give perfunctory support to the program, its lack of leadership and direction gave them a lukewarm attitude as well until recently. The academy does enjoy good facilities and equipment. Students complain about the computer instruction being “canned” and uninspiring, however, and there has been virtually no private sector involvement in its development. There is also little coordination between the academic and technical curriculum, beyond occasional use of the computers for assignments in English and social studies. In fact, the most glaring weakness is the almost complete absence of private sector involvement in the academy. There is no Advisory Board with private sector representatives. There were a few speakers this year, primarily from Sacramento-area junior and technical colleges, and one field trip to San Francisco. But there were no mentors or jobs, nor regular contact of any kind with the private sector. The academy does provide a supportive, family-like atmosphere for students, which they like. It also has good contact with parents, who the teachers call regularly. Help is provided to the students in career planning, which includes practice in completing applications, writing resumes, and understanding job descriptions. Attendance is good in the academy classes, although this pattern has not generalized to non-academy classes. The plans for next year are encouraging. Virtually all features of an academy are now arranged for, including a full class schedule at all three grade levels, common planning periods for teachers, release time for one teacher to develop private sector support, and a prescribed student selection process. This selection process was effectively employed this spring, through the determined efforts of the two academy teachers and support from the school’s two counselors. However there is a long way to go, and the loss of the principal who supplied the leadership to put this plan together leaves in question what will happen next year. Program Resources/Costs District Support The academy’s 1.4 full-time-equivalent teachers represent an increase of .7 additional teachers over what would be required without the smaller class sizes in the program. The program has no specific contribution of administrative time, nor any aide. No staff development activities occurred during the year. THE FOOTHILL COMPUTER ACADEMY 8 9 Private Sector Support There were no estimated contributions of any variety from the private sector this year. Other categories of support: - Facilities. Regular high school classrooms. - Eguipment. The TRS-TV computers provided by the school for the computer lab originally; these are not used exclusively by the academy. - Supplies. Workbooks, disks, and paper from the “academy budget.” - Transportation. School bus transportation provided by the district. . Rewardflspcial events. Students earned the funds here through their business club. Activities included an awards night, and a trip to San Francisco. Student Outcome Data The most objective data by which to judge the program’s effectiveness are measurable indicators of student performance. These include attendance, credits earned, grade point average, courses failed, and retention in school (the opposite of dropouts). While these are not the only indicators of the program’s effect on students, they provide the most statistically convincing measures. There are now data across three years on these measures by which to judge the program’s performance: that from 1987—88 for all three cohorts; from 1986—87 for cohorts one and two; and 1985—86 for cohort one. Table 2 presents differences in outcomes between each cohort of academy students and their matched comparison groups. These differences have been adjusted to account for differences between the two groups in age, gender, race or ethnicity, and academic performance in the preceding year. In this table, positive numbers for “attendance,” “credits,” and “GPA” mean that academy students compiled better records than the comparison group. Negative numbers for “courses failed” and “dropout probability” also mean that academy students performed better than the comparison group. Only when an asterisk appears next to the value is the difference large enough to be statistically significant. 9 0 PENINSULA ACADEMIES 1987—88 EVALUATIOP TABLE 2. Adjusted Differences Between Academy and Comparison Groups, by Class, Foothill Computer Academy Atten- Courses Dropout dance Credits GPA Failed Probability Cohort entering fall 1985 (seniors): 1985-86 Outcomes 2.2 7.8* 0.4* —0.3 i 1986-87 Outcomes NA 0.8 0.3 0.1 —0.02 1987-88 Outcomes —0.9 —2.5 1.6 0.8* i Cohort entering fall 1986 (juniors): 1986-87 Outcomes NA 2.3 0.7* -1.0 i 1987-88 Outcomes ' i —4.8# -0.4 0.2 i Cohort entering fall 1987 (sophs): 1987-88 Outcomes 2.5 —1.8# 0.04 0.3# i # Estimated from separate regessions for academy and comparison groups i Insufficient number of dropouts for analysis * Statistically significant Of the 22 differences between academy students and comparison groups shown in this table, 13 favor the academy students and nine the comparison groups. Of those favoring the academy groups, three are statistically significant; of those favoring the comparison students, one is significant. The first cohort (seniors) of academy students outperformed its comparison group in credits and grade point average its first year; it was outperformed by its comparison group in courses failed its third year. The second cohort of academy students (juniors) outperformed its comparison group in grade point average its first year. With respect to dropouts, because the actual number (as opposed to the percent) of students who drop out is relatively small when analyzed by individual cohort, few statistical tests of differences between academy and comparison groups will prove to be THE FOOTHILL COMPUTER ACADEMY 9 1 significant, even when differences in the attrition between the two groups is substantial. For this reason, the following table presents the actual differences between the two groups. The overall “attrition” (what is usually meant when the term “dropout” is used) is broken out into its three components: known school dropouts (official withdrawals), transfers to other high schools, and probable dropouts (students lacking records as of June 1988 who neither officially withdrew nor transferred). Table 3 presents the figures by cohort. For the first cohort, seniors in 1987-88, the figures represent the cumulative rates across the three years of the program; for the second cohort, juniors, the cumulative rates over two years; and for the third cohort, sophomores, the rates for just the last year. TABLE 3. Known Dropouts, Probable Dropouts, and Transfers, by Cohort, for Academy and Comparison Group Students Academv Comparison Group Known Prob. Trans- Known Prob. Trans- Drous Drous fers Drons Drons fers Cohort 1 0 0 29% 0 8% 54% Cohort 2 O 0 57% 8% O 68% Cohort 3 O 0 O 0 0 0 These figures suggest that dropouts in this high school are generally low, and that the same is true of the academy students. However, transfers to other high schools are high, and the academy does seem to have reduced these somewhat, particularly among the first cohort. Student Questionnaires Academy students complete quesrionnaires at the point they enter the program, at the beginning of their sophomore year, and again at the end of each school year. Some of the questions—on careenrelated plans, and attitudes related to career, school, and self—appear in parallel form on the “pre” and “post” version of the questionnaires, allowing assessment 92 PENINSULA ACADEMIES 1987-88 EVALUATIOlV of changes over the course of the program. Another set of questions appears only on the “post” version, and asks for feedback on various program activities, and ratings of certain program components. The sections below present feedback from these questionnaires. Responses have been averaged across cohorts to simplify their presentation, except where clear differences appear between cohorts. Nineteen students are represented in these data, five juniors and 14 sophomores; senior data is missing. Progr_am Activities. Average number of classes students reported having in the academy was 2.7 for sophomores and 2.8 for juniors. No students reported having had a mentor. None reported having had a job. SOphomores reported having had one business speaker, juniors two. Juniors went on one field trip, sophomores none. Program Ratings. Fourteen percent of the sophomores rated the equipment highly; none of the juniors did so. Half the sophomores rated the materials highly; 20 percent of juniors did so. One-third of sophomores liked the academy schedule; none of the juniors did so. Two-thirds of the sophomores saw a close connection between what they were studying in school and their postgraduate plans; 20 percent of juniors did so. Eighty percent of students in both classes liked the academy better than the regular program. finer—Related Plans. One-fourth of sophomores and one-half of juniors reported having a definite post- graduate plan. Of these, two-thirds are planning on college (up 34 percent among juniors from their pre-program responses); 40 percent are planning on a job. Slightly less than half have a long-term career goal (down 25 percent from pre-program responses). AttitudesRelated to Career. School. and Self. One—third of sophomores (down somewhat from pre-program responses) and two—thirds of juniors view it as important to make career plans and gain job skills while in high school. Forty—six percent of sophomores and 75 percent of juniors found their classwork interesting during the last year", no students in either class had reported this in their pre- program responses. Eighty-two percent of sophomores and 75 percent of juniors also “feel good about themselves” (both sizable increases over pre-program responses). THE FOOTI-IILL COMPUTER ACADEMY 9 3 School Personnel Responses Three teachers from Oak Ridge High School completed a questionnaire at the end of the school year. Responses from these questionnaires: 0 The equipment and facilities were rated “excellent,” and the instructional materials and curriculum “good.” 0 Non-academy teachers and counselors were rated as “somewhat supportive,” while administrators were rated as “very supportive.” 0 Program planning was rated “average”; one respondent complimented the teachers in this respect, but criticized the district. 0 Cooperation and support from the private sector was rated “non-existent.” o Other aspects of the program were rated variously. Student selection and drop procedures were criticized. Motivational activities, individual student attention, and counseling were all given good ratings. Parents were rated from “very” to “non” supportive and interested. 0 On a seven point scale (1 = very weak, 7 = very strong), the academy received an average rating of 6 “in concept” and 4.7 “as implemented.” . The greatest weaknesses were seen to be discipline problems with the academy students, lack of district leadership and support, poor selection procedures, and lack of parent commitment. o The greatest program strengths were seen to be the small class size, individual attention and tutoring, and support for students who would otherwise be lost on a comprehensive high school campus. Parent Responses One parent completed a questionnaire at the end of the school year. Her responses included the following: 9 4 PENINSULA ACADEMIES 1987—88 EVALUATIOD 0 She rated the program as “good.” 0 She saw her child as “much more interested in school and learning” since joining the academy. 0 She had been involved with the program through attending an Open house at the school and helping her child with homework. - Asked how well she had been kept informed of her child’s progress in school, she said “very well.” 0 The features of the academy that she liked best were the teachers, who she commented were “all very good.” 0 The only weakness mentioned was that “they give no homework.” No “Mentor” or “Private Sector Representative” questionnaires were received here. Summary The student outcomes analysis for 1987-88 shows only one statistically significant difference, and that one favors the comparison group; the academy seniors failed significantly more courses than their matched comparison group. Of the other ten differences between academy and comparison groups, seven favor the comparison groups. Over the three years of the program, three differences have significantly favored the academy students and one the comparison group. The student questionnaire feedback reflects the lack of many of the activities usually associated with an academy: there were no mentors or jobs, and few speakers or field trips. In spite of the fact that program ratings were not generally very high, however, students reported sharp increases in their liking of school compared with pre-program ratings, and most said they liked the academy better than the regular high school program. Teacher and parent ratings reflected both positive and negative reactions to the program— the small classes and family-like atmosphere of the program are well—liked, but the student selection procedures, lack of district support, and private sector involvement are found wanting. THE FOOTHILL COMPUTER ACADEMY 9 5 What can one conclude from all this? This program is not yet a true academy. After three years, it lacks fundamental features that define an academy, in particular private sector involvement. It has also operated long enough at a half-stage of implementation that its image now makes it difficult to recruit students. This is fair to neither the teachers nor students in the program. It is time for the school and district administrators at this site to decide if they really want an academy or not. If they do, they should support it so that it becomes a true academy. If they do not, it is time to end the halfway efforts that have defined this program to date. The program at Oak Ridge High School does justice to neither the students there nor the academy name statewide. The Health Academy Oakland Technical High School Oakland Unified School District Oakland, California This high school continued to operate a full-fledged academy with virtually all of the model’s components in place. The program won a second state competition for a “California Academic Partnership (CAP)” grant, and so had approximately double the usual academy funds on which to operate. This also gave the program a link to the Samuel Merritt College of Nursing, and not only curriculum development and tutoring support from college students and staff but access for students to professional meetings in the field. The Health Academy continued to enjoy strong school and district support. One problem here was an unintentional foul-up in the scheduling of students in the fall, interfering with the academy-only structure of the classes and causing delays in the fall curriculum. However, eventually this was resolved and students received training in first aid, AIDS counseling, personal health, health occupations, and computers, in addition to the basic courses in English, science, and math. Oakland has a district-wide academy director for the four Oakland academies, who helped to organize an advisory committee and support various functions. Perhaps the most impressive feature of this academy is the strong private sector support. There was an active program of speakers and field trips. Mentoring developed, although this occurred more through job supervisors and other activities than as a separate program. All qualified students had summer jobs in health, most in local hospitals, and many had volunteer positions during the school year. Kaiser Hospitals in Oakland, Walnut Creek, and San Francisco all proved supportive, both through management and labor unions, as did Children’s Hospital in Oakland. Private doctors and dentists hired students as well. A number of seniors were accepted to college, several with scholarships. Setting With a population of approximately 350,000, Oakland is the third largest city in the San Francisco Bay area. Located on the east side of the bay, Oakland has a relatively high minority population. Its economy, while perhaps less robust than that of San Francisco 97 9 8 PENINSULA ACADEMIES 1987—88 EVALUATIONS. and the Silicon Valley to the west, is nevertheless increasingly diverse and healthy. Oakland Technical High School is one of six comprehensive high schools in the Oakland Unified School District. It is housed in a recently refurbished and attractive older two-story building in an integrated section of North Oakland. Its approximately 1,600 students are 18 percent Asian, 73 percent black, 3 percent Hispanic, and 6 percent white. Its statewide socio-economic status percentile is 27 (based on CAP testing), and its 1986—87 CAP percentile scores are 14 (reading) and 45 (math). Student Profile There are few students at Oakland Tech High School who are not “at risk” in some sense usually associated with this term. Thus the selection process here involves not so much finding students who are unusually at risk, but rather average students who will benefit from the program because of its motivational features and focus on health careers. Much of the selection comes through recommendations of teachers and counselors at the ninth grade. The school uses a tracking system, and with rare exception those in the “GATE” (Gifted and Talented) track are eliminated from consideration. Thus the participants come from the “survey” track, which is average students at the school (one or two years below grade level in reading), and the “skills” track (three or more years below grade level). Among these, students are sought who appear to be performing below their ability as reflected in low credits or poor attendance. Interest in the program and health careers is also an important consideration. The table below provides a profile of the 1987—88 academy classes, and their matched comparison groups. THE HEALTH ACADEMY 9 9 TABLE 1. Student Profile, The Health Academy Academv Comparison Group Class (N) Sophomores 45 37 Juniors 21 26 Seniors 26 11 Race/Ethnicity (%) White 5 4 Black 79 88 Hispanic 2 2 Asian 10 3 Other 4 2 Gender (%) Male 35 35 Female 65 65 Program Management The key to this academy’s management continues to be the lead teacher and coordinator, who also teaches English. She is the only teacher whose time is fully dedicated to the program. There are also teachers in biology (sophomores), physiology (juniors), chemistry (seniors), and math, as well as those who provide training in computers, personal health, and health occupations. The academy teachers have a common preparation period and meet at least weekly to plan activities and resolve student problems. Other staff at the high school are supportive of the program. All the high school’s counselors help counsel program students concerning school and personal problems. They also help students plan careers, and contribute to the selection of new students. The principal, a dynamic and effective leader, is very supportive. The program has an impressive home room with good equipment and supplies. The CAP grant has allowed extensive curricular development this year, through the help of teachers and students at Samuel Merritt College of Nursing. The advisory 100 PENINSULA ACADEMIES 1987—88 EVALUATION committee has also helped in contributing to the curriculum. Teachers have had a series of after-school workshops and retreats as part of the Bay Area Writing Project. More are planned next year, to incorporate writing into the teaching of math and science. Parental contact has continued this year. The program put out a weekly newsletter, and there were two open houses and a potluck dinner. Parents continued to be telephoned when students had problems. This year students began to help in this telephoning, reinforcing the peer pressure to perform well in the academy. The private sector has contributed in many ways: 0 There has been approximately one speaker per week, mostly hospital employees and physicians. 0 There have been many field trips, including to Stanford, the Academy of Sciences, the Exploratorium, Children’s and Kaiser Hospitals, St. Mary’s College Science Department, Piedmont Gardens (a senior center), and Cutter’s Lab (the first time it was opened to visitors). 0 Samuel Merritt College films a bi-weekly television show, and the academy students acted as an audience for this, asking health-related questions of speakers. - The Red Cross provided 20 hours of instruction in AIDS peer counseling, and academy students are using the skills developed here in informing other student groups on this topic. - Local ambulance companies let students accompany them on runs to get a first-hand look at this type of health work. 0 A mentor breakfast resulted in 200 attendees, and individual contact between students and employees in the field is extensive. - All qualified juniors had summer jobs in the health field, most in hospitals. Some seniors and sophomores were also provided jobs. 0 Students attended five professional conferences during the year, including ones focused on medical students, law students, and black health. ° Students will be involved in BART emergency training, serving as “victims” of train accidents. THE HEALTH ACADEMY 1 0 1 - Oakland Tech High School sponsored a “Labor Education Week,” during which the hospital unions supporting the academy informed students of union history and contributions to the health field. ° One doctor was so enthused about the academy he personally donated several thousand dollars to start a scholarship program. Program Resources/Costs District Support The academy’s four full-time-equivalent teachers represent an increase of approximately 1.75 additional teachers over what would be required without the program’s smaller class sizes. The academy also employs an aide about one-third time. The academy director and resource person each receive one additional preparation period per day. The district academy director devotes quarter—time to this academy, and has an assistant who spends 40 percent time on the academy. Teachers spent 64 days this year in staff development activities, primarily developing curriculum. Private Sector Support As detailed above, this academy enjoyed a wealth of private sector support. An estimated 1,307 days of time were contributed by representatives of local employers—25 in advisory committee meetings 40 speaking to groups of students 50 organizing and leading field trips 112 serving as mentors 20 arranging job placements 960 supervising students on the job and 100 helping to plan and evaluate the academies. Other categories of resources: . Facilities. Use of classroom refurbished by the district at the high school; use of facilities at Samuel Merritt College of Nursing, UCSF Medical School, and Oakland Kaiser Hospital. - Eguipment. Approximately $10,000 was spent by the district on science equipment, and a $5,000 grant was contributed by Kaiser Hospital for computers. 1 O 2 PENINSULA ACADEMIES 1987—88 EVALUATION ° Supplies. The district budgeted $5,000 for academy supplies. In addition, donations were contributed by Samuel Merritt College of Nursing, Kaiser Hospital, and the American Red Cross for videos, books, overhead transparencies, models, lab supplies, paper, and printing. Won. Most resources in this category were contributed by a hospital worker’s union, the Black Health Network, and the California Academic Partnership grant. Examples include bus rentals, tickets for public buses and BART, and private cars. Rewardslsgcial events. Many events took place in this category: potluck dinners, luncheons, an ice cream social, academy T-shirts, award certificates, pins, attendance at plays and films, a student-mentor breakfast, and so forth. An estimated $5,000 was spent, coming from various sources. ° Other. Clorox provided a $10,000 grant for a leadership training program; Kaiser Foundation provided $5,000 toward this, also. Student Outcome Data The most objective data by which to judge the program’s effectiveness are measurable indicators of student performance. These include attendance, credits earned, grade point average, courses failed, and retention in school (the opposite of dropouts). While these are not the only indicators of the program’s effect on students, they provide the most statistically convincing measures. There are now data across three years on these measures by which to judge the program’s performance: that from 1987—88 for all three cohorts; from 1986-87 for cohorts one and two; and from 1985—86 for cohort one. Table 2 presents differences in outcomes between each cohort of academy students and their matched comparison groups. These differences have been adjusted to account for differences between the two groups in age, gender, race or ethnicity, and academic performance in the preceding year. In this table, positive numbers for “attendance,” “credits,” and “GPA” mean that academy students compiled better records than the comparison group. Negative numbers for “courses failed” and “dropout probability” also mean that academy students performed better than the comparison group. Only when an asterisk appears next to the value is the difference large enough to be statistically significant. THE HEALTH ACADEMY 1 O 3 TABLE 2. Adjusted Differences Between Academy and Comparison Groups, by Class, Oakland Health Academy Atten- Courses Dropout dance Credits GPA Failed Probabilitv Cohort entering fall 1985 (seniors): 1985-86 Outcomes 9.3* 27.0* 1.1* —3.4* i 1986—87 Outcomes 15.6 26.2* 0.6 «3.7 —O.29* 1987—88 Outcomes 2.2# 1.1# —0.5* 0.4 i Cohort entering fall 1986 (juniors): 1986-87 Outcomes 8.0* 21.3* 1.0* —3.1* i 1987—88 Outcomes ' 5.3#* -0.2 —O.2 —0.6 i Cohort entering fall 1987 (sophs): 1987—88 Outcomes 12.1* 2.4* 1.1* —2.6* i # Estimated from separate regressions for academy and comparison groups i Insufficient number of dropouts for analysis * Statistically significant Of the 25 differences between academy students and comparison groups shown in this table, 21 favor the academy students and four the comparison groups. Of those favoring the academy groups, 15 are statistically significant; of those favoring the comparison students, one is significant. The first cohort (seniors) of academy students outperformed its comparison group in attendance, credits, grade point average, and courses failed its first year, and in credits and dropout probability its second year. This cohort’s comparison group outperformed the academy group in grade point average the third year. The second cohort of academy students (juniors) outperformed its comparison group in attendance, credits, grade point average, and courses failed its first year, and in attendance its second year. The third cohort (sophomores) of academy students outperformed its comparison group in attendance, credits, grade point average, and courses failed its first year. 104 PENINSULA ACADEMIES 1987-88 EVALUATION With respect to dropouts, because the actual number (as opposed to the percent) of students who drop out is relatively small when analyzed by individual cohort, few statistical tests of differences between academy and comparison groups will prove to be significant, even when differences in the attrition between the two groups is substantial. For this reason, the following table presents the actual differences between the two groups. The overall “attrition” (what is usually meant when the term “dropout” is erroneously used) is broken out into its three components: known school dropouts (official withdrawals), transfers to other high schools, and probable dropouts (students lacking records as of June 1988 who neither officially withdrew nor transferred). Table 3 presents the figures by cohort. For the first cohort, seniors in 1987—88, the figures represent the cumulative rates across the three years of the program; for the second cohort, juniors, the cumulative rates over two years; and for the third cohort, sophomores, the rates for just the last year. TABLE 3. Known Dropouts, Probable Dropouts, and Transfers, by Cohort, for Academy and Comparison Group Students Academv m i n r Known Prob. Trans- Known Prob. Trans- Drons Drops fers Drops Drops fers Cohort 1 3% O 10% 3% 15% 55% Cohort 2 0 O 23% 3% 8% 24% Cohort 3 0 2% 12% O O 0 These figures suggest that the academy has helped to retain students in high school at Oakland Tech, and in particular has reduced transfers to other high schools among the first cohort. Given the nature of this high school, the small number of actual dropouts among academy participants is quite remarkable. THE HEALTH ACADEMY 1 O 5 Student Questionnaires Academy students complete questionnaires at the point they enter the program, at the beginning of their sophomore year, and again at the end of each school year. Some of the questions——on career-related plans, and attitudes related to career, school, and self—appear in parallel form on the “pre” and “post” version of the questionnaires, allowing assessment of changes over the course of the program. Anorher set of questions appears only on the “post” version, and asks for feedback on various program activities, and ratings of certain program components. The sections below present feedback from these questionnaires. Responses have been averaged across cohorts to simplify their presentation, except where clear differences appear between cohorts. Thirty-three students are represented in these data, five juniors and 28 sophomores; senior data are missing. 0 Program Activities. Average number of classes reported in the academy was 3.8 for sophomores and 3.4 for juniors. Three-fourths of juniors reported having had a mentor, as did 44 percent of sophomores. Half the juniors reported having had a summer job. Both classes reported having had 16 speakers during the year. Juniors had gone on three field trips, sophomores on 13. o Proor== am Ratings. Almost all students (97 percent) rated the equipment and materials highly. Ninety-six percent of sophomores liked the academy schedule; this was true for 75 percent of juniors. Eighty-six percent of sophomores saw a close connection between what they were studying in school and their postgraduate plans; 75 percent of juniors did so. Ninety-five percent of students liked the academy better than the regular program. . ' r r-R lat P]. n . Half the SOphomores (12 percent more than in pre-program responses) and all the juniors reported having a definite post-graduate plan. Of these, 87 percent were aiming for college, and 47 percent (a 27 percent increase) for a job. Three-fourths of the students have a long-term career goal (up 12 percent from pre-program responses). . Attitudes Related to Career. School. and Self. Eighty percent of academy students view it as important to make career plans and gain job skills while in high school. Sixty-nine percent of sophomores found their classwork interesting during the last year (an increase of 19 percent from pre-program responses); this was true for 10 6 PENINSULA ACADEMIES 1987-88 EVALUATIOl 50 percent of juniors. Eighty-five percent of academy students “feel good about themselves.” Other Questionnaires No questionnaires were received from this site for school personnel, parents, private sector representatives, or mentors. Summary For the first time this year there was one difference between academy and comparison group students favoring the latter at a statistically significant level, among the senior cohort. But this was clearly an exception from the continuing pattern of superior performance among academy students. Indeed, at the sophomore level, during the program’s first year of effect on students, this is now so clear and consistent as to be unmistakable. Student attitudes are also strongly positive about the program and its effects on them. The one shortcoming in the evaluation is the lack of feedback from adults associated with the program. But all the evidence that is available continues to confirm that this is a an academy that is clearly successful. The reasons for this are apparent. The full-fledged implementation of the academy model here has been rarely matched in the replications to date. Virtually every feature of an academy that is called for has been put in place. The community resources that have been assembled in support of the program are many and rich, and a testament to the work and dedication of the program’s staff. The leadership given the program by its director is also unusual. Support from the school and district has been strong. In short, the Oakland Health Academy is a testament to what can be achieved with this model with the right mix of staff, hard work, support, and vision. The program continues to set a standard of excellence for others to emulate. The Rio Cazadero Academy Rio Cazadero High School Elk Grove Unified School District Sacramento, California A number of problems that appeared last year were not resolved, and their growth this year hindered this program’s initial success, and ultimately resulted in the academy’s transfer to another high school. Many of these related to the difficulties of trying to fit the school- within-a-school model into a small alternative high school campus, where space and scheduling needs came into conflict with the school’s other staff and students. The alternatives became either transforming the entire campus into an academy, or moving the program to one of the district’s comprehensive high schools next year. The latter course was chosen. In addition to the problems of structure, weaknesses in the implementation of several of the academy’s components emerged this year also. Selection of students with relatively high ability on this continuation campus gave the program an unfortunate elitist image. Development of the joint electronic and computer curricula caused the latter to suffer. Problems were particularly acute with private sector components—there was no advisory body with private sector representation, just a few speakers, only one field trip, and virtually no jobs. Consequently, participants became disillusioned, and there were many dropouts, particularly among seniors. The hope now is that the program can be reconstituted at Elk Grove High School; in effect it will be a new academy in 1988—89. Setting Elk Grove is just to the south of Sacramento, with the district lying partly within the city limits. Rio Cazadero High School, which draws from the district’s two comprehensive high schools, is located in a suburban area just inside Sacramento. It is quite new, and its five small buildings are all one-story. It is surrounded by a residential area of middle- income ranch style houses and condominiums. Its student population is approximately 200, which consists of the following ethnic groups: 57 percent white, 24 percent black, 15 percent Hispanic, 3 percent Asian, and 1 percent American Indian. No data were available from the state on Rio Cazadero’s socio-economic ranking or CAP scores. 107 l O 8 PENINSULA ACADEMIES 1987-88 EVALUATIONS Student Profile From the start, this program has used a well-defined student selection process, based on the Peninsula Academy model. Ninth grade students in the district’s two comprehensive high schools are invited to an informational meeting about the program Those who show interest are examined in terms of their background. Criteria used for selection include: 0 Ten or more credits behind in ninth grade 0 Standardized test scores at the 30 percent to 50 percent range in reading and math, putting them below grade level 0 A pattern of poor attendance In addition, teachers look for other evidence of reasonable ability but poor motivation, through contacts with ninth grade teachers, counselors, and administrators. An unfortunate TABLE 1. Student Profile, The Rio Cazadero Academy Academv Comparison Group Class (N) Sophomores 25 50 Juniors 18 14 Seniors 11 19 Race/Ethnicity (%) White 70 72 Black 9 9 Hispanic 12 14 Asian 3 1 Other 6 3 Gender (%) Male 48 56 Female 52 44 THE RIO CAZADERO ACADEMY 1 0 9 pattern of selection of high-ability students, predominantly white, increased in 1987-88; this tended to alienate non-academy teachers and students. Program Management The Rio Cazadero Academy continued to be coordinated by the lead teacher who had been responsible for the past two years, although the principal who had been closely involved previously took a job elsewhere and was not present to support her efforts. The new principal encountered substantial resentment toward the program among non-academy staff, and determined over the course of the year that it was wisest to move the program elsewhere. Fitting the academy model onto this small campus created several problems that emerged strongly this year: ° Space is tight, with only five small modular buildings; the program in effect completely took over one of these with its twin computer and electronics labs, removing its use for other students. c The normal student schedule here is a half-day one, in which students attend three classes mornings or afternoons; the academy required them to attend five (with a 2—hour lab), throwing this into disarray. 0 On this campus where all students had problems, the academy students had been selected for their ability and potential; this fact and the program’s reputation ironically gave them an elitist image, causing resentment among non-academy students and staff. This was exacerbated by the fact that almost no minority students were selected. - Organizing the classes separately into academy and non-academy ones was especially difficult, with the small number of teachers and courses. While this was done for sophomores, it was not generally done for juniors and seniors. The program also lacked dedicated academy teachers, and extra preparation periods for them. Thus what had seemed in many ways an ideal setting proved otherwise. 1 10 PENINSULA ACADEMIES 1987—88 EVALUATIONSI Problems also developed in implementing the academy model, particularly related to private sector involvement. Without the original principal’s help in developing this involvement, it became difficult for the lead teacher to accomplish. An organizational meeting took place for the advisory board, but this body never really materialized. Consequently there was little curricular input from the private sector. The computer curriculum was seen to be weak at higher levels; in fact, there was no real program for juniors and seniors in computers. There were a few business speakers who generally directed their remarks to employment preparation skills. There were mentors for all students who wished them; this was a strong element of the program, supported by a local Rotary Club. There was one field trip to the Sacramento State University campus computer and electronics labs, and students were invited to work there over the summer. Virtually no progress was made in developing jobs. This, of course, created resentment among students who had been promised these. While many obtained jobs on their own, few if any of these were related to the training. ‘ Some portions of the program remained strong. Students continued to like the small, family-like atmosphere, and the friendly, supportive attitude of teachers. Parental contact was substantial, and the program aide developed a good relationship with many of these. The equipment was good, both in electronics and computers. One difficulty here was that the Commodore computers that comprise the bulk of those in the lab were felt to be poorly matched to those used in business. The deterioration in campus support for the program contributed to its difficulties as the year continued. It became hard to recruit business or solve problems once it was known the academy was going to change campuses. And the transfer itself will probably be difficult. It is not clear, for example, that the new comprehensive high school housing the academy will even attempt to establish a junior and senior program—it may just begin an entirely new academy with a new set of sophomores. Almost an entirely new set of staff will be involved. In short, it will be a new program. THE RIO CAZADERO ACADEMY 1 1 1 Program Resources/Costs District Support The alternative high school setting, small size of the school, and integration with non- academy elements makes estimating numbers here difficult. The academy employed one full-time-equivalent teacher, the same as would have been required without the academy. However, the lead teacher was given 40 percent release time to administer the program, and a full-time aide. While there were no staff development activities as such, high school teachers spent 2.5 hours at staff meetings per week, some of which was spent discussing academy needs. Private Sector Support An estimated 13 days of time were contributed by representatives of local employers: one at advisory committee meetings, two speaking to groups of students, two organizing and leading field trips, two arranging job placements, and six supervising students on the job. While there were mentors, no estimate of their time was provided. Other categories of support: . Facilities. The computer and electronics labs were provided by the district at the time the program began. No new additions were made this year. - Equipment. Again, a variety of electronics equipment and computers were provided by the district at the time the program began, but no new additions were made this year. - Supplies. The district, ROP program, and academy all helped to supply computer paper, textbooks, etc. . Transportatien. The district supplied bus service from its two comprehensive high schools to the academy site. 0 Rewardslsecial events. Academy funds were used for mentor luncheons and field trips. 1 12 PENINSULA ACADEMIES 1987—88 EVALUATIONS Student Outcome Data The most objective data by which to judge the program’s effectiveness are measurable indicators of student performance. These include attendance, credits earned, grade point average, courses failed, and retention in school (the opposite of dropouts). While these are not the only indicators of the program’s effect on students, they provide the most statistically convincing measures. There are now data across three years on these measures by which to judge the program’s performance: that from 1987—88 for all three cohorts; from 1986—87 for cohorts one and two; and 1985—86 for cohort one. Table 2 presents differences in outcomes between each cohort of academy students and their matched comparison groups. These differences have been adjusted to account for differences between the two groups in age, gender, race or ethnicity, and academic performance in the preceding year. In this table, positive numbers for “attendance,” “credits,” and “GPA” mean that academy students compiled better records than the comparison group. Negative numbers for “courses failed” and “dropout probability” also mean that academy students performed better than the comparison group. Only when an asterisk appears next to the value is the difference large enough to be statistically significant. Of the 27 differences between academy students and comparison groups shown in this table, 21 favor the academy students and six the comparison groups. Of those favoring the academy groups, 16 are statistically significant; of those favoring the comparison students, two are significant. The first cohort (seniors) of academy students outperformed its comparison group in attendance, credits, grade point average, and courses failed its first year, in grade point average and courses failed its second year, and in grade point average its third year. It was outperformed by the comparison group in attendance and credits the third year. The second cohort of academy students (juniors) outperformed its comparison group in attendance, credits, grade point average, and courses failed its first year, and in grade point average and courses failed the second year. The third cohort (sophomores) outperformed its comparison group its first year in credits, grade point average, and courses failed. Interviews with teachers this year revealed some potential biases in the above comparisons, which must be reported here. The comparison groups are made up of students in the district’s two comprehensive high schools. This was done because it was THE R10 CAZADERO ACADEMY 1 1 3 TABLE 2. Adjusted Differences Between Academy and Comparison Groups, by Class, The Rio Cazadero Academy Atten- Courses Dropout dance Credits GPA Failed Probabilitv Cohort entering fall 1985 (seniors): 1985—86 Outcomes 12.7* 25.4* 1.5* —1.6* -0.08 1986—87 Outcomes —O.3 4.5 1.3* —1.8* 0.12 1987—88 Outcomes -11.8* -13.4#* 1.3#* —O.4 i Cohort entering fall 1986 (juniors): 1986—87 Outcomes 4.5* 19.7* 1.3* —2.9* ~0.05 1987—88 Outcomes - —2.5 —2.4 0.5* -1.4* i Cohort entering fall 1987 (sophs): 1987—88 Outcomes 1.17 26.4#* 1.8* —2.8* i # Estimated from separate regressions for academy and comparison groups i Insufficient number of dropouts for analysis * Statistically significant felt it would be too hard to separate the effects of the small Rio Cazadero campus from the influence of the academy otherwise. Also, there were too few non-academy students to form an adequate matched group at Rio. Unfortunately there appear to be differences in the way credits and grades are given between Rio Cazadero and the comprehensive high schools. At Rio, a credit is given for 12 hours of work; at the other high schools, for 17 hours. No D’s and F’s are given at Rio, while they are at the other schools. Thus the comparisons on credits, courses failed, and grade point average are unreliable, and the above analyses which reflect superior academy student performance on these measures are untrustworthy. There is no precise way to know how much bias is present, and there may still be some improved performance resulting from the academy influence, but the numbers above must be regarded skeptically. 1 14 PENINSULA ACADEMIES 1987—88 EVALUATIONS; With respect to dropouts, because the actual number (as opposed to the percent) of students who drop out is relatively small when analyzed by individual cohort, few statistical tests of differences between academy and comparison groups will prove to be significant, even when differences in the attrition between the two groups is substantial. For this reason, the following table presents the actual differences between the two groups. The overall “attrition” (what is usually meant when the term “dropout” is erroneously used) is broken out into its three components: known school dropouts (official withdrawals), transfers to other high schools, and probable dropouts (students lacking records as of June 1988 who neither officially withdrew nor transferred). Table 3 presents the figures by cohort. For the first cohort, seniors in 1987—88, the figures represent the cumulative rates across the three years of the program; for the second cohort, juniors, the cumulative rates over two years; and for the third cohort, sophomores, the rates for just the last year. TABLE 3. Known Dropouts, Probable Dropouts, and Transfers, by Cohort, for Academy and Comparison Group Students Academv Comparison Grouo Known Prob. Trans- Known Prob. Trans- Droos Droos fers Drops Droos fers Cohort 1 16% O 44% 7% 2% 44% Cohort2 7% O 21% 7% 7% 51% Cohort 3 O O O O 0 0 There is little evidence of the academy reducing actual dr0pouts among its participants, although it does appear to have reduced transfers to other high schools among the second cohort students. THE R10 CAZADERO ACADEMY 1 1 5 Student Questionnaires Academy students complete questionnaires at the point they enter the program, at the beginning of their sophomore year, and again at the end of each school year. Some of the questions—on career-related plans, and attitudes related to career, school, and self—appear in parallel form on the “pre” and “post” version of the questionnaires, allowing assessment of changes over the course of the program. Another set of questions appears only on the “post” version, and asks for feedback on various program activities, and ratings of certain program components. The sections below present feedback from these questionnaires. Responses have been averaged across cohorts to simplify their presentation, except where clear differences appear between cohorts. Eighteen students are represented in these data: one senior, nine juniors, and eight sophomores. ~ Proorg am Activities. Senior data are missing in this section. Average number of classes students reported having in the academy was two. Ninety percent of juniors reported having had a mentor. One junior reported having had a job. Sophomores reported having two speakers, juniors six. Both classes reported having gone on two field trips. 0 Program Ratings. Ninety percent of students rated the equipment highly. Half the sophomores and two-thirds of the juniors rated the materials highly. Half the students liked the academy schedule. About half the academy students saw a close connection between what they were studying in school and their postgraduate plans. Three-fourths liked the academy better than the regular program. ° Career-Related Plans. Fifteen percent of participants reported having a definite post-graduate plan. Of these, all were planning on college, and 80 percent on a job as well. Eighty-three percent of sophomores have a long-term career goal (up 33 percent since their pro-program responses), while 44 percent of juniors do so. - Attitudes Related to Career School and Self. About half the academy students view it as important to make career plans while in high school. Two-thirds feel gaining job skills in high school is important. In both cases, these are declines from pre-program responses. Half the students found their classwork interesting during the last year, a substantial increase from pre-program responses. Two- thirds “feel good about themselves,” a slight increase over pre—program responses. 1 16 PENINSULA ACADEMIES 1987-88 EVALUATIONS School Personnel Responses Seven staff from Rio Cazadero High School completed a questionnaire at the end of the school year: four teachers, an aide, an attendance clerk, and the school principal. Responses from these questionnaires: The equipment and instructional materials were rated “average,” the facilities and curriculum from “average” to “poor.” ' Non-academy teachers and counselors were rated as “non-supportive” of the program, while administrators were rated as “slightly supportive.” 0 Program planning was rated from “average” to “poor.” 0 Cooperation and support from the private sector was rated from “somewhat effective” to “not very effective.” ° Other aspects of the program were rated variously. Student selection and motivational activities were rated low; individual student attention and counseling were given high ratings; and parents were rated as “somewhat supportive and interested.” - On a seven point scale (1 = very weak, 7 = very strong), the academy received an average rating of 4.8 “in concept” and 2.7 “as implemented.” - The greatest weaknesses were seen to be the difficulties in fitting the academy schedule and space requirements into this small alternative campus, poor leadership and direction, an inappropriate elitist attitude fostered within the program, and the unfulfilled promises made to students. 0 The greatest program strengths were seen to be the individualization and close family atmosphere, which are particularly important at a large comprehensive campus. THE RIO CAZADERO ACADEMY 1 1 7 Parent Responses Three parents completed questionnaires at the end of the school year. Their responses: One rated the program as “excellent,” the other two as “good.” All three saw their children as “much more interested in school and learning” since joining the academy. Parents had been involved with the program in several ways: attending school events, helping their child with homework, meeting with teachers and counselors, and helping with a program activity. Asked how well they had been kept informed of their children’s progress in school, one said “very well,” the other two “moderately well.” The features of the academy that parents liked best were the teachers’ individual attention and support of students, and the mentor program. Suggestions offered for improvements included the need for more mentor involvement and “finding a home for the program where the teachers aren’t against it.” No private sector representative questionnaires were received here; private sector involvement was mainly via the mentor program. Mentor Responses Five mentor questionnaires were received. Their responses showed that: On average, during the school year mentors got together with their proteges nine times in person, and talked by telephone 17 times. Three mentors visited their students at school; four brought the students to their places of work. Asked whether the academy’s curriculum would prepare students for jobs in local companies, four of the five respondents said “yes,” the other “I don’t know.” 1 1 8 PENINSULA ACADEMIES 1987-88 EVALUATIONS 0 Asked to rate the mentor program, two said “highly useful,” and three said “somewhat useful.” 0 The program’s biggest problem was seen to be the need for more contacts among mentors and between them and teachers, beyond the initial student match-up meeting. - The greatest strengths of the program were seen to be the personal relationship a student could enjoy with a role model, the networking between schools and community, and the positive effect on school retention. Summary While the student outcomes analysis continued to show significant program impact on participants, the differences discovered in the way credits and grades are allocated between Rio Cazadero, a continuation high school, and the comprehensive high schools from which the comparison groups were drawn, throws these differences into doubt. The number and size of differences favoring the academy students may still indicate some improved performance on their part, but there is no way to determine this precisely, and therefore no conclusions in this direction can be drawn. Both student and adult questionnaire feedback reflect the increasing problems of this program during the 1987—88 school year, which were present but less predominant during the previous two years. While there remains some positive feedback, particularly about the close family-like atmosphere of the program, and the mentor program supported by the local Rotary Club, on balance this academy gives cause for disappointment. What appeared to be a healthy and effective program disintegrated, and even the evidence of its earlier success came into question. It can only be hoped that the program will be reestablished at Elk Grove High School and will become effective there. The Santa Barbara Electronics Academy San Marcos High School Santa Barbara High School District Santa Barbara, California The academy at San Marcos High School began in the fall of 1987 with relatively low enrollment and some disappointments regarding equipment for its electronics lab. Although the program’s official enrollment of sophomores was 19, by the end of the school year only 11 students remained in the program. The electronic equipment which was to be paid for through Regional Occupational Program funds came very late in the school year, which detracted from the quality of instruction for that class. Still, despite the slow start the San Marcos Academy appears to show great promise. As indicated below, in 1987—88 academy students at San Marcos outperformed the comparison group in all areas, with the positive differences in attendance and credits reaching statistical significance. Enrollment in 1988—89 jumped significantly, with 27 sophomores and 12 juniors participating. Both the new lab equipment and a change in electronics instructors have also boosted the program. School-level support for the academy is particularly high, given the continuing personal involvement of the principal. District support appears to be less strong and this comes in pan from the fiscal realities of declining enrollment both in the district as a whole and at San Marcos High. During the first year of operations, private sector support was not greatly visible, but much was done to enlist future support. The academy uses the existing Santa Barbara Indusz Education Council for its private sector advisory body, and that council is an official sponsor of the program. The electronics theme was chosen for the program because of the existence of a substantial electronics industry in the area, much of which is concentrated at an industrial park just minutes away from San Marcos High. Setting San Marcos is one of three large comprehensive high schools in its district. It is located 10 miles northwest of downtown Santa Barbara and was built in 1958 on a 40-acre parcel. It 120 PENINSULA ACADEMIES 1987-88 EVALUATIONS is a sprawling, suburban campus, with an outdoor amphitheater capable of seating the entire student body. Each of the three schools in the Santa Barbara High School District faces continuing enrollment decline, although San Marcos has experienced the most dramatic drop in recent years—from more than 2,000 students in 1982-83 to about 1,500 in 1988—89. The enrollment decline is expected to continue for another five years before beginning to turn around in the mid 1990s. One reason for the decline is the greatly inflated housing prices in virtually all of Santa Barbara City. New families with children cannot afford to move in; families with children already owning homes are selling to move to the northern part of the county where a comparable house may cost 30 to 40 percent less. The student body at San Marcos is largely white and middle class. In the fall of 1987, whites made up 66 percent of the student body, Hispanics were 27 percent, Asians were 4 percent, and blacks were 2 percent San Marcos’s socio—economic status percentile (based on CAP testing rankings) is 85 percent—third highest among all academies. Percentile scores on the CAP tests in 1986—87 were 71 percent in reading and 84 percent in math. School principal Bob Ferguson estimates that well over 70 percent of graduates go on to higher education. Student Profile Since San Marcos is a grade 9—12 school, recruitment for the academy is done among its own 9th graders. Presentations were made either in English or social studies classes in the Spring of 1987 to gain interest in the program. The criteria used for admission to the program were consistent with academies across the state: San Marcos aimed to serve students (a) somewhat behind on standardized tests but with potential, (b) with a history of school alienation as manifested by attendance and behavior problems, and (c) with an interest in electronics. As the table below indicates, the program attracted a disproportionately large number of Hispanics—not surprising since Hispanics do make up a larger percentage of at-risk students in the school. As already noted, first-year recruitment resulted in a 19-student sophomore class which, through attrition largely caused by transfers out of San Marcos, diminished to 11 students by the end of the year. For 1988-89, 27 new students were recruited to the new sophomore class. THE SANTA BARBARA ELECTRONICS ACADEMY 1 2 1 TABLE 1. Student Profile, San Marcos Academv Comparison Group Class (N) Sophomores 19 3 1 Race/Ethnicity (%) White 21 28 Black 5 3 Hispanic 68 59 Asian 5 9 Other Gender (%) Male 63 68 Female 37 32 Program Management Principal Bob Ferguson is clearly responsible both for an academy being created at San Marcos and for on-going management of the program. Given its small size in the first year, he handled virtually all the administrative side of the program in 1987—88 and continues in that role in 1988—89. Four different teachers work with academy students who are block scheduled into academy English, math, social studies and electronics. They share a common planning period (although they do not have an extra period for planning) and meet frequently to discuss their students. Three of the four teachers who began with the program in 1987—88 continued in 1988—89; only the electronics teacher was replaced. Private sector representation, as mentioned, comes through a very active Indusz Education Council. Ferguson meets with them fairly often and keeps them well informed about the program. In 1988—89, mentors were provided for virtually all 11 juniors. Local 1 22 PENINSULA ACADEMIES 1987-88 EVALUATIONS companies have also provided limited financial support and donated equipment to the program. Academy students also participate in field trips to electronics companies. The elecuonic lab serves as the center for the program, with other classes scattered throughout the school. There is no office space for the program, although this may change as it grows. Program Resources/Costs District Support An additional 1.1 full-time-equivalent teachers are employed by the academy. The total number of FTE academy teachers is 2.2. There are four staff development days which are used for academy staff meetings and visits to other academy programs. Private Sector Support Employers’ representatives donated approximately 88 days to the program during the year. The bulk of these days (45) were spent for advisory committee meetings. Twelve days were for speaking to groups, six for organizing and leading field trips, 11 for serving as mentors to individual students, three for arranging job placements, and 11 for supervising students on the job. The academy made use of the following additional resources: - Fagilities. One classroom provided by the school. 0 Eguipmpnt. Computers, worktables, workbooks, and lab volt supplied through the academy, school funding, and business. - Suppligs. Miscellaneous supplies needed for teaching the academy classes were supplied by the academy and the school. . Transportation. Bus transportation as needed was supplied by the academy funding and the school. THE SANTA BARBARA ELECTRONICS ACADEMY 1 2 3 ’ MW. Numerous dinners, lunches, special trips, etc. were funded by the academy funding and the school. Student Outcome Data The most objective data by which to judge the program’s effectiveness are measurable indicators of student performance. These include attendance, credits earned, grade point average, courses failed, and retention in school (the opposite of dropouts). While these are not the only indicators of the program’s effect on students, they provide the most statistically convincing measures. There are data from just one year by which to judge the program’s performance: that from the 1987—88 first cohort of San Marcos Academy students. Table 2 presents differences in outcomes between first cohort academy students and their matched comparison group. These differences have been adjusted to account for differences between the two groups in age, gender, race or ethnicity, and academic performance in the preceding year. In this table, positive numbers for “attendance,” “credits,” and “GPA” mean that academy students compiled better records than the comparison group. Negative numbers for “courses failed” and “dropout probability” also mean that academy students performed better than the comparison group. TABLE 2. Adjusted Differences Between Academy and Comparison Groups, by Class, San Marcos Atten- Courses Dropout dance Credits GPA Failed Probability Cohort entering fall 1987 (sophs): 1987—88 Outcomes 4.4* 8.4#* 0.4 —O.7 i # Estimated from separate regressions for academy and comparison groups i Insufficient number of dropouts for analysis * Statistically significant 124 PENINSULA ACADEMIES 1987—88 EVALUATIONS Of the four differences between academy students and comparison groups shown in this table, all favor the academy students. Two of these are statistically significant. Academy students in their first year outperformed the comparison group in attendance and credits. With respect to dropouts, because the actual number (as opposed to the percent) of students who drop out is relatively small when analyzed by individual cohort, few statistical tests of differences between academy and comparison groups will prove to be significant, even when differences in the attrition between the two groups is substantial. For this reason, the following table presents the actual differences between the two groups. The overall “attrition” (what is usually meant when the term “dropout” is used) is broken out into its three components: known school dropouts (official withdrawals), transfers to other high schools, and probable dropouts (students lacking records as of June 1988 who neither officially withdrew nor transferred). TABLE 3. Known Dropouts, Probable Dropouts, and Transfers, for Academy and Comparison Group Students Academv Comparison Group Known Prob. Trans- Known Prob. Trans- Drops Drops fers Drops Drops fers Sophomores 0 O O O O 0 Obviously this table reflects no differences between academy and comparison group students. School Personnel Responses At the end of the year, three teachers completed questionnaires about the program. Their responses include the following: THE SANTA BARBARA ELECTRONICS ACADEMY 1 2 5 Non-academy teachers were rated “very” to “slightly” supportive of the program, counselors “very supportive,” and administrators “very” to “somewhat” supportive. Parents were rated “somewhat supportive” to “unsupportive.” - Other aspects of the program received various ratings. Facilities were “excellent” to “good,” equipment “good” to “average,” books and other learning materials “good” to “poor.” - The overall program planning and management for the year was rated “good” to “average” with comments that it was “good for a beginning year,” and “better recruitment and more coordination are necessary.” 0 The level of cooperation and interaction between the private sector representatives and academy was rated “somewhat” effective with suggestions for more field trips and site visits. - Counseling for school-related problems and planning future careers received one each “good,” “poor,” and “don’t know.” 0 The greatest weaknesses of the program were seen as “first year program,” “students show little or no interest and are poorly representing the potential of the academy,” and “need a teacher with sufficient release time to supervise and manage extra workload.” - The greatest strengths of the program were seen to be “good teachers,” “good potential support from high tech community,” “student camaraderie,” “mentors,” and “getting jobs for students.” - On a seven point scale (1 = very weak, 7 =v ery strong), the academy received an average rating of 6.3 “in concept” and 4.0 “as implemented.” Parent Responses Four parents completed questionnaires at the end of the year. Their responses included the following: o The overall quality of the program received two “good” ratings and two “don’t know.” 12 6 PENINSULA ACADEMIES 1987—88 EVALUATIONS 0 All three noticed their children had more interest in school and learning, evidenced by more studying and better grades. 0 Parents were involved in the program by attending open house, helping children with homework, and meeting with teachers and counselors. - Three parents felt they were “moderately” well informed about their children’s progress and one felt not informed at all. Two were “moderately” informed about the special program activities and two were not informed at all. Private Sector Representatives Responses Four private sector representatives completed questionnaires at the end of the school year. Their responses included: , ° The companies were involved with the academy program by helping to develop a brochure, giving company tours, donating equipment and supplies, serving on the steering committee, and keeping the program technology current. - Three responded “yes” and one “I don’t know” when asked if academy curricula was providing students with skills needed by local private business. 0 All were satisfied with the current procedures and criteria for selecting students for the academy. - All rated job motivation and enthusiasm, dependability, and ability to get along with others as “very important” factors that might play a role in hiring a high school graduate. - The mentor program, summer employment positions, and school—year employment positions were all rated “highly useful” student activities. - Comments on satisfaction with academy students as employees included: “highly motivated and dedicated to learning a skill,” “once committed, it is a pleasure to observe the eagerness they exhibit when acquiring new knowledge and skills,” and “the caliber of the students and their dependability have exceeded my expectations.” - The steering committee received three “good” and one “excellent” ratings. THE SANTA BARBARA ELECTRONICS ACADEMY 1 2 7 ° Quality or communication between the academy and private sector was rated “excellent.” ° The greatest weakness of the program was seen as funding. The greatest strengths were “the eager staff and students.” ° On a seven point scale (1 = very weak, 7 = very strong), the academy was rated 6.25 both “in concept” and “as implemented.” Summary The San Marcos Academy experienced many of the typical start-up problems of a new academy, but survived its first year with good promise for the future. Although the number of students in the program was very low, those who did participate outperformed the comparison group, particularly on attendance and credit. While private sector support was relatively light during the academy’s first year, groundwork was laid for the future and private sector representatives appear to be very supportive and willing to act when asked. Enrollment gains in 1988—89 are promising for an academy which will eventually have an enrollment large enough to receive the maximum state support available. But several aspects of the academy could improve over the next two or three years as the program grows. As program enrollment expands, teachers can play a larger role in the program by teaching more academy classes and thus increasing their commitment to the program. The principal will need to transfer many aspects of the administration of the program to a lead teacher, and the selection of such a teacher deserves careful attention. Private sector support will need to increase substantially as the numbers of mentors and jobs needed increase. indeed the private sector needs to become the major champion for the program. chapter 3 Cross-Site Analyses and Conclusions This is the third year of this evaluation. Information in the report covers the performance of the California Partnership Academies during the 1987—88 school year, and builds on data gathered during the previous two school years. This chapter summarizes the information across the twelve academies studied during the past year. This summary is divided into the following sections: 0 Academy implementation. including: enrollments; students’ socio-economic status; program implementation; and resource allocations - Student outcomes, including: standardized test scores; statistical tests of differences between academy and comparison groups on attendance, credits, grade point averages, and courses failed; high school dropouts; and student questionnaire feedback 0 Conclusions and recommendations Academy Implementation Enrollments Table 3.1 presents enrollment figures across the eleven academy sites (Inglewood, which was still largely at a planning stage, is not included) for the 1987—88 school year. The first column presents the number of students at each site, and subsequent columns present breakouts for gender and race/ethnicity. Totals are presented at the bottom of the table. The headings for race/ethnicity are as follows: A — Asian/Pacific Islander B —— Black H —— Hispanic W —— White 0 —— Other 129 13 O PENINSULA ACADEMIES 1987—88 EVALUATIONS TABLE 3.1 Academy Enrollments Statewide, 1987—88 Gender (%) Race/Ethnicitv (%) Site* # M F A B H W O Bakersfield 79 39 61 O 11 54 33 1 Hiram Johnson 100 50 50 9 3O 20 37 3 Independence 98 64 36 10 13 36 37 4 Menlo—Atherton 104 56 44 5 45 27 19 4 Mountain View 44 67 33 3 12 12 68 5 Oak Ridge 39 52 48 O 0 O 100 0 Oakland Tech 92 35 65 10 79 2 5 4 Rio Cazadero 54 48 52 3 9 12 70 6 San Marcos 19 63 37 5 5 68 21 O Sequoia 124 54 46 3 11 39 46 1 Silver Creek 93 56 44 5 25 42 26 2 Total 846 52 48 5 26 29 36 3 As this table shows, there were 846 academy participants this year. This figure represents the number enrolled at the end of the school year, discounting those lost to attrition. The academy participants were fairly equally distributed between males and females. In terms of racial/ethnic background, 63 percent of academy participants were minority, primarily black and Hispanic. Soda-economic Status Several items on the student questionnaire ask academy participants to describe their family living arrangement and the educational and employment status of their parents or guardians. The table that follows provides a summary statewide of responses to these items, giving a CROSS-SITE ANASYSES AND CONCLUSIONS 1 3 1 picture of the socio-economic status of academy participants. These figures are presented separately by cohort: those students who entered the program in the fall of 1985, 1986, and 1987. While not all students completed the student questionnaire, enough did so as to give a fairly accurate picture on this dimension. TABLE 3.2 Socio-Economic Status of Academy Participants Cohort 1 Cohort; Cohort 3 With what adults do you live? Mother only 18% 20% 27% Father only — 4% 3% Mother and father or one parent and 70% 55% 57% another adult One or more other adults, not mother 6% 12% 5% or father Mother and father and other adults 4% 5% 6% Highest grade level completed by adults in household Eighth grade or less 5% 9% 8% Some high school 14% 7% 13% High school graduate only 24% 30% 20% Some college 30% 26% 31% College graduate or more 28% 28% 27% Number of adults with paid job outside the house (mean figures) In household with one adult .79 .83 .86 In household with two adults 1.62 1.52 1.57 Number of adults with undesired unemploy- ment in the past year (mean figures) In households with one adult .22 .22 .25 In households with two adults .25 .18 .28 Program Implementation One central question in this evaluation is the degree to which each of the replications is successful in implementing the academy model. This is a rather complex model, requiring not only the school-within-a—school structure, but a clearly defined student selection process, a variety of private sector involvement, and certain well—defined types of school 1 3 2 PENINSULA ACADEMIES 1987—88 EVALUATIONS and district support. The Partnership Academy “Checklist of Components” that follows provides a list of twenty-seven such components that together define what an academy is. Partnership Academies Checklist of Components YES NO PARTIAL Student Selection Defined Criteria For Program Entry Aimed at high-risk Students Recruitment and Application Process Appropriate Numbers Entering Yearly Program Structure School—Within-a-School S tructure 1 Technical Class, grades 10-12 3 Academic Classes, grades 10-11 (English, Math, Sci./S.S.) Block Scheduling Coordination bet. acad. & voc. curr. II II Private Sector Support Business-Educ. Advisory Body Input to technical curriculum School speakers Field trips to businesses Mentor Program—Grades 11 & 12 Avail. of Jr. Work Experience, Sumr. Avail. of Sr. Work Experience, Sch-Yr. Graduate Placement Assistance | llllllll llllllll District and School Support Dedicated Academy Teachers Common Extra Teacher Prep. Period Counselor Support Administrator Support Adequate Facilities, Equip. Adequate Curricular Materials llllll Miscellaneous Staff Orientation, Development Parental Contact Adequate Career Planning Program “home base” room CROSS-SITE ANASYSES AND CONCLUSIONS 1 3 3 This checklist is used in the site visits made to each academy each spring to determine the degree to which the program has implemented the full model. A point system is then used to give each academy a rating. A full point is awarded for full implementation of each component. A half point is awarded if the component is partially achieved (e. g., some but not all academy teachers have an extra preparation period; some but not all students are provided with a mentor). If the component has been achieved little or not at all, no points are awarded. Across the five categories in the checklist there are twenty-seven components, meaning the best possible score is twenty-seven (less if the program is not yet at its third year, and full stage of implementation—see the following table). Table 3.3 that follows provides the ratings each of the academies received for the 1987—88 school year. It should be noted that these ratings are to some degree based on the judgments of the evaluator and the observations made in the site visit and other discussions with program staff. Every effort has been made to be fair and equitable in these ratings, but these are individuals’ judgments. The column headings in the table stand for the following: - Stud. Sel. —- Student Selection 0 Prog. Str. -— Program Structure 0 PS Sup. — Private Sector Support - Adm. Sup. — School and District Administrative Support - Misc. — Miscellaneous As this table shows, no academy receives a perfect implementation score. There is a broad range of scores, with some sites receiving nearly perfect scores, and others failing to implement many components of the model. Of particular interest later in this chapter is the correlation seen between these implementation ratings and the results of the student outcomes analyses. Resource Allocations One way of judging the level of effort put into implementing the academy in each site is to examine the amounts and types of resources provided the program. Often evaluations measure precisely the “outcomes” of a program, but not the “inputs.” To obtain information on this subject, each site was asked to complete a program “Resource Form.” This asked for information in three categories: 13 4 PENINSULA ACADEMIES 1987-88 EVALUATIONS TABLE 3.3 Implementation Scores, by Site Site Stud, Sel. Frog. Str. PS Sup. Adm. Sun. Misc. Total Maximum Possible 4 5 8 6 4 27 Bakersfield 4 4.5 5 5 3.5 22 Hiram Johnson 4 4 5 5.5 3.5 22 Independence 4 4.5 6* 6 3.5 24* Menlo-Atherton 4 4 7.5 5 4 24.5 Mountain View 3.5 4 4 3 2 16.5 Oak Ridge 4 2.5 1.5 3.5 2.5 14 Oakland Tech 4 5 8 5 .5 4 26.5 Rio Cazadero 4 2 3 2.5 3 14.5 San Marcos 3.5 5 2** 4.5 3 18** Sequoia 4 4.5 7 5 4 24.5 Silver Creek 4 4.5 6* 6 3.5 24* "' Maximums possible are 2 less in these cases, due to the stage of the academy ** Maximums possible are 4 less in these cases, due to the stage of the academy - School staff, in particular the FTE teacher time dedicated to the academy over and above regular school allocations (to cover the smaller class sizes and extra preparation periods), as well as aides, administrator time, and staff development - Private sector involvement, including advisory committees, business speakers, field trips, mentors, and job placement and supervision - Non—personnel resources, such as facility improvements, equipment, supplies, transportation, and rewards and social events CROSS-SITE ANASYSES AND CONCLUSIONS 1 3 S The information provided by each academy rests on the perceptions of staff who completed the form, and should not be treated as absolutely precise, although one can gain from it a reasonably accurate picture of resources being provided each program. To avoid presenting detailed numerical summaries, the table below provides ratings of each academy in each of the three categories defined above. “Strong” ratings mean that all or almost all resources needed in that category have been provided; “weak” ratings indicate that few if any have been; “average” ratings mean the program is somewhere in between. Detailed descriptions of these resources are provided in each of the case studies in Chapter 2. TABLE 3.4 Resource Allocation Ratings, by Site Jchool/District Private Sector Non-Personnel Sites Str. Avg. Weak Str. Avg. Weak Str. Ava. Weak Bakersfield x i. x x Hiram Johnson x X X Independence x x x Menlo—Atherton x x x Mountain View x x x Oak Ridge x x x Oakland Tech x x x Rio Cazadero x x x San Marcos x x x Sequoia x x x Silver Creek x x x There are a few generalities that apply to most of the academies in terms of resource allocations. One is that school-level administrative support tends to be stronger than 1 3 6 PENINSULA ACADEMIES 1987—88 EVALUATIONS district-level support. Particularly in large districts, the academy tends to get lost in the bureaucracy, a problem that needs to be addressed in the replications generally. Another common feature is that academies struggle with providing enough personnel time to develop and coordinate all the private sector involvement. This is a substantial job, particularly during the second and third years when the mentor and job programs come into play. While companies are often willing to provide the personnel time for these, they are not usually ready to organize and administer them. As the above table shows, most of the “weak” ratings occur in the category of private sector support. Again, this is a problem that needs to be addressed generally in the academy replications. Student Outcomes Standardized Test Scores In the spring of both 1987 and 1988, the evaluators administered the Comprehensive Test of Basic Skills to academy students. The Basic Skills Tests in Reading and Math, Form U, Level I were used. Tests were not administered to comparison group students. Tables 3.5 and 3.6 present the results of this testing for reading and math, respectively. For each cohort at each school, the mean score and number tested (N) is given, in national percentile form. In addition, a “demographic norm” is provided, showing the mean score expected for students from a school with the socio-economic composition of that school. Several comments are relevant to Tables 3.5 and 3.6. The norm scores vary substantially from site to site, reflectin g the variation in schoolwide test score averages at the various sites. There is also substantial variation in academy student scores, and in these scores relative to the schools’ norm scores. About half of the academy students’ scores are below the schools’ norm, and about half above them; about half show improvement from one year to the next, both in terms of their own percentile and relative to the norm, and about half decline. Two factors make it difficult to clearly interpret these scores. First, there was considerable attrition from the first year to the second among the students tested in many of the cohorts, meaning they represent comparisons among differently constituted groups. Second, to some degree the scores reflect selection procedures used in each academy. In higher SES high schools, students with among the poorest academic records are usually selected, while in lower SES schools, more average students are selected. And it is in higher SES schools such as Menlo-Atherton, Mountain View, and Oak Ridge that academy students perform below the school‘s norm, and in low SES schools such as Hiram Johnson and Oakland Tech that they perform above the norm. CROSS-SITE ANASYSES AND CONCLUSIONS 1 3 7 TABLE 3.5 CTBS Mean Reading Scores for Academy Students, Spring 1987 and Spring 1988. . NP (N) NP Norm NP (N) NP Norm Sites SD, 87 86-87 SD. 88 87-88 Bakersfield Cohort 1 23 (15) 37 30 (12) NA Cohort 2 27 (28) 34 32 (19) 37 Cohort 3 —— 23 (23) 34 Hiram Johnson Cohort 1 34 (36) 31 27 (25) NA Cohort 2 34 (35) 29 42 (21) 31 Cohort 3 —- 35 (16) 29 Mountain View Cohort 1 32 (11) 64 Too Few to Analyze Cohort 2 3O (14) 62 25 (15) 64 Cohort 3 -— 30 (13) 62 Oakland Tech Cohort 1 52 (25) 30 47 (11) 29 Cohort 2 55 (21) 28 6O (13) 3O Cohort 3 — 58 (32) 28 Oak Ridge Cohort 1 54 (13) 64 49 (14) NA Cohort 2 27 (14) 62 38 (9) 64 Cohort 3 — 36 (8) 62 Rio Cazadero Cohort 1 Too Few to Analyze Cohort2 62 (18) 48 58 (11) 51 Cohort 3 —— 57 (11) 48 Independence Cohort 1 37 (87) 37 37 (63) 39 Cohort 3 —— 43 (21) 37 Silver Creek Cohort 1 20 (68) 28 3O (42) 30 Cohort 3 — 20 (39) 28 Menlo Atherton Cohort 1 Not included in evaluation 34 (23) 42 Cohort 2 " 25 (23) 44 Cohort 3 " 42 (31) 4O Sequoia Cohort 1 Not included in evaluation 36 (28) 39 Cohort 2 " 31 (26) 4O Cohort 3 " 21 (31) 37 13 8 PENINSULA ACADEMIES 1987-88 EVALUATIONS TABLE 3.6 CTBS Mean Math Scores for Academy Students, Spring 1987 and Spring 1988. NP (N) NP Norm NP (N) NP Norm Sites Sp. 87 86-87 SD. 88 87-88 Bakersfield Cohort 1 25 (15) 39 3O (12) NA Cohort 2 3O (28) 9 33 (20) 39 Cohort 3 — 35 (23) 39 Hiram Johnson Cohort 1 45 (29) 31 41 (25) NA Cohort 2 26 (34) 32 33 (21) 31 Cohort 3 -— 3O (16) 2 Mountain View Cohort l 43 (12) 64 Too Few to Analyze Cohort 2 48 (14) 64 43 (15) 64 Cohort 3 — 41 (13) 64 Oakland Tech Cohort 1 61 (26) 30 54 (13) 28 Cohort 2 57 (19) 3O 49 (13) 30 Cohort 3 — 64 (31) 30 Oak Ridge Cohort 1 31 (11) 6O 35 (14) NA Cohort 2 12 (13) 6O 4O (8) 60 Cohort 3 — 48 (12) 60 Rio Cazadero Cohort 1 Too few to Analyze Cohort 2 35 (18) 45 25 (10) 47 Cohort 3 -— 26 (10) 45 Independence Cohort l 62 (86) 48 62 (63) 47 Cohort 3 —— 62 (21) 48 Silver Creek Cohort 1 37 (67) 34 35 (44) 35 Cohort 3 — 37 (39) 34 Menlo Atherton Cohort 1 Not included in eval. 58 (25) 47 Cohort 2 " 51 (23) 47 Cohort 3 " 71 (31) 48 Sequoia Cohort 1 Not included in eval. 38 (28) 45 Cohort 2 " 58 (26) 47 Cohort 3 " 30 (30) 45 Accordingly, these scores should probably not be interpreted as clear reflections of program effects. The academy programs do include basic academic courses combined with technical training, which coupled with improvements in motivation might influence standardized achievement test score performance. However, such influence seems likely to CROSS-SITE ANASYSES AND CONCLUSIONS 1 3 9 selected, while in lower SES schools more average students are likely to be slight, and given all the interpretation problems evident, there is little one can conclude clearly about the programs from these data. Summary of Statistically Significant Adjusted Differences in Student Outcomes Table 3.7 that follows presents the differences between academy and comparison students on outcome measures that were statistically significant during the past three years. The statistical procedure for testing significance is explained in Appendix A. The basic approach is to adjust the raw differences in outcomes reported above in order to correct for pre-existing differences between academy and comparison students in prior school performance, gender, race or ethnicity, and date of birth. These pre-existing differences between academy and comparison groups were partially eliminated by the procedure for selecting matched comparison groups. By adjusting for any remaining differences between academy and comparison students, it is possible to test more precisely whether differences in outcomes are due to the academy experience itself, rather than to pre-existing differences between the two groups of students. The statistical test used actually measures year-to-year changes in performance, once pre-existing differences have been taken into account. Thus a “plus” sign in Table 3.7 may indicate that the academy group improved significantly more than the comparison group, declined significantly less, or improved while the comparison group declined. A minus sign means the same for the comparison group. The actual numbers are shown in the case studies in Chapter 2. One way to read this table is to compare rows. This reveals which sites had more success for academy students relative to the comparison groups. Some academies, such as those at Independence, Oakland Tech, Rio Cazadero, and Silver Creek show clearest evidence of positive effects on students (although the findings at Rio Cazadero are questionable, due to statistical inconsistencies in the way the data exist there, as explained in the chapter on that site). No academies show consistent evidence of negative effects, with the possible exception of Cohort 2 at Bakersfield (although a poor match between the program and comparison group for this cohort throws this finding into question; this is also explained in the chapter on this site). Another way to read this table is to compare columns. This reveals that the academy programs have fairly consistent effects on the four in-school outcome measures: attendance, credits, grade point average, and courses failed. While there are few statistically significant differences shown on the fifth outcome measure, dropout 140 PENINSULA ACADEMIES 1987—88 EVALUATIONS TABLE 3.7 Summary of Statistically Significant (p < .05) Adjusted Differences in Outcomes Between Academy and Comparison Groups Atten- Courses Dropout Impl. dance Credits GPA Failed Probabilitv Rating CM]. 1985-86 Outcomes No Implemen- Bakersfield + tation Franklin + + Rating Hiram Johnson + + this Independence + + Year Mountain View + Oak Ridge + + Oakland Tech + + + + Rio Cazadero + + + + Silver Creek — 1986—87 Outcomes Max=25 Bakersfield 18.5 Franklin — 18 Hiram Johnson 21 Independence + + + + 23 Mountain View 18 Oak Ridge 10.5 Oakland Tech + + 22 Rio Cazadero + + 19 Silver Creek 22.5 1987-88 Outcomes” Max=27 Bakersfield 22 Hiram Johnson 22 Independence 24 Menlo—Atherton 24.5 Mountain View — 16.5 Oak Ridge — 14 Oakland Tech - 26.5 Rio Cazadero — — + 14.5 Sequoia 24.5 Silver Creek + + 24 CROSS-SITE ANASYSES AND CONCLUSIONS 1 4 1 TABBE 3.7 Atten- Courses Dropout Impl. (continued) dance Credits GPA Failed Probabilitv Rating £22m 1986-87 Outcomes Max=25 Bakersfield 18.5 Franklin + + 18 Hiram Johnson + + 21 Mountain View 18 Oak Ridge + 10.5 Oakland Tech + + + + 22 Rio Cazadero + + + + 19 1987-88 Outcomes** Max=27 Bakersfield — — — — 22 Hiram Johnson 22 Menlo-Atherton . 24.5 Mountain View 16.5 Oak Ridge 14 Oakland Tech + 26.5 Rio Cazadero + + 14. 5 Sequoia + 24.5 1987—88 Outcomes Max=27 Bakersfield 22 Hiram Johnson 22 Independence + + 24 Menlo-Athenon + 24.5 Mountain View 16. 5 Oak Ridge 14 Oakland Tech + + + + 26.5 Rio Cazadero + + 145 San Marcos + + 15.5 Sequoia 245 Silver Creek + + + + 24 * Independence and Silver Creek added no new cohort this year. ** Menlo-Atherton and Sequoia were added this year, while Franklin, where the academy was terminated, was dropped. *** San Marcos, which replaced Franklin, was added this year. 14 2 PENINSULA ACADEMIES 1987—88 EVALUATIONS probability, this is due primarily to the inability of the statistical test on this measure to reach significance because of the small number of cases in each site. The actual differences in dropouts between academy and comparison group members are presented in the next table (Table 3.8), and discussed there. A third comparison can be made in this table between cohorts and years. Each cohort (class of students) enters at grade 10 and proceeds through three years. Thus there are three years of data on the first-year effects, two on second-year effects, and one on third- year effects. There is a clear pattern of results associated with each program year. Of the 108 possible differences between academy and comparison group students in the first program year (among attendance, credits, GPA, and courses failed), there are 47 that are significantly positive and only one that is significantly negative. Of the 64 possible differences in the second program year, 11 are positive and four are negative. Of the 40 possible differences in the third program year, three are positive and five negative. Thus, while across the three years there are 61 statistically significant positive differences and 10 negative ones, there is a strong first—year effect that declines in the second year and disappears altogether in the third. There are a number of possible explanations for this. One is that the program is concentrated primarily in grades 10 and 11, where the model calls for three academic and one technical class. In grade 12, the academic classes drop out, so there is considerably less program “treatment.” Academy teachers also point out that there is a “senioritis” phenomenon operating that year, even among academy students, which results in less devotion to classes and grades as seniors look to their post-graduate plans. They also note that it is relatively rare for students to make it to the senior year and still drop out, and so they tend to target more effort on grades 10 and 11, where dropouts are more common. It is also at grade 10 that the program has its initial impact, and students experience the school-within-a-school structure and other academy features for the first time. It may be that the effect of these program structures gradually wears off after the first year. An alternative explanation for this decline in program effect is the analysis itself. Since each year’s assessment is based on the relative change in the academy and comparison students’ performance from the previous year, a sharp improvement in academy students’ performance in grade 10 makes it more difficult for them to show relative improvement thereafter. A review of the actual mean scores across the three years suggests that this does explain a portion of the seeming decline in program effect, but not most of it. The decline in program effect over time is thus largely real. One message this suggests for the academies is the need to maintain the program’s intensity for the full three years. CROSS-SITE ANASYSES AND CONCLUSIONS 1 4 3 Dropouts The fifth outcome measure tracked in this evaluation, and perhaps the most important one, is dropouts; While the statistical tests used in the evaluation did not find statistically significant differences between academy and comparison students on this dimension in most sites because of the relatively small number of students who fall into this category, it is nevertheless important to examine the dropout data. Table 3.8 that follows presents these data. It is difficult to gather precise dropout numbers. Students who leave high school often do so without any official notification to the school. Sometimes they transfer, and schools know they have re-enrolled elsewhere when the student requests his or her transcript be sent to the new high school. Sometimes they do notify a teacher or counselor that they are leaving and not re-enrolling elsewhere. And sometimes they just stop coming. To compile the most accurate information possible, we decided to track data in three categories: - Dropouts—those known to have left high school . Probable dropouts——those who did not officially notify the school of their departure, but for whom no spring semester records exist and no transcript has been requested . Transfers—those who have left and requested a transcript elsewhere The total of these three equals the “attrition” figure for each cohort, or the total number of students who began in the 10th grade and have since left. Since the figures in the table are cumulative across years, this means that for Cohort 1 they cover grades 10-12, for Cohort 2 grades 10—1 1, and for Cohort 3, grade 10. Table 3.8 shows several things. First, most students who leave high school do so because they transfer elsewhere. Thus, attrition figures that fail to distinguish between transfers and dropouts give falsely high estimates of dropouts. At the same time, the number of students who leave the high school in which they begin tenth grade is high. Among academy students the attrition figure over three years for the first cohort is 32.9 percent, and for comparison group students it is 47.7 percent. However, the known plus the probable dropout totals across the three years are 7.3 percent for academy students and 14.6 percent for the comparison group. This suggests that the academies, on average and across three years, are reducing the dropout rate by about half, and the attrition rate by about one-third. 144 PENINSULA ACADEMIES 1987-88 EVALUATIONS TABLE 3.8 Known Dropouts, Probable Dropouts, and Transfers, Across Sites and Cohorts Academv Maximum Drop- Kn own Prob. Trans- Known Prob. Trans-. out Drops Drops fers Drops Drops fers Diff. Cohort l Bakersfield 9% 7% 27% 7% 2% 19% -7% Hiram Johnson 9% 0 24% 35% 8% 31% +34% Independence 3% O 16% 5% 0 21 % +2% Menlo—Atherton l 1% 0 19% 1 1% 0 52% 0 Mountain View 0 0 63% 6% 2% 42% +8% Oak Ridge 0 0 29% 0 8% 54% +8% Oakland Tech 3% 0 10% 3% 15% 55% +15% Rio Cazadero 16% 0 44% 7% 2% 44% -7% Sequoia 14% 0 7% 21% 0 4% +7% Silver Creek 6% 6% 37% 7% 2% 30% -3% Statewide Average 6.6% 0.7% 25.6% 11.0% 3.6% 33.1% +7.3 Cohort 2 Bakersfield 3% 6% 14% 2% 2% 17% -5% Hiram Johnson 5% 15% 13% 24% 9% 17% +13% Menlo-Atherton 0 0 7% 7% 0 9% +7% Mountain View 0 0 45% 3% 2% 54% +5% Oak Ridge 0 O 57% 8% 0 68% +8% Oakland Tech 0 0 23% 3% 8% 24% +1 1% Rio Cazadero 7% 0 21 % 7% 7% 51% +7% Sequoia 8% 0 5% 16% 0 15% +8% Statewide Average 3.1% 3.5% 20.9 10.3% 4.0% 29.2% +7.7 Cohort 3 Bakersfield 7% 6% 21% 5% 0 5% —8% Hiram Johnson 5% 0 5% 6% 0 9% +1 % Independence 0 0 4% 2% 0 2% +2% MenIo-Atherton O 0 0 O 0 0 0 Mountain View 0 7% 7% 0 0 0 -7% Oak Ridge 0 0 0 0 0 0 0 Oakland Tech 0 2% 12% 0 0 0 -2% Rio Cazadero 0 0 0 0 0 0 0 San Marcos 0 0 0 O 0 0 O Sequoia 3% 0 5% 4% 2% 8% +3% Silver Creek 0 0 3% O 0 5% 0 Statewide Average 1 4% 1.4% 6.3% 2.0% 0.2% 3.4% -0.6 CROSS-SITE ANASYSES AND CONCLUSIONS 1 4 5 Student Questionnaire Feedback Table 3.9 on the following two pages provides a summary of the pre-post questionnaire responses of academy students statewide. The “pre” version of the questionnaire is administered to sophomores at the point they enter the program. The “post” version is administered at the end of each school year, giving yearly updates on the items included. The pre questionnaire contains items on career-related plans, and attitudes toward career, school, and self. In addition to these, the post version also contains sections that allow students to describe program activities, and to rate various features of the academy. It is useful to remember that all information in this table reflects student perceptions only. The figures in Table 3.9 present an average from all academies statewide. Thus while they provide a sense of the activities and influence of the academies across sites, they do not reflect individual programs. Some are doing better than the table reflects, and some worse. The first two sections of the table, on program activities and ratings, come from just the “post” version of the questionnaire. For these, data are presented for each cohort for each year. For the sake of simplicity, the sections with both “pre” and “post” data (career-related plans, and attitudes toward career, school, and self) are shown with all pre and all post data collapsed across cohorts and years. Thus one can see the difference between the “total pre” and “total post,” and estimate the effect of the program on each measure statewide. Only data for those students for whom all data was available, both pre and post across all years, are included in the table. What does Table 3.9 show? Concerning program activities, it suggests that many of the academies are providing the full complement of three academic and one technical class, but not all. Most career and job information comes to students via their academy teachers, their parents, relatives, and friends, and school counselors/career center staff. While most of the first cohort had mentors their junior year, as intended, this fell off to less than half for the second cohort; about three-fourths of participants find this experience useful when it does happen. Very few students were obtaining jobs through the academies; this is clearly a weakness in implementation to date. Academies are doing better at providing business speakers and field trips, which are averaging several per semester statewide. Again, most students find these experiences useful. And a little over half of academy students are receiving the extra help they need with school work, most from academy teachers, friends, and parents. Student ratings of the academies indicate that most are fairly satisfied with the equipment and materials in the programs. About half like the academy schedule better than the regular school program, and this seems to drop off somewhat over time. About two- thirds of academy students see a strong connection between what they are studying in school and their post-graduate plans. And most like the academy better than their regular high school program. 14 6 PENINSULA ACADEMIES 1987—88 EVALUATIONS TABLE 3.9 Student Questionnaire Responses Cohort 1 41mm Son8LSD. 87 SD. 88 Son 87 So. 88 So. 88 How many academy classes are you currently taking? 4.0 3.8 3.1 3.2 2.6 3.3 Who have you asked about career or job information this year? a. Academy teacher 54% 42% 69% 51% 70% 63% b. Other high school teacher 4% 9% 6% 5% 5% 9% c. Parents or other relatives 50% 56% 42% 35% 28% 37% (1. Friends 30% 41% 36% 30% 30% 32% e. Academy mentor 4% 38% 44% 7% 12% 7% f. Employer 4% 6% 13% 5% 2% 7% g. School counselor 19% 30% 20% 21% 14% 23% h. School career center staff 24% 9% 17% 23% 28% 15% Students who had a mentor this year 7% 92% 71% 16% 40% 12% Of these, those who found it useful 99% 75% 75% 99% 75% 74% Students who had a job arranged through the program: Summer job 7% 4% 1% 9% 14% 4% School-year job 2% 1% 4% 2% 2% 4% Of these, those who found it useful — — — 50% 50% 31% How many guests from business spoke in your classes this year? Fall semester 5.7 5.0 5.7 2.1 2.6 4.0 Spring semester 4.5 4.5 6.0 2.3 2.6 4.6 Those who found speakers helpful 82% 82% 83% 71% 67% 82% How many field trips did you attend with the academy? Fall semester 3.5 3.3 2.9 .2 1.7 2.1 Spring semester 3.0 3.1 3.3 .5 1.4 2.6 Those who found field trips useful 80% 83% 77% 77% 82% 81% Did you get the extra help you needed with your school work? 41% 54% 66% 57% 57% 59% Who gave you this help? Teacher in class with problem 67% 61% 63% 88% 64% 62% Another academy teacher 23 % 13 % 2907 26% 3 1% 33% A peer tutor 4% 9% 7 '0 --— 10% 9% Another student or friend 41 % 52% 60% 29% 21% 32% Parent(s) 53% 48% 36% 26% 14% 23% CROSS-SITE ANASYSES AND CONCLUSIONS 1 4 7 Cohort 1 M h So. 86 So. 87 So. 88 So. 87 So. 88 SD. 88 Program Ratings Academy equipment good/excellent 82% 92% 82% 54% 61% 79% Academy materials good/excellent 86% 7 9% 83% 52% 59% 72% Academy schedule an improvement 65% 54% 38% 48% 36% 60% Felt strong connection between what studying and post-graduate plans — 72% 70% 80% 48% 68% Liked academy better than regular program 99% 83% 77% 77% 64% 84% r r-R l Pl . Total Ero Total Post Students with a definite plan for the first two years after graduation 29% 34% If have a plan, it includes: A job 36% 44% Junior college, college, or university 69% 77% Vocational or technical training 13% 14% Military service 13% 1 1% Marriage and housekeeping 3% 5% Students with a long-term career goal 54% 59% Attituoos Toward t2aroor, S ohool, and Solf Feel it is important to make career plans while in high school 69% 74% Feel it is important to gain job skills while in high school 79% 78% Found classwork interesting during last year 34% 56% Feel good about self after past year 71% 80% Would not want to be someone different 68% 72% Feel success is not mainly due to luck 74% 74% 14 8 PENINSULA ACADEMIES 1987-88 EVALUATIONS Only about a third of academy students have a definite plan for the first two years after graduation, and this increases only about 5 percent from pre to post responses. For those who do have such a plan, college is the most common objective (77%), followed by a job (44%), and both of these have increased 8 percent from pre to post responses. Slightly over half of participants have a long-term career goal, and this increases 5 percent pre to post, also. About three-fourths of academy participants feel it is important to make career plans and gain job skills while in high school; neither change much during the program. There is an increase of 22 percent in the number of participants who find classwork interesting after being in the academy; this is a significant change apparently due to the program. About three-fourths of students seem to have good self-esteem, slight increases from pre to post responses, and to feel success is not due mainly to luck, suggesting an internal locus of control. Benefit-Cost Analysis of Academies as Dropout Prevention Programs The Peninsula Academies and replications have been conceived primarily for the purpose of keeping likely dropouts in high school. Recent legislation authorizing continuation and expansion of the replication effort has explicitly included student retention as an outcome on which will depend the flow of state money to participating school districts. One reason for emphasizing this particular outcome is that graduation from high school—unlike grades, attendance, or other indicators of students’ performance while in school—has evident economic value, since it is well known that high school graduates generally do better than dropouts in the labor market. If the additional earnings of high school graduates can be attributed to their additional schooling rather than to pre-existing differences between graduates and dropouts, then these additional earnings can be used as a measure of the additional economic output produced by high school graduates. Here we present estimates of how many would—be dropouts in fact graduated from high school as a result of participating in one of these academy programs. We also estimate the cost per dropout saved, and compare it with the economic benefit, as measured by the average difference in earnings between high school graduates and non~graduates in the population at large. To estimate the number of dropouts saved in each academy program, we compared the actual number of academy students counted as known or probable dropouts with the number that would have occurred if academy students dropped out at the same rate as students in the comparison group. This analysis can be done only with students who were CROSS-SITE ANASYSES AND CONCLUSIONS 1 4 9 sophomores in 1985—86, since they would normally have graduated in June, 1988, when our most recent data were collected Table 3.10 shows the results for the eight academy programs that had students in this grade cohort. The predicted number of dropouts from each academy program is the dropout rate in the relevant comparison group, multiplied by the initial number of academy sophomores in 1985-86. In six of the eight academies, the dropout rate among the comparison group was higher than among academy students. The total number of dropouts saved was approximately 29, of whom 21 or 22 were at Hiram Johnson. Some readers may wonder why we used such a simple procedure to estimate the number of dropouts saved, rather than employing multivariate methods to control for prior differences between academy and comparison students, as we did in testing for effects on grades, attendance, numbers of credits and courses failed. The reason is that these outcomes can take on many more values than the high school dropout variable, which is either yes or no. Ordinary least-squares regression is appropriate to analyzing these many- valucd outcomes, and has better small-sample properties than procedures such as logit and probit, which are appropriate for analyzing binary outcomes but are valid only in large samples. Therefore, in estimating the numbers of dropouts saved, we use a simpler procedure. The validity of this procedure depends on the assumption that students in each comparison group were not systematically different from students in the corresponding academy group, prior to their sophomore year. TABLE 3.10 Estimated Numbers of Dropouts Saved from Cohort Entering Academics as Sophomores 1985-86, and Cost per Dropout Saved No. of dropouts Number saved Cost per dropout saved Actual Predicted Predicted—Actual To societv To taxpavers Academy Bakersfield 9 5.6 - 3 .4 —— — Hiram Johnson 6 27.5 21.5 $ 8,120 $ 6,558 Mtn. View 0 1.8 1.8 59,443 45,447 Oakland Tech. 1 6.8 5.8 66,005 18,683 Rio Cazadero 5 3 .0 —2.0 — — Menlo— Atherton 4 4.2 0.2 682,860 524,400 Sequoia 6 9 .4 3 .4 64,184 40,482 Total 31 60.3 29.3 $ 41,006 $ 25,506 l 5 0 PENINSULA ACADEMIES 1987-88 EVALUATIONS Table 3.10 also shows the cost per dropout saved. The components of cost are detailed in Table 3.9. Additional teacher time is given a value of $40,000 per FT E. Aides and administrators are valued at $20,000 and $50,000 per FTE, respectively. The value of extra facilities or equipment given to the academy program is converted to an annual expense, using an annualization factor of 0.2. Time donated by local employers’ representatives is given a value of $200 per day. It is probably not necessary to state that these numbers are somewhat arbitrary: valuing a teacher at less than $40,000, or an employer‘s representative at more than $200 per day, obviously would give somewhat different results. However, the numbers we used are certainly within the range of plausibility. The amounts of resources reported for 1987—88 in these eight academies were spent on sophomores, juniors, and seniors, but we want to relate costs to benefits of dropout prevention, and the only cohort for whom final graduation rates can be computed at this time are the seniors. Therefore, we had to allocate some fraction of the total 1987-88 cost to the 1987—88 seniors, i.e., the 1985—86 sophomores. The fraction of cost we allocated to the 1985—86 sophomore cohort is the original number of academy students in that cohort, divided by the original number of students in all three cohorts attending the academy in 1987—88. After computing the cost in 1987—88 allocated to the 1985-86 sophomore cohort, we multiplied it by three to estimate the total cost of resources spent on that cohort of students during their three years in the academy. The value of all resources spent is an estimate of the cost to society. This includes the imputed value of time donated by local employers’ representatives. Table 3.12 shows how this time was reportedly used: most of it was spent in direct contact with students, either as mentors or job supervisors. Because this time could have been spent on other productive activity, it must be counted as a cost to society. But since no public money was paid to these representatives of local employers, the time they gave the academy students is not a cost to taxpayers. The last two columns of Table 3.11 show the three-year cost incurred to taxpayers and to society as a whole on behalf of the 1985-86 sophomore cohort in each academy program. The last two columns of Table 3.10 show these costs divided by the number of dropouts saved from this cohort in each academy. In the two academies where the dropout rate was higher among academy students than among their comparison group, the cost per dropout saved is undefined. In the other six programs, the cost to society per dropout saved ranges from approximately $8,000 to almost $700,000. For these eight academies as a group, the three-year cost to society is approximately $41,000 per dropout saved, and the three-year cost to taxpayers is approximately $25,500. CROSS-SITE ANASYSES AND CONCLUSIONS 1 5 1 TABLE 3.11 Reported Costs of Academy Programs in 1987-88, and 3-Year Cost Attributed to Cohort Entering as Sophomores in 1985-86. n , i - Fraction 3-yr cost attributed for cohort Add’l Local to cohort entering teacher Facilities emps.’ entering - time Aides Admin. & equip. reps. 1985-86 To society To taxpayers Bakersfld $36,000 $8.000 $10,000 $7,000 $ 3,800 0.46 $89,424 $84,180 Hiram Johnson 80,000 0 10,000 4,000 22,000 0.50 174,600 141,000 Mtn. View 62,000 6,000 10,000 2,200 24,700 0.34 106,998 81,804 Oakland Tech. 70,000 6,700 16,500 10,000 261,400 0.35 382,830 108,360 Oak ' Ridge 28,000 0 0 0 0 0.42 35,280 35,280 Rio Cazadero 16,000 20,000 0 6,000 2,600 0.43 57,534 54,180 Menlo- Athenon 56,000 20,000 10,000 6,000 27,800 0.38 136,572 104,880 Sequoia 88,000 20,000 10,000 6,000 72,600 0.37 218,226 137,640 Total $436,000 $80,700 $66,500 $41,200 $415,300 0.40 $1,201,464 $747,324 How do these costs compare with the economic benefit of dropout prevention? The main economic benefit to society from dropout prevention is the value of extra output produced by graduates, compared to dropouts. An approximate measure of this extra output is the average difference in pre-tax earnings between graduates and dropouts. Using this difference to measure the economic benefit of dropout prevention is valid if the difference is attributable to the effects of the extra schooling, rather than to pre-existing dissimilarities between dropouts and graduates. There is some evidence that the difference in labor market outcomes experienced by dropouts and graduates is not, in fact, attributable to observed, pre-existing dissimilarities (Stern, Paik, Catterall, and Nakata, 1989, forthcoming). l 5 2 PENINSULA ACADEMIES 1987—88 EVALUATIONS TABLE 3.12 Reported Numbers of Days Spent by Local Employers’ Representatives in Academy-Related Activities, 1987-88. 3 I . . | Advisory Speaking Job committee to groups Field Job super- meetings of students trips Mentors placemts vision Total Amiga! Bakersfield 0 3 3 2 3 8 19 Hiram Johnson 35 40 10 22 4 1 112 Mtn. View 2 9 7 88 6 12 124 Oakland Tech. 25 40 50 1 12 20 960 1,307 * Oak Ridge 0 0 0 0 0 0 0 Rio Cazadero 1 2 2 — 2 6 13 Independence and Silver Creek 40 15 10 338 75 75 553 Menlo-Atherton 24 5 5 80 10 15 139 Sequoia 20 8 25 200 10 100 363 San Marcos 45 12 6 ll 3 ll 88 Total 192 134 118 853 133 1,188 2,718* * Total includes 100 days reported helping to plan and evaluate programs. We therefore use the difference in average lifetime earnings between graduates and dropouts as a measure of the economic benefit from dropout prevention. One good source of data on employment and earnings of graduates and dropouts by age is the 1984 Survey of Income and Program Participation (US Department of Commerce, Bureau of the Census, 1987). Using a five percent discount rate, the present value at age 18 of the average difference in earnings between individuals who have not finished high school and individuals who have finished high school but no more, is $77,500. Using a 10 percent discount rate gives a present value of $42,000. However, five percent (or less) is closer to the prevailing real (inflation-free) rate of interest, and a real interest rate is the appropriate rate to use with the cross-sectional earnings data, which are all in same-year (1984) dollars. CROSS-SITE ANASYSES AND CONCLUSIONS 1 5 3 The benefit figure of $77,500 is based on the assumption that all the extra earnings of graduates compared to dropouts are due to the additional schooling rather than to pre- existing dissirnilarities. To the extent that unobserved characteristics such as ambition or self-discipline are causes both of finishing high school and of higher earnings, the $77,500 figure overstates the true benefit of dropout prevention. On the other hand, this figure also leaves out other benefits, such as the lower risk of incarceration for people who finish high school—some of which may be attributable to the effects of schooling, or at least to the effect of not being pejoratively labeled a dropout. On balance, then, a number in the neighborhood of $77,500 (in 1984 dollars) seems a reasonable estimate of how much society benefits from keeping one student in school through graduation. Table 3.10 shows that the eight academies for which high school graduation rates are now available saved an estimated 29 students from dropping out, at a three-year cost to society of $41,000 per dropout saved. If the social benefit of saving one dropout is $77,500, then the “profit” or net benefit to society was approximately $36,500 per dropout saved, or a total of more than one million dollars for this cohort of academy students. Table 3.10 also shows that five of these eight academy programs achieved a positive net benefit by keeping their cost to society per dropout saved less than the benefit figure of $77,500. Conclusions and Recommendations Academy Implementation As was true a year ago, implementation of the academies continues to be uneven across sites (see Table 3.3). While no academies ended operations altogether this year, one or two appear to be close. The mean implementation rating improved somewhat, from 18.0 a year ago to 20.7 this year. Converted to “numerical grades,” the mean grade increased from 72 percent to 81 percent, or from a C— to a B—. This is certainly a step in the right direction, but it still leaves much room for improvement. Most implementation problems were centered in either lack of private sector or school district involvement and support. It is a substantial job to organize and manage all the various forms of business support intended in an academy. Those sites that have either provided part of an administrator’s time or given a lead teacher a substantial reduction in teaching responsibilities for this have generally had success. Those that have done less have not. As for district support, those districts that have taken the academy approach seriously, providing smaller class size and extra teacher preparation time as well as other forms of support, have generally seen success while those that have not have seen less. 15 4 PENINSULA ACADEMIES 1987-88 EVALUATIONS Student Outcomes A year ago, of 108 tests of differences between academy and comparison-group students on five outcome variables (attendance, credits, grades, courses failed, and dropout probability), 41 were statistically significant in favor of academy students and two in favor of comparison groups. This year, with a third year’s results in place, of 212 tests of such differences now conducted, 61 have been statistically significant in favor of academy students and 11 in favor of comparison groups. As was true a year ago, these results are not spread evenly across sites. Three have clearly positive results. The remaining sites range from several where most differences favor academy students but generally fail to reach statistical significance, to others where there is little pattern to the differences between academy and comparison goup differences. There are no sites at which the academy seems to have had any systematic negative effects. A clear pattern did emerge this year regarding the relationship between the year of program treatment and the. resultant program effects. During their first year of academy participation students experience a generally strong positive effect. During the second year this effect remains, but weakens. During the third year it seems to disappear altogether. While part of this pattern may be due to a statistical artifact caused by the nature of the analysis, most of it appears to be genuine. It suggests that the intensity of the program needs to be maintained over a longer period of time. Information on school dropouts is far more complete than was true last year (see Table 3.8). This is presented in temis of three types of attrition: known dropouts, probable dropouts, and transfers to other high schools. Most of the attrition is due to transfers. For the first cohort, which is the only one for which three years of data are available, the transfer rate to other high schools among academy students was 25.6 percent and for the comparison students 33.1 percent The average of known plus probable dropouts for academy participants was 7.3 percent and for comparison students 14.6 percent. This suggests the academies are having some success in reducing transiency between schools, and substantial success in reducing dropouts. The student questionnaire feedback suggests that academies are doing well in terms of providing business speakers and field trips, fair in terms of providing mentors, and rather poorly in providing jobs for juniors and seniors. Most students like the academy equipment and materials they work with. see a clear connection between their academy studies and post- graduate plans, and like the academy better than their regular high school program. Relatively few students are developing career plans through the academics, and most plan to attend some form of college upon graduation. A significant proportion of students report an increased liking of their classwork after being in the academy. CROSS—SITE ANASYSES AND CONCLUSIONS 1 5 5 Correlation Between Implementation and Student Outcomes There is a fairly wide variety among academy sites with respect to the way the model has been implemented. There is also a fairly wide range of student outcomes. One question is the relationship between these two. Are the sites that are implementing the academy model faithfuuy also having better outcomes in terms of student performance? Table 3.13 on the next page presents four types of information: - The latest implementation rating, presented as the percent of the best score possible (like a numerical grade) 0 The number of statistical tests favoring the academy students over their comparison student counterparts over the past three years, and the number of these that are statistically significant 0 The number of statistical tests favoring the comparison students over the academy students, and the number of these that are significant - The attrition (known dropouts, probable dropouts, and transfers) for academy and comparison group students for cohort one‘ through June 1988 Table 3.13 presents a somewhat complex picture of the relationship between program implementation and student outcomes. Sorting the sites into two groups helps to clarify this picture. There are five sites with implementation ratings of 90 percent or above. These five sites have: - Sixty-seven differences favoring academy over comparison students, of which 31 are statistically significant - Seventeen differences favoring comparison over academy students, of which two are statistically significant 0 An average attrition difference of 9.4 percent in favor of academy students Contrastingly, the remaining six sites have: - Seventy-seven differences favoring academy over comparison students, of which 15 are statistically significant 1 5 6 PENINSULA ACADEMIES 1987-88 EVALUATIONS 0 Forty-two differences favoring comparison over academy students, of which seven are statistically significant 0 An average attrition difference of 5.4 percent in favor of academy students TABLE 3.13 Comparison of Implementation Ratings, Differences Between Academy and Comparison Students on Outcome Measures (Attendance, Credits, GPA, and Courses Failed), and Differences Between Academy and Comparison Students on Attrition* Difference Difference Difference in Impl. in Favor of Acad in Favor of CG. Attrition Rating All Sig. All Sig. Acad. C.G, Bakersfield 81% 19 1 10 4 43% 28% Franklin** 72% 11 4 3 1 NA NA Hiram Johnson 811% 13 4 12 0 33% 74% Independence 96% 13 8 3 0 19% 26% Menlo-Atherton 91% 10 l 3 0 30% 63% Mountain View 61% 17 1 8 1 63% 50% Oak Ridge 52% 13 3 9 1 29% 62% Oakland Tech 98% 21 15 4 1 13% 73% San Marcos*** 78% 4 2 0 O O O Sequoia 91% 14 l O O 21% 25% Silver Creek 96% 9 6 7 1 43% 39% Meanffotal 81% 144 46 59 9 33% 48% * Data from Rio Cazadero have been omitted because of their questionable validity ** The Franklin data are forjust two years, 1985—86 and 1986—87 *** The San Marcos data are for just one year, 1987—88 To summarize, the first group of academies seems to be having a strong positive effect on its participants, with very few examples of negative effects. The second group, while it is also having a net positive effect, is doing so far less strongly, and with more frequent instances of negative effects. CROSS-SITE ANASYSES AND CONCLUSIONS 1 5 7 It should also be made clear that there are many individual variations among the sites that make broad generalizations difficult. For example, some academies are having a positive impact on in-school measures, but are not retaining students well. Contrastingly, some academies are effectively retaining students but are not seeming to improve their school performance. It must be kept in mind that these programs operate in very different settings, with different staffs, and with many other individual variations too numerous to detail here. Perhaps the fairest conclusion about the relationship between implementation of the academy model and resultant effectiveness with students is that while full implementation does not guarantee success, it notably improves its chances, while failure to implement the model faithfully substantially weakens the likelihood of positive impact. Recommendations The conclusions reached a year ago in this evaluation continue to remain valid. First, the academy program model is capable of producing substantial positive effects on students who participate in it. Second, the model is complex to implement, has been implemented unevenly to date, and the quality of its implementation appears to be closely related to its success with students. These findings lead to several recommendations: 1. Given the evidence now in existence concerning the ability of the academy model to achieve positive outcomes with at-risk high school students, there is reason to proceed cautiously with the expansion of this model to new sites. The legislation underlying the academics, SB 605, contains language providing for up to 15 new replications of the model each year. The degree to which this rate of expansion is followed is up to the legislature and governor to determine. It is our feeling that the quality with which new replications are implemented is critical to their success, and the ability of the state Department of Education and those working in cooperation with it in this effort to ensure such quality should be an important determining factor in the rate of academy expansion. 2. Related to the above, szgfj‘icient training and technical assistance should be provided to each new academy replication to ensure the best possible chance of success. In those sites that have floundered or been terminated, there was insufficient help given to teachers and both school and district administrators in understanding the full model and how to implement it. Given the complexity of the academy model, such training and technical assistance is important. In our judgment, a relatively small investment in this respect can protect the far larger investment in new programs, and increase their chances of success substantially. 1 5 8 PENINSULA ACADEMIES 1987—88 EVALUATIONS 3. The legislation which now provides a basis for the academics, SB 605, includes a performance-based funding formula, which takes effect during the 1988—89 school year. The terms of this bill should be implemented carefully and consistently to ensure that those academies performing well are continued and allowed to flourish, while those that are not are encouraged to cease operations. The state Department of Education seems determined and equipped to meet this recommendation, which is encouraging. 4. It will inevitably seem self-servin g in an evaluation report to recommend further evaluation. Nevertheless, in our judgment, while this can take varying forms, and need not necessarily be as elaborate as it has been to date, some form of continuing evaluation is advisable. We would not currently know what we do about the performance of academies and the advisability of furthering their replication without the evaluation of the past three years. While evaluation is not inexpensive, it requires only a small proportion of the funding for the programs, and if the academies are to be expanded in an informed way, is an important component of their expansion. Evaluation serves several purposes. It keeps the sites themselves focused on the centrally important student outcomes, thus strengthening the accountability of the programs. It shows the academies themselves whether they are working or not, and where they need to focus efforts to improve on weaknesses. It provides evidence of success when it does occur, strengthening the credibility of the programs and their ability to obtain private-sector support And it provides evidence at the state level of whether the academy model is worth investing more resources in, and if so, how to do this in a way that will ensure the best returns on such an investment. To date, all evaluation efforts of the academies have been sponsored through private foundations, even though SB 605 contains language calling for an evaluation of the academies. We feel the state would serve its own interests best by honoring the terms of SB 605 and providing state support for evaluation. 5. Related to the above, while the academies remain a rather limited venture in California, the evidence that now exists of their success, both here and elsewhere, is leading to wider interest in the use of the model. The US. Department of Education has offered the academies as one of only a few suggested models for replication in two recent federal funding competitions for dropout prevention programs. A number of individual school districts have shown interest in beginning academies on their own, and several such academies exist already in California. The model includes virtually all the elements recommended in a number of studies to meet the needs of at—risk high school students. Given this, the efi’orts of California to replicate and evaluate this approach take on added importance, and place the state in a position of leadership. We urge an awareness of the broader potential of and interest in the academy model as a context for this report and its recommendations. Appendix A Multivariate Analysis of Academy Program Effects Underlying this statistical model is the notion that a student's performance in a given year can be predicted to some extent by that student's performance in the preceding year and by the student’s gender, race or ethnicity, and date of birth. The question is: when differences between academy and comparison students in these predictor variables are taken into account, are academy students performing better than their counterparts in the comparison group? The exact relationship between predictor variables and this year's performance may or may not be the same for academy students as for students in the comparison group. Therefore, the first step in the analysis is to estimate the relationship between predictor variables and student performance in the academy and comparison group at each site and to test whether that relationship is the same. Specifically, let (1) mt: a + inj,t-1 + cxij denote the estimated relationship between predicted student performance and predictor variables. Here Yij,t denotes the predicted performance (attendance, credits, grades, or courses failed) of the ith individual student in the jth program (either an academy or comparison group) in year t. Yij,t-1 is the previous year's value of the same performance measure for the same student in the same program. xij stands for a set of predictor variables (gender, race or ethnicity, and date of birth) that pertain to that same student but do not change from year to year. Finally, a, b, and c represent regression coefficients estimated from the data. In equation (1), these coefficients have the same value for academy and comparison—group students at each site. That is, the model in equation (1) assumes that the relationship between predictor variables and student performance does not differ between the two groups of students at each site. Equation (1) is estimated by pooling data for both groups at each site and computing the ordinary least—squares regression coefficients. To test the assumption that the relationship is the same for both groups, two alternative models are estimated. (2) ’y‘ij,t = aj + inj,t-1 + CXij Equation (2) is the same as equation (1) except that the intercept coefficient, a, is now allowed to take a different value for the academy group than for the comparison group. 159 160 PENINSULA ACADEMIES 1987—88 EVALUATIONS However, in equation (2) the slope coefficients, b and c, are still constrained to be the same in both groups. Equation (2) is estimated by pooling the data for both groups, but adding a binary variable which denotes whether a student belongs to the academy or comparison group. (3) ?ij,t = aj + ijij.t-1 + ijij In equation (3), both the slope coefficients, b and c, and the intercept coefficient, a, are estimated separately for the academy and comparison groups. Estimating equation (3) requires computing a separate regression for each of the two groups. Equation (3), which "custom tailors" the model to fit each group of students, can account for more of the observed variance in student performance than equations (1) or (2). However, estimation of separate coefficients for each group of students requires that a larger number of coefficients be estimated. The statistical criterion for deciding whether to use model (3) instead of model (1) or (2) is whether the better fit obtained with equation (3) is "worth" the degrees of freedom that are used up in estimating separate coefficients for each group. Formally, this question is answered by performing an F test. To test the null hypothesis that slope coefficients have the same value for both groups of students, while the intercept coefficient may have different values, the F statistic is computed as follows. (EZ-E3)/(n3-n2) F(n3-n2;N-n3) = E3/(N-r13) where E2 = the sum of squared errors around regression equation (2); E3 = the sum of squared errors around the separate regressions represented by equation (3); n2 the number of regression coefficients estimated in equation (2); n3 = the total number of coefficients estimated in the separate regressions represented by equation (3); and N = total number of students in the academy and comparison groups combined. APPENDIX A 1 61 If this F—statistic is smaller than the critical value, one concludes that it is appropriate to pool the data on academy and comparison students in order to estimate a predictive model. The choice then is between equation (1) and equation (2). That choice simply depends on the estimated coefficient on the binary variable denoting group membership in equation (2). If that coefficient is significantly different from zero, equation (2) is the best model; otherwise, equation (1) is best. To say that equation (2) is better than equation (1) is equivalent to saying that being in the academy makes a statistically significant difference. The size of that difference is the estimated coefficient on the variable denoting academy membership. If the F—statisu'c above is greater than the critical value, then it is apropriate to use separate regressions for the academy and comparison groups. One then uses the following procedure to test whether performance of the average student in an academy program was better than would be predicted if that student had been in the comparison group instead. Let § AA = predicted value of outcome (attendance, credits, grades, or courses failed) for a student whose predictor variables had values equal to the mean for students in an academy program, where the predicted outcome is computed from regression coefficients estimated for the academy group; A . . . y AC predicted value of outcome for a student whose predictor variables had values equal to the mean for students in the academy program, where the predicted outcome is computed from regression coefficients estimated for the comparison group; predicted value of outcome for a student whose predictor variables had values equal to the mean for students in the comparison group, where the predicted outcome is computed from regression coefficients estimated for the comparison group; and 9m 9C A = predicted value of outcome for a student whose predictor variables had values equal to the mean for students in the comparison group, where the predicted outcome is computed from regression coefficients estimated for the academy group. Comparison of ’y A A and 9 AC tells us whether a student with the average characteristics of the academy group did better in the academy than he or she would have been predicted to do in the comparison group. Likewise, comparing ycc and y‘c A indicates whether a student with the average characteristics of the comparison group did better in the comparison group than would be predicted if he or she had been in the academy program instead. If both of these comparisons favor the academy program, it is apparent that the 162 PENINSULA ACADEMIES 1987—88 EVALUATIONS academy program is having a positive effect. Statistical significance of those differences is tested by determining whether 9 AC lies outside the 95 percent confidence interval for predictions around 9 A A , and whether 9C A lies outside the 95 percent confidence interval for predictions around 9cc. Appendix B AMENDED IN ASSEMBLY SEPTEMBER 9, 1987 AMENDED IN ASSEMBLY SEPTEMBER 8, 1987 AMENDED IN ASSEMBLY AUGUST 18, 1987 AMENDED IN SENATE MAY 19, 1987 AMENDED IN SENATE APRIL 8, 1987 SENATE BILL N o. 605 Introduced by Senator Morgan (Coauthor: Senator McCorquodale) (Coauthors: Assembly Members Duplissea, O'Connell, Sher, V Speier, and Vasconcellos) February 25, 1987 An act to amend Sections 54690, 54691, and 54692 of, to amend the heading of Article ‘5 (commencing with Section 54690) of Chapter 9 of Part 29 of, to add Sections 54695 and 54696 to, and to repeal and add Sections 54693 and 54694'of, the Education Code, relating to education, and making an appropriation therefor. LEGISLATIVE COUNSEL‘S DIGEST SB 605, as amended, Morgan. Schools: Partnership Academies. (1) Existing law makes various findings regarding the Peninsula Academies Model Program. Existing law requires the Superintendent of Public Instruction, commencing with the 1984—85 fiscal year, from funds appropriated for that purpose, to provide specified apportionments to high school districts meeting specified eligibility criteria, for purposes of establishing not more than 10 academies under the Peninsula Academies Model Program, and to the Sequoia Union High School District. 94 40 163 SB 605 — 2 — This bill would make specific findings regarding the success of the Peninsula Academies Model Program and would provide for the establishment of additional academies, to be known as Partnership Academies. This bill would require the Superintendent of Public Instruction, commencing with the 1987—88 fiscal year, from funds appropriated for that purpose, to issue grants in specified amounts to high school districts meeting specified eligibility criteria for purposes of planning, establishing, and maintaining Partnership Academies. This bill would authorize the superintendent to issue a maximum of 15 grants per year for the 1987—88, 1988—89, 1989—90, and 1990—91 fiscal years. This bill would require the superintendent, when issuing the grants to school districts, to ensure that the grants are equitably distributed among high wealth and low wealth school districts in urban, rural, and suburban areas. This bill would provide for the identification of eligible students, making presentations to prospective students, and selection of students. (2) Existing law requires the Educational Technology Committee to assist the superintendent and requires the State Department of Education to evaluate the Peninsula Academies Model Program and to report to the Legislature regarding specified issues 2 years after the effective date of the existing law. This bill would repeal these provisions and instead would require the superintendent to establish eligibility criteria, as specified, for school districts applying for grants, to contract for a 3-year independent review of the effectiveness of Partnership Academies, and to report the results of the review to the Legislature by January 1, 1991. This bill would prohibit a school district from establishing additional academies until the district has successfully operated its existing academy or academies. This bill would also require the superintendent to develop guidelines with respect to the Partnership Academies, as Specified. (3) This bill would appropriate $635;999 $535,000 to the State Department of Education for the 1987—88 fiscal year for allocation as specified. Vote: majority. Appropriation: yes. Fiscal committee: yes. State-mandated local program: no. 8E3NND—‘t—‘HHHHHD—IHF— HowmdeAwmwoom-QQUIACAN— wuwwwwuuwm —3—- SB 605 The people of the State of California do enact as follows: SECTION 1. ‘ The heading of Article 5 (commencing with Section 54690) of Chapter 9 of Part 29 of the Education Code is amended to read: Article 5. Partnership Academies SEC. 2. Section 54690 of the Education Code is amended to read: 54690. The Legislature hereby finds and declares that the Peninsula Academies Model Program has proven to be an effective school-business partnership program to provide occupational training to educationally disadvantaged high school students who present a high risk of dropping out of school. “Educationally disadvantaged high school students,” for the purposes of this article, means students enrolled in high school who are at-risk of dropping out of school as indicated by at least three of the following four criteria: (a) Past record of irregular attendance. (b) Past record of underachievement (at least one year behind in the coursework for the student’s respective grade level). ’ (c) Past record of low motivation or a disinterest in the regular school program. ‘ (d) Disadvantaged economically. The Legislature further finds that the success of the program is directly related both to the participation of private entities which have provided personnel, equipment, and training positions and to the initial development of the program conducted by the Sequoia Union High School District. The Legislature further finds and declares that although the original Peninsula Academies were established to provide training in the high technology fields of computers and electronics, the model program has been developed successfully in the fields of medicine, finance, and food services, as well as high technology. It is therefore the intent of the Legislature that additional a: U! GD 00 -J CD CH Jk C0 to h— 605 —-4— academies, to be known hereafter as Partnership Academies, be established in California. SEC. 3. Section 54691 of the Education Code is amended to read: 54691. Commencing with the 1987—88 fiscal year, from the funds appropriated for that purpose, the Superintendent of Public Instruction shall issue grants to school districts maintaining high schools which meet the specifications of Section 54692, for purposes of planning, establishing, and maintaining academies, as follows: (a) For the 1987—88, 1988—89, 1989—90, and 1990—91 fiscal years, the superintendent may issue a maximum of 15 grants per year, for purposes of planning Partnership Academies and developing the curriculum. The Superintendent of Public Instruction, when issuing the grants to school districts, shall ensure that the grants are equitably distributed among high wealth and low wealth ‘ school districts in urban, rural, and suburban areas. Each planning grant shall be in the amount of fifteen thousand dollars ($15,000). (b) For the 1987-88 fiscal year, and each fiscal year thereafter, the superintendent may issue grants for the implementation and maintenance of academies initiated by the Sequoia Union High School District, the Peninsula Academies Model Program, or planned pursuant to subdivision (a). Implementation and maintenance grants shall be calculated in accordance with the following schedule: (1) Districts operating academies established under the Peninsula Academies Model Program or the Sequoia Union'High School District, may receive seven hundred fifty dollars ($750) per year for each qualified student enrolled in an academy, provided that no more than sixty-seven thousand five hundred dollars ($67,500) may be granted to any one academy for each fiscal year. (2) Districts operating one or more academies may receive two thousand two hundred fifty dollars ($2,250) for each qualified student enrolled in an academy during the first year of that academy’s operation, provided that no more than sixty-seven thousand five hundred dollars 94 100 — 5 — SB 605 ($67,500) may be granted to any one academy for the initial year. (3) Districts operating one or more academies may receive one thousand five hundred dollars ($1,500) for each qualified student enrolled in an academy during the second year of that academy's operation, provided that no more than ninety thousand dollars ($90,000) may be granted to any one academy for the second year. (4) Districts operating one or more academies may receive seven hundred fifty dollars ($750) per year for each qualified student enrolled in an academy during the third year and for each year of operation thereafter, provided that no more than sixty-seven thousand five hundred dollars ($67,500) may be granted to any one academy for each fiscal year. ‘ (c) For purposes of this article, a qualified student is a student who is enrolled in an academy for the 10th, 11th, or 12th grade, successfully completes a school year in the academy with an attendance record of no less than 80 percent positive attendance, and obtains 90 percent of the credits each academic year in courses that are required for graduation. ((1) At the end of each school year, school districts that have been approved to operate academies pursuant to this article shall certify the following information to the Superintendent of Public Instruction: (1) The number of qualified students enrolled during the just completed school year, by grade level, for each academy operated by the district. (2) The operation of each academy in accordance with this article, including Sections 54692 and 54694. (3) The amount of matching funds and the dollar value of in-kind support made available to each academy in accordance with subdivisions (a) and (b) of Section 54692. (e) The superintendent shall adjust each school district’s grant in accordance with the certification made to him or her pursuant to subdivision (d) or in accordance with any discrepancies to the certification that may be revealed by audit. Notwithstanding the 94 120 SB 605 Emigu—wr—t—v—Hr—r—r—t—I Hooooqcncngcowr—ocooo-qcsmhwmr— NNNNNN 0340.01.50: 29 30 31 32 33 34 35 36 37 38 39 40 _5___ provisions of this section, the superintendent may advance the funding as he or she deems appropriate to districts that are approved to operate, or plan to operate Partnership Academies. (f) Funds granted to school districts pursuant to this article may be expended without regard to fiscal year. SEC. 4. Section 54692 of the Education Code is amended to read: 54692. In order to be eligible to receive funding pursuant to this article, a district shall provide all of the following: (a) An amount equal to a 100 percent match of all funds received pursuant to this article, in the form of direct and in-kind support provided by the district. (b) An amount equal to a 100 percent match of all funds received pursuant to this article, in the form of direct and in—kind support provided by participating companies or other private sector organizations. (c) An assurance that each Partnership Academy will be established as a “school within a school” in the same manner as the Peninsula Academies Model Programs. (d) Assurance that each academy student will be provided with: (1) Instruction in at least three academic subjects each regular school term that prepares the student for a regular high school diploma. When possible, these subjects should relate to the occupational field of the academy. (2)A “laboratory class” related to the academy's occupational field. (3) A class schedule which limits the attendance to the classes required in paragraphs (1) and2 (2 ) to pupils of the academy. (4) A mentor from the business community during the pupils 11th grade year (5) A job related to the academy s occupational field or work experience to improve employment skills, during the summer following the 11th grade. A student that must attend summer school for purposes of completing graduation requirements is exempt from this paragraph. cocoqcacnaswrop- 10 12 13 14 15 16 17 18 19 20 21 23 24 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 — 7 — SB 605 (6) Additional motivational activities with private sector involvement to encourage academic and occupational preparation. SEC. 5. Section 54693 of the Education Code is repealed. SEC. 6. Section 54693 15 added to the Education Code, to read. 54693. The Superintendent of Public Instruction shall establish eligibility criteria for school districts that apply for grants pursuant to this article. When establishing criteria, the superintendent shall consider the commitment and need of the applicant district. The superintendent may consider district indicators of need such as the number or percent of pupils in poverty or with limited-English proficiency, and the dropout rate. No school district with less than three high schools may establish more than one academy until it has operated the existing academy successfully for two years. No school district with more than two high schools may establish more than two academies until it has operated the existing academy or academies successfully for two years. The years in which districts receive planning grants pursuant to subdivision (a) of Section 54691, shall not be considered years of operation for purposes of this article. SEC. 7. Section 54694 of the Education Code is repealed. SEC. 8. Section 54694 is added to the Education Code, to read: 54694. The Superintendent of Public Instruction shall develop guidelines with respect to the Partnership Academies. The guidelines shall include, but not be limited to, enrollment provisions, application procedures, and student eligibility. SEC. 9. Section 54695 is added to the Education Code, to read: 54965. (a) The ninth grade teachers and counselors in schools maintained by school districts approved to operate academies pursuant to this article shall identify students eligible to participate in a Partnership Academy. (b) Teachers and counselors in schools maintained by m U: {DCDNIQUIACJI‘OU— 605 —-8— school districts approved to operate academies pursuant to this article, business representatives, and academy students of academies that are operating in the area shall be encouraged to make presentations to prospective students and their parents. (0) The staff of each Partnership Academy shall select students from among those who have expressed an interest in the academy and whose parents or guardians have approved the student’s participation. SEC. 10. Section 54696 is added to the Education Code, to read: - 54696. The Superintendent of Public Instruction shall contract for a three-year independent review of the effectiveness of the Partnership Academies, and shall report the results of the review to the Legislature by January 1, 1991. The independent review shall include, but not be limited to, the following: (a) An analysis of the extent and degree of success of business and industry involvement, including in-kind contributions, mentor services, summer jobs, and assistance in job placement. . (b) The number of pupils entering advanced training programs, obtaining employment, and enrolling in postsecondary institutions. (c) Attendance rates. (d) The number of pupils completing their high school education and graduating. SEC. 11. (a) There is hereby appropriated from the General Fund to the State Department of Education the sumofsixhundredt—hiflhflfivetheuaanddellarsw five hundred thirty-five thousand dollars ($535,000) or the 1987-88 fiscal year for allocation in accordance with the following schedule: (1) In augmentation to Budget Item 6100-166-001 of the 1987 Budget Act, for funding the two school districts which received 1986—87 planning grants ...................................................... $135,000 94 180 mmqmmawtow __9._ SB 605 (2) In augmentation of Budget Item 6100-166—001 of the Budget Act, for funding the existing 10 academies pursuant to Section 54691 of the Education Code .................................. , ' 75,000 (3) For the planning grants to 15 academies pursuant to Section 54691 of the Education Code .......... (4) For the independent evaluation pursuant to Section 54695 of the Education Code .................................. 50,000 225,000 9(5) For the State Department of Education administrative costs for the purposes of Article 5 (commencing with Section 54690) of Chapter 9 of Part 29 of the Education Code .................................. 50,000 (b) It is the intent of the Legislature that funding in future fiscal years for the Partnership Academies program, established in Article 5 (commencing with Section 54690) of Chapter 9 of Part 29 of the Education Code, be provided in the annual Budget Act. It is further the intent of the Legislature that the Superintendent of Public Instruction shall include a line item for Partnership Academies in the superintendent’s annual budget report to the Governor and the Legislature. 94 210 wan”. .‘v - wr 9w...» 'luw'ww h-u« 4 .V .. . c an. «e .‘ g n . d. \ ‘ - ‘.' i . \1 .'j M A . a ' . v F x r v x , . i u _‘ I .' - _. \ . h ’ I. . ‘ 4, ~ M. \ = m. . 4 I n. m (a \ ‘ . - << f» ,. mu.) _ , ». g s ‘ \ UC. EEEEEEEEEEEEEEEEE IWilli/lll/II/l/I/I/I/l/llll/Ill/Ill/l/I/l/Il/III/l/Illll Cll‘]?UDSE'3