Public Private Partnerships to Improve Academic Achievement of Margialized Students: Evaluation Results
I. Escuela Nueva, Colombia
The Escuela Nueva program was introduced in the rural schools of Colombia in 1975 and has expanded, having incorporated by 1992, 27,000 Colombian rural schools. The objective of the program is to promote a new concept of rural school, on the basis of the methodology developed by Escuela Nueva, which includes:
developing active and participative learning, oriented toward rural life
allowing each student to advance at his/her own pace, without repeating grades
ensuring that the teacher operates as a facilitator and not as an information transmitter
promoting leadership and cooperation between the administrative body and the teachers of the school, and
involving the community and parents in the schooling of their students.
Program Evaluation
Evaluation studies aimed to establish the cost-effectiveness of Escuela Nueva and its impact on students' academic achievement. Student achievement is evaluated and repetition and dropout rates of Escuela Nueva are compared with those at traditional rural schools. Social changes in the communities benefiting from the project are also recorded.
The evaluation model uses data collected through a survey of educational quality in three districts of Colombia in 1992. The evaluated population includes 52 rural schools, 24 of which are Escuela Nueva program schools. Survey respondents included the school principal, teachers and students. Test scores for Maths and Spanish were used to measure student achievement. Schiefelbein (1991) calculates the unit cost of the project at about 10% of the traditional unit cost, without considering the purchase of books for the students. Since at present textbooks are supplied, the unit cost of Escuela Nueva is reduced to an increase of 5% due to the expenses of training teachers and supervision of the project.
Evaluation Results
Repetition and dropout rates in Escuela Nuevaare lower than those in the traditional rural schools. In spite of a lack of didactic materials (many schools lack most of the elements included in the reform; less than half use the official textbooks and one third have no library), the average effect of Escuela Nueva is positive, especially in third grade. Student academic achievement in Escuela Nueva is superior to that of traditional schools, although this positive difference diminishes in 5th grade. This could be due to the better retention rates in Escuela Nueva (which retains the low achievers) or because this kind of education declines in the upper grades.
A first analysis shows that the cost per student is higher in Escuela Nueva. This is due to the larger number of teachers, which the Escuela Nueva methodology demands. Community participation appears to be successful. Some additional benefits can be observed which should be evaluated more carefully, such as: increase in agricultural productivity, reduction of the fertility levels among women, and improvements in health and nutrition.
Reference:
McEwan, P.J. 1995. "Primay School Reform for Rural Development: An Evaluation of Colombia New Schools" Inter-American Development Bank-IDB, Washington DC.
Schiefelbein, E.. 1991. "In Search of the School of the XXI Century". UNESCO-UNICEF, Santiago.
Schiefelbein, Ernesto, J. Swope, S.J. and P. Schiefelbein. 2000. "Cost-Effectiveness of Education Programs to Prevent and Eliminate Child Labor." (Unpublished memo).
II. Fe y Alegría (FYA), Bolivia, Colombia, Ecuador, El Salvador, Guatemala, Nicaragua, Paraguay, Peru and Venezuela
FYA is a regional federation of national educational organizations, each of which provides a wide range of educational services (primarily formal primary education) in highly marginalized communities. FYA has been establishing schools for marginalized populations in urban areas and in isolated rural settings since 1955.
Program Evaluation
Evaluation study assessed the internal efficiency of the FYA primary schools in nine FYA countries-- Peru, Bolivia, Venezuela, Nicaragua, Ecuador, Guatemala, Colombia, El Salvador and Paraguay. Data was collected from 290 of the 523 FYA schools in these countries, and a similar number of public schools in highly marginalized areas. The total number of students covered in the study was 321,252 (153,684 from the FYA schools and 167,568 from the public schools), a universe which represents approximately 1.57% of the total primary enrolment in the countries studied in 1995 (20,335,974).
Indicators used for the study included student achievement, rates of repetition and dropout, and overall rates of timely progress and retention in the nine countries. FYA and public schools were compared on the basis of seven independent criteria (or strategies to keep children in school). These strategies include: prevention, compensatory, multiple promotion opportunities, family economic incentives and community involvement, introduction of pre-school and secondary technical programs to expand school offerings.
The evaluation design considered quantitative and qualitative measures. In the schools surveyed, information on student educational progress was gathered from school records with respect to a four year period (1991-1994), on the cohort which began first grade in 1991. A wide range of data was gathered in this part of the study with respect to each of the participating FYA and public school. This included information on student achievement, enrolment, infrastructure, equipment, teaching staff, and location of the school. A semi-structured questionnaire was also given to school principals dealing with organizational culture of the school. Questions were asked about the principal’s role, topics of teachers’ meetings, criteria for selecting and evaluating teachers, the socio-economic status of the students enrolled, perceived causes of educational failure and success, and process of structuring and contents of the school day. Questionnaires were distributed to the principals of the participating schools, while data gatherers worked through the school records. Moreover, in four countries (Bolivia, Guatemala, Paraguay and Peru), interviews were conducted with staff members of the national and sub-national co-ordinating offices to qualitatively assess the FYA schools. School principals and personnel were asked about pedagogical, financial and administrative management of the FYA schools. Focus group sessions were also held with with teachers and parents concerning school-based pedagogical, financial and administrative management issues. At the school level, analysis of documents and some classroom observations was also carried out.
Evaluation Results
The evaluation of Fe y Alegría costs shows data with respect to the state contribution per student, individual school contributions, and national and international grants for two countries (Guatemala and Paraguay). First, the average state contribution per student was calculated for both countries (107). That was considered the basic public expenditure per student. Then the contributions of the schools and the grants were added, and again the average was calculated (169), considering the average per student investment added by FYA. Finally, this figure was calculated as the percentage of the basic public investment [(169-107)/107]. Nevertheless, there is no cost increment for the state, and hence the cost-benefit is much higher for the state.
The average real repetition rate in the nine countries with schools integrated into FYA is 14% less than that of the public schools. The average definitive dropout rate in the nine countries with FYA schools is 3% lower than that of public schools. The average of the overall rate of opportune progress of the FYA schools in the countries evaluated is 44% higher than that of public schools. The overall retention rate of the FYA schools in the nine countries evaluated is 11% higher than that of the corresponding public schools.
Reference:
Schiefelbein, Ernesto, J. Swope, S.J. and P. Schiefelbein. 2000. "Cost-Effectiveness of Education Programs to Prevent and Eliminate Child Labor." (Unpublished memo).
Swope, J. Y M. Latorre and F. Celedón.1998a. "Estudio Eficiencia Interna de las Escuelas Formales de Fe y Alegría", CIDE, Santiago, Chile.
Swope, J. and M. Latorre. 1998b. "Comunidades Educativas, Donde Termina el Asfalto: Escuelas Fe y Alegría en América Latina". CIDE-PREAL-USAID-IABD, Santiago, Chile.
Swope, J. Y M. Latorre. 1998c. "Fe y Alegría: una propuesta alternativa de educación primaria en América Latina". Persona y Sociedad 12 (2-3):225-242.
III. Corporation Sponsored (CS) Schools, Brazil
This program educates the children of employees of corporations that are located in isolated areas. These schools are managed through contract by the private educational organization Sistema Pitágoras de Ensino, which traditionally managed only elite schools. This organization manages the schools as private schools do, but incorporates into the classroom the children of all the employees of the corporation (from the highest ranks down to the lowest paid workers). The Pitágoras de Ensino system is in charge of management of 13 CS schools, with a population of about 20,000 pupils in ten states of Brazil and two other countries.
Program Evaluation
Evaluation of this school scheme tests two hypotheses. First, the effects of schooling are greater in underdeveloped than in developed countries and, (ii) the inputs –like good teachers and textbooks- lead to a great difference in academic results. Student academic achievement is considered as an indicator. The sample of schools under study consists of eleven schools: three CS, two elite (also administered by the Pitagoras de Ensino System), three public schools and three municipal schools. The latter six were selected according to their proximity to the CS evaluated schools. The CS schools were selected according to the similarity of their students to those of the public schools (therefore they might not be representative of all CS schools).
Data was collected through (i) questionnaires to obtain socio-economic data from the pupils; (ii) Math and Portuguese test results and, (iii) interviews with teachers, principals, local educational authorities and members of the involved corporations.
Program Evaluation
Academic achievement of the CS schools, compared to public schools, is not always significantly higher and it remains lower than the performance of the CS elite schools. Possible reasons include: (i) not all of the schools have been managed for a uniform time period by Pitagora; (ii) the social composition of the schools, because these students are especially isolated; (iii) teaching methods could be inadequate.
Available information does not allow assessment of the efficiency of CS schools (in comparison to that of public schools). Nevertheless, in contrast to the repetition culture observed in public schools, the idea that all students should be promoted exists in the CS schools.
Reference:
Castro, C., S. Guimaraes, J. Oliveira and S. Ribeirol. 1996. "Dealing With Poor Students", in Bridsall and Sabot (eds), Opportunity Foregone, Education in Brazil. The Johns Hopkins University Press, Chapter 17, pp.497-519, Washington DC.
Schiefelbein, Ernesto, J. Swope, S.J. and P. Schiefelbein. 2000. "Cost-Effectiveness of Education Programs to Prevent and Eliminate Child Labor." (Unpublished memo).
IV. Educación con Participación de la Comunidad (EDUCO), El Salvador
The EDUCO program seeks decentralization of management of pre-school education with a greater decision-making role of parents and community groups in the educational process. To that end, a group of parents is elected, who must take up responsibility for a school’s administration (control of budget and personnel). The philosophy of the program is that local management, rather than central government, can better manage a school. The three main objectives of EDUCO are:
increase access to education for the poorest rural communities;
promote local participation in education, and
improve the quality of pre-school and primary education.
EDUCO has expanded the coverage of pre-school and primary education in the rural areas of El Salvador. Participation in this program increased at the pre-school level, from 3,072 pupils in 1991 to 43,168 in 1996, and at the primary level from 5,344 in 1991 to 125,760 in 1995.
Program Evaluation
Evaluations measure the effects of decentralization and parental and community involvement in school management on student achievement results. Indicators used include: student academic achievement, quantity and quality of educational materials in the schools, and teacher attendance patterns. Certain social changes at the EDUCO communities are also observed.
The evaluation is based on the idea that the results at school depend on a combination of factors. An input nomenclature for the EDUCO schools and for the control schools was arranged: principals, parent associations (EDUCO’s and the traditional type) and individual parents. First, difference in input nomenclature was analyzed, then the differences in administration associated with those inputs, and finally the results achieved in the schools. These are academic performance, repetition, attendance and dropout.
The evaluation group comprised of a sample of EDUCO schools and a control group of public schools with the same features (with respect to size, location and enrolment). The sample is 311 schools (with or without EDUCO) with at least a section from first to third grade, with 10 or more students enrolled.
Data was collected through:
five questionnaires (for principals, teachers, students, parents of those students, and parents from the parents' association [from EDUCO or from traditional schools]);
two test results (Math and Spanish) applied in two different opportunities to all pupils of the 3rd grade.
As the achievement improvements are measured in regressive equations, here we defined a cost increment of 1% to estimate the effect on achievement.
Evaluation Results
Evaluation confirmed that EDUCO effectively serves the poorest segment of the population.
The academic achievement of EDUCO students does not differ from that of the traditional rural schools, even though: (i) the socio-economic and physiological circumstances of the former is inferior and, (ii) EDUCO schools are in a worse situation with respect to infrastructure and basic services.
The 3rd grade students of EDUCO schools have educational materials of better quality and in greater number than the traditional rural schools. The EDUCO program reduces significantly lost teaching days, because direct monitoring of the parents is associated with lower rates of teacher absenteeism.
Parents of EDUCO pupils feel they have a personal influence on the decision-making at school. Therefore, they participate more in their children's' school activities (than parents at the traditional rural schools), contributing to the maintenance of infrastructure and equipment, gaining access to materials, sending their children to school, monitoring school attendance, visiting classes, preparing annual plans for the school and raising funds. Even though EDUCO parents are poorer and less educated than those of the traditional rural schools, they pay the same amount of attention to their children's schoolwork.
Reference:
Jimenez, E. and Y. Sawada. 1998. "Do Community-Managed Schools Work? An Evaluation of El Salvador’s EDUCO Program". Working Papers Series on Impact Evaluation Reforms, Paper No.8, Development Research Group, The World Bank, Washington, DC.
Schiefelbein, Ernesto, J. Swope, S.J. and P. Schiefelbein. 2000. "Cost-Effectiveness of Education Programs to Prevent and Eliminate Child Labor." (Unpublished memo).
Umansor, S., I. Soriano, M.R. Vega, E. Jimenez, L. Rawlings and D. Steele. 1997. "El Salvador’s EDUCO Program. A First Report On Parent’s Participation in School-Based Management". Working Papers Series on Impact of Education Reforms, Paper No.4, National Research and Evaluation Division Ministry of Education, El Salvador, Development Economics Research Group, Poverty and Human Resources, The World Bank, Washington DC.
|
