Blog

By Aïcha Sidi, Pierre Varly

Translation from french to english by Pierre Stjepanovic

This post was drawn from the end of study project « Identifying the macro and micro factors influencing educational quality (Structural Equation Modelling, Partial Least Square) », done by Aïcha Sidi for the graduation of her statistical engineering degree at INSEA, Rabat. The full report is available RAPPORT DE STAGE-10 juin.

Introduction

In the wake of the 2000 Dakar Forum on Education For All, 2002 saw the creation of the Fast Track initiative, renamed Global Partnership for Education, in 2009. This multilateral initiative aims, to get countries on the road to universal school enrolment through working out sectorial plans, coordinating both international and local aid better as well as bigger financing.

The Global Partnership for Education published, on 21 November 2012, a Results Report showcasing the evolution of the participating poor countries and others (so-called eligible) and taking stock of ten years of intervention. Chapter 4, written by yours truly, shows considerable learning quality problems and makes proposals to better take into account the results of studies on achievements in educational policies. Chapter 3 focuses on enrollment and completion and offers a history of indicators as well as forecasts based on demographic projections.

In Parallel, Similarly, a draft graduation from Rabat School of statistics Rabat (INSEA), produced from February to June 2012 by Aïcha Sidi, now a statistical staff at Varlyproject, has embarked on modeling completion rates and data on school achievement and, incidentally, on measuring the impact of the Global Partnership for Education on said rates. The aim is also to highlight the educational policy parameters that contribute to greater academic achievement. Is there some kind of universal model to follow to achieve universal enrollment or do we have to devise appropriate and purely domestic solutions?

Fast Track indicative framework, an essential tool

As soon as 2003, the Fast Track Secretariat (renamed GPE) has proposed a set of benchmarking indicators to achieve universal primary education. These indicators have been identified by analyzing the trajectories of those developing countries that have managed to achieve universal primary education in recent years.

These analyzes were conducted for the 2003 report: A Chance For Every Child, written by Barbara Bruns, Alain Mingat and Ramahatra Rakotomalala, on behalf of the World Bank. On 55 countries of more than one million people, eight are identified as having attained universal primary education (UPE) (more than 90% of children complete primary school), four countries in Eastern Europe and four Central Asian countries (Albania, Azerbaijan, Bolivia, Indonesia, Kyrgyzstan, Uzbekistan, Vietnam and Zimbabwe). Yet these countries are excluded from the analysis as indicated in a footnote page that we think worth reproducing here:

“Four of the eight countries in the sample that met the criterion were countries in Eastern Europe and Central Asia. Given the unique institutional legacy of these countries, it would bias the analysis of success factors if these countries retained this weight in the successful group.”

Although presented in a scientifical perspective the argument is due more to political reasons (institutional legacy). It has been avoided at all costs to make countries from the former Soviet bloc appear as models. Remember that this report is financed by the World Bank. The study retains thus 10 countries with a “relative success” i.e. a completion rate of over 70%.

The report identifies key indicators that shape the indicative framework, originally called Benchmarks for Primary Education Efficiency and Quality:

• Average student teacher ratio
• Average Teachers’ wages
• Repetition rates
• Share of expenditure on primary education

But also:

• The share of primary current expenditure (excluding wages)
• The unit costs of school construction
• % of pupils enrolled in private institutions

The framework thus construed makes sense. A country with overpopulated classrooms, a high repetition and low education spending simply cannot achieve full school enrolment of its children. The relationship between repetition and dropout symbolized by the following graphic is very clear.

Repetition and dropout rates by country

Empirically, it is interesting to test this framework on new data, or even seek to expand it. Are the conditions imposed by this framework both necessary and sufficient to produce an educational policy leading to the expected outcomes?

Above all, we have to reintroduce poor countries which have achieved universal primary education, unjustly removed from analyzes, which can broaden perspectives.

Completion rates: an end in itself?

The countries that allowed an entire age group to reach the end of primary school have not as such attained the goals of Education for All, which states that education must be adequate in quality. If in the matter of completion rates targets can be easily established, the concept of quality is not amenable to measurement, unless reduced to concepts such as the quality of learning outcomes and the capacity of one’s education to generate income and growth (external efficiency).

Studies of educational achievement have increased in recent years with the introduction of new tools to measure basic skills in literacy and numeracy that have created a sort of competition between international measuring programs. The figures, which sometimes involve small-sized sample show significant differences between grade levels in the South and the North. It is estimated, based on data from SACMEQ and TIMSS international survey placed on the same scale, that a 2nd year student from the North has a level equivalent to a 6th grade student in the South. See GPE (2012) p.116

More specifically, these enormous difficulties result in a very high proportion of non-reading pupils (75%), unable to read aloud any word in their own language. Although studying sectorial plans and joint review reports from governments and technical and financial partners shows that more data on the quality of education are produced and used, sometimes politicians remain deaf and blind to these facts. Like their dumb readers.

It is not possible to achieve a complete enrolling of all students in such conditions of poor teaching. Parents who find that their children cannot read and learn nothing may decide to withdraw them from school. Social demand is thus affected by the poor quality of education. Even though forced enrollment may have been a leverage factor in some countries, universal primary education cannot be achieved without social demand. Until recently, the indicative framework Fast Track and its derivative, the Results Framework of the Global Partnership for Education contained very little quality indicators, with the exception of actual teaching time.

Since then the Result Framework, constantly being revised, contains indicators such as:

• % of students who can read in the 2nd year;
• % of students who have acquired basic skills at the end of the cycle;
• The number of textbooks per student.

But it was not until almost 2009 that these indicators were included in the international mechanisms and 2012 for UNESCO to publish comparable data on textbooks in Africa.

These findings beg the question, Do we need to change our tune, temporarily curb the explosion in students numbers, to ensure a minimum quality of education for all before resuming mass schooling as a universal vocation? Despite the relatively optimistic forecasts of the GPE, the completion rates curves show signs of slowing in some countries (Togo, Cameroon, Ivory Coast, for instance), which can be attributed to poor teaching conditions, especially repeated teachers’ strikes.

Forecast evolution of completion rates by group of countries

<<–Source: GPE 2012

Forecasts clearly show that universal primary education will not be achieved by 2015 or even in 2020. These projections are based on demographic models that are not to be called into question but rather complemented by another method. The idea is not to predict future rates but to identify poor countries that reach good rates, describe the policies that they follow and provide a model that estimates a possible completion rate under given conditions.

A revealing lack of data

Countries received technical and financial support to provide better data and since 2000 and the information has greatly improved be it in the numbers of enrollment data figures or the level of student achievement. However, data are not always disseminated, sometimes due to a lack of transparency it has to be said, or simply not produced when the country is in a crisis (and the central government is not “master” of the territory).

The countries that provide the less data are often those who combine political and institutional difficulties or suffer from governance problems. This link is confirmed by the correlation between corruption index (Transparency International) and response rates to international investigations. Countries participating in the Global Partnership for Education provide more data than others and are more involved in international surveys of achievement. This is a first positive effect of the Partnership.

The three countries that provide the less data (North Korea, Somalia and Zimbabwe) have very important governance problems, which suggest a causal link between data supply and governance. While this may seem obvious, the relationship between governance and provision of data can introduce bias in the analysis, excluding countries with specific characteristics (selection bias). But unlike the work done here in 2003 during the preparation of the indicative framework, this is no a priori exclusion for reasons other than scientific.

In addition, it was necessary to make some imputations of missing data to fill some holes and it was not possible to make a detailed analysis of the evolution in time of completion rates or obtain comparable data on learning on an adequate sample of countries.

The scope of analyses

The purpose of this study is to highlight the variables that have an impact on the quality of formation of human capita. This work is not a repetition of the many studies that deal with the relationship between education and growth, in particular the work of Hanushek & Kimko (2000), Lee & Barro (2001) and Altinok (2010) but also Doudjidingao (2011) on Africa. Indeed, these works that fail to stabilize a model and sometimes contradict themselves, mixing developing and developed countries, the last taken as models. This seems a fallacy to compare Finland with Cameroon, because economic differences put aside, there are constraints drawn from the range of socio-linguistic situations. 280 languages are spoken in Cameroon, population growth is high and much higher than that of Finland and the countries are in very different situations, so much so that one may wonder about the relevance of replicating analysis and policies.

We therefore confined ourselves to a sample of countries of lower and lower middle income brackets as well as some of the upper middle bracket that either belong to the geographic areas of interest or seemed to have special educational systems: Botswana, Mauritius, Namibia, South Africa, Jordan, Tunisia, Thailand, Gabon, Maldives, Ecuador, Albania, Azerbaijan, Cuba and Latvia.

The models explain and predict well completion rates of the countries in our base but does not work on high income countries.

We can get a first glimpse of each country’s performance in education through this map that shows completion rates:

Completion rates map

<<–Source : Edstats, accessed to in June 2012

Countries of West Africa and Central Africa, French-speaking for the most part, are in the lowest category of completion rates, i.e. less than 71%. English-speaking African countries rank higher.

Introducing cultural factors affecting the quality of education

We will in addition to the variables mentioned in the literature, consider the demographic and cultural realities, in order to see their impact in achieving a good quality of education.

At the macro level, outside the indicative framework mentioned above, several factors have been identified in the literature as being necessary for a good quality primary education. This is the Gross Domestic Product (GDP) per capita and adult literacy rate. These two indicators are widely involved in explaining the completion rate because they are linked by causal relationships. Better per capita income can provide greater education and literacy rates stems from past schools enrollment (to which must be added literacy campaigns). The literacy rate is thus a kind of starting point or initial score.

In addition to this list of factors, we saw fit to add an index for measuring linguistic diversity which can be an obstacle to the quality of education. In fact, if students in the same class do not share the same mother tongue, it will be impossible to instill in them notions in the language they understand best (their mother tongue). It would also be difficult for the State to provide primary education in national languages when they are numerous.

We can also consider the rate of Internet users as a proxy for access to information; several websites not only facilitate the development of young people but also various kinds of learning. We therefore believe that this variable can have a positive effect on the quality of education. It is also an indicator of investment in new technologies and therefore development.

The index of Transparency International tells us about the level of corruption in each country or of transparency. Indeed, the more corrupt a country is, the more spending planned by the State for education will not really be invested on the ground. We therefore adjust the variable ‘education spending’ by a corruption index, a kind of proxy for the attrition rate between planned spending and spending that truly affect schools.

We will also test whether the size of the population has a significant effect on the quality of education. Indeed, outside of the federal states (such as India and Nigeria), the size of the administration of education is not really proportional to the size of the country. It can be assumed that the most populous countries (with comparable central government size) are more difficult to manage than less populous ones.

It is also important to add to that the effect of population growth. This is the average rate of population growth from 2004 to 2009. We believe that countries with high population growth encounter more difficulties in having a good quality of education than others. This relationship is mechanically proven when one considers the completion rate as analysis variable. Indeed, this rate is calculated as the new entrants at the end of primary school divided by the population of the age of completing the cycle.

It should also be noted that several countries have signed on to the Global Partnership for Education, so we will introduce a variable that measures the duration in years of participation in the Partnership. Finally, we calculated for each country a response rate or information quality score, understood as an indicator of governance.

Last but not least, the percentage of female teachers is also introduced in the analysis, based on the graphic conducted in March 2012 on this blog. See here

Most of the indicators are based on average per country over the period 2004-2009. Our study will be done using the method of “partial least square” which is adapted to the context of education. It makes it possible to circumvent the existence of collinearity present in the variables of education, by aggregating the latent variables.

Classifying and grouping factors

A principal component analysis is used to group factors along two axes which appear to be organized along a first axis of variables related to a better quality of education and a greater level of development in general and a second axis associated with more spending on education and governance issues.

Representation of variables in the foreground of the principal component analysis

<<–Source : authors

In addition to the variables identified in the literature as indicating a good quality of education, namely literacy rate of parents (literacy), the rate of female teachers (female) and per capita income (loggdp), we note the rate of internet users (internet) is also related to the quality of education while corruption (corrupt) and the growth rate of the population (popgrowth) are indicators associated with poor quality of education. We also see that the variables GPE and education spending are similar, suggesting that GPE countries actually spend more for education. A second positive point for the Global Partnership for Education. Similarly, the language variable is close to the cost variable, which seems to confirm our hypothesis that greater linguistic diversity requires more education spending.

Finally, two variables of the indicative framework, i.e. the pupil teacher ratio (PTR) and repetition rate (repet) are very eccentric and associated with a lower quality of education.

Positioning the countries as higher and lower achievers

The principal component analysis is used to project the countries in the design formed by our variables. Rather clearly countries are grouped according to their geographical areas (suggesting common problems and specific to certain areas), but also in terms of political system, which are represented by color codes.

The representation of countries in the plan also shows a distinction between the quality of education across countries in English-speaking Africa (Anglo), Arab countries (arab), countries of Asia (Asian), French and Portuguese-speaking Africa (Franc_lus), islands (island), the Latin countries (Latin) and the countries of the former Soviet bloc (Soviet).

It is these latter countries, some of which were missing from the analysis conducted in 2003, which stand at the top right for a better quality of education, both in terms of enrollment and spending.

We also see that the countries of French and Portuguese-speaking Africa spend correctly in education but do not have a good completion rates. Some may see it as a colonial legacy particularly in terms of repetition practices. So there are strong effects of types of political organization somewhat mixed with geographical groups. Without condoning collectivist organization systems that led to many abuses in the world (and even in Africa with the revolutionary people’s courts of Sankara by instance) and may have been prone to enrollment to force-feed certain values to the younger, should we completely ignore their successes in terms of education? One particularly thinks of Cuba, who is far beyond the other Latin American countries in international investigations on the achievements and enjoys a high literacy rate.

Identifying the performance factors

Since the adoption of such poorly democratic political systems appears neither desirable nor realistic, let us fall back on the detailed parameters that significantly influence the results.

Ordinary least squares

The regression model of completion rates by ordinary least squares gives only three significant variables:

• The adult literacy rate has a positive effect on the quality of education;
• repetition rate and class size have a negative effect on the quality of education.

The Fast Track indicative framework is thus still relevant today and it seems that we still have to insist on repetition reduction (10%) and limiting class sizes to manageable proportions (40 pupils maximum).

When the model is done again without the literacy rate, the rate of female teachers is significant with the same results.

Partial least square

This method allows the aggregation of collinear variables in a latent variable to get independent explanatory variables before applying ordinary least squares regression.

The effect of latent variables is summarized in the following chart. The blue color denotes a positive effect, while red denotes a negative effect. If economists speak of poverty trap, considering that some countries start from a level of economic development so low that they cannot take off, the important contribution of literacy rate to differences in completion rates suggests that some countries may also be trapped in an “ignorance trap.” However on a more positive note, this variable thereafter called “literacy” also includes the proportion of female teachers which has a strong significant effect on enrollment rates. It therefore seems that an important lever of educational policy, namely women’s access to the teaching profession, has been somewhat neglected in the purely economic approaches that have prevailed until now.

Effect of latent variables in the PLS model

<<–Source : authors

Some school systems promote repetition and large class sizes, we see here that it has a significant negative effect on the results. The graph above shows the relevance of the Fast Track indicative framework as it was developed in 2003 through the contribution of the factor category called School System.

In addition to the variables found in the literature, we observe a high rate of Internet access has a connection with completion rates, just as the Global Partnership for Education also has a positive impact on student enrolment. On the other hand, a large population or a high rate of growth is a huge brake on children enrollment. Linguistic diversity also has a negative impact on completion rates.

This finding should strongly encourage governments to promote education in national languages but also to implement policies to control the population growth. Achieving universal primary education in many countries of Central Asia, such as the former republics of the Soviet Union, is to be seen in conjunction with a low birth rate and an aging population in these countries. Similarly Vietnam has probably reached universal primary education thanks to its policy of birth control inspired by China but here limited to two child per family.

We also conducted a predictions charts which shows the reliability of our models and can also be very useful for countries participating in the Global Partnership for Education to set a completion rate target that reflects their own social and demographic parameters. In this model, we took into account the fragility of states listed by international agencies.

It therefore seems that the macroeconomic model fit well with the observed values. In 63 out of 90 countries, predicted completion rates deviate only ten points from the observed rate; in 36 countries there is only a 5 points gap. Successful countries are those which, in given social and economic terms, get a better completion rate than predicted by our model.

However, a macroeconomic study alone cannot fully explain the level of education and the results are to be primarily interpreted in terms of correlations, not cause and effect.

Nevertheless, this study allows us to formulate a number of recommendations:

1. Organizing study trips to Havana rather than Helsinki
2. Insisting on reducing class sizes and a better distribution of teachers (no more than 40 students per class)
3. Limiting repetition rate to a minimum (10%)
4. Recruiting women teachers through incentives
5. Developing the use of new technologies in education and in the population
6. Quantifying the cost of linguistic diversity
7. Implementing effective policies of demographic control
8. Better regulation to develop private schools
9. Promoting transparency and accountability in the education sector
10. Seriously considering a temporary halt to the explosion of enrollment rates to ensure quality education for all
11. Using the chart below to identify the best performing countries whose context gives good performances

For any questions, please contact Aïcha Sidi: aside@varlyproject.com

References:

Altinok Nadir (2010), ‘Do School Resources Increase School Quality?’, Working Paper IREDU.

Doudjidingao Antoine (2011), ‘Education et croissance en Afrique’, Etudes Africaines, L’Harmattan

Emmanuel Jacobowicz (2012), ‘Les modèles d’équations structurelles à variables latentes’, Cours de Statistique Multivariée Approfondie, Addinsoft/Xlstat

Eric A. Hanushek & Ludger Woessmann (2010) ‘The economics of international differences in educational achievement’, Working Paper 15949, National Bureau of Economis Research, Cambridge

Bruns B., Mingat A. et Rakotomalala R. (2003),A chance for every child , Banque Mondiale.

Disclaimer:

This post is drawn from the end of study project by Aïcha Sidi for her statistical engineering degree at INSEA, Rabat, whose complete version is available here. The interpretations and opinions contained in it are solely those of the authors. ©Varlyproject

Annex: Completion rates predictions given by the OLS model