Measure 4: Output

How ‘Output’ was measured and the results the study produced.

Output

A good higher education system provides the nation with a well-trained and educated workforce that meets the country’s needs, provides a range of educational opportunities for people with different interests and skills, and contributes to national and world knowledge.  To capture these desired outcomes we use measures of research output and impact, student throughput, the national stock of researchers, the number of excellent universities, and employability of graduates.  Each measure is now explained in turn.

 

  • Q1:  Total articles produced by higher education institutions, 2005-2009
  • O2: Total articles produced by higher education institutions per head of population, 2005-2009.    

We use the SCImago data, based on the Scopus database, that calculates research output from over 3000 research institutions classified as Government, Health, Higher Education, Private Companies and Other.   The entries on higher education institutions, which we use, comprise around two-thirds of the entries.   An alternative to O2 would be to use a productivity measure and divide output by the number of staff employed in higher education.  This was not used because of our concerns about the availability and international comparability of data on staff numbers.                                                

  • O3: An impact measure calculated from the SCImago database, 2005-2009.

The measure is a weighted average of the Karolinska Institute normalized impact factor for each higher education institution, where the weights are each institution’s share of national publications from higher education institutions. 

  • O4:  A measure of the depth of good universities in a country.

For this measure we use a weighted average of the number of institutions listed in the top 500 according to the 2011 Shanghai Jiao Tong index divided by country population.  The measure can be thought of as a rough indicator of the probability of a person in a country attending a university ranked among the top 500 in the world.  The weights used are the scores out of 100 for each university.  In the Shanghai ranking, universities ranked below the top 100 are banded in groups: 100-150, 151-200, 201-300, 301-400 and 401-500; within each band we use the average score.  

  • O5:  A measure of the research excellence of a nation’s best universities.  

The quality of a nation’s best universities sets national standards and facilitates knowledge transfer at the frontier of new research.  For this measure we again use the 2011 Shanghai Jiao Tong Index and average the scores for each nation’s three best universities, with a zero weight for institutions not in the top 500.

  • O6: Enrolments in tertiary education as a percentage of the eligible population, defined as the five-year age group following on from secondary education, 2009.
  • O7: Percentage of the population aged over 24 with a tertiary qualification, 2009
  • O8:  Number of researchers (full-time equivalent) in the nation per head of population, 2009                              
  • O9: Unemployment rates among tertiary educated aged 25-64 years compared with unemployment rates for those with only upper secondary or post-secondary non-tertiary education, 2009.  The measure is calculated as the ratio of the latter to the former in order for larger values to indicate the value of higher education and to accommodate cases where unemployment is higher for tertiary than secondary.     

 

Results

 

The United States dominates the total output of articles from higher education institutions. Chinese universities publish just over 40 per cent of the United States total; Japan and the United Kingdom 25 per cent.   On a per capita basis Sweden produces the most journal articles.  The next highest performers are Finland, Switzerland, Denmark, Australia, the Netherlands and Canada in that order. The nations whose research papers, on average, have the greatest impact are Switzerland, the Netherlands and the United States.  These countries are followed by the United Kingdom and Denmark.  

 

Using as the criterion the top three universities in a country, the United States and the United Kingdom have the world’s very best universities.  But on a weighted per capita basis the depth of world class universities is best in Switzerland and Sweden, with Israel and Denmark next in rank order.  

 

The extent of education and training is measured in two ways:  the flow of those currently being trained (O6) and the stock of the adult population with a tertiary qualification (O7).  As expected, developing countries perform better on the flow variable than the stock, but over time these will converge.  The variation across countries is much less for the flow measure than for the stock measure.  Korea has the highest percentage of young people enrolled in tertiary institutions but is ranked eighth on the percentage of the working population with a tertiary qualification, well behind the best performing country, Russia.  Other countries with high participation rates are Finland, Greece, the United States, Canada and Slovenia.   After Russia, countries with relatively large stocks of tertiary educated workers are Canada, Israel, the United States, Ukraine, Taiwan and Australia.

 

The stock of tertiary educated workers is a blunt measure of the performance of the tertiary sector.  Is the distribution across disciplines appropriate?  Do the qualifications meet the demands of a modern economy?  One measure of the contribution to the economy is the number of researchers per head of population.  This ratio is highest in Finland and Denmark, followed in rank order by Singapore, Norway, Japan, Sweden, Korea and the United States, but the United States figure is only 60 per cent of that for Finland.  Unemployment data provide a measure of the appropriateness of education and training.  The best performing countries are the eastern European countries Slovakia, Czech Republic and Hungary.  In three countries, Chile, Mexico and Indonesia, the unemployment rate was higher for those with tertiary education than those who completed only secondary school, perhaps indicative of a tertiary education sector that is not producing the needed mix of graduates.

 

The score for the broad category Output is obtained by averaging over the 9 output variables and giving a weight of 4 to total output (equivalent to a weight of one-third in the output variable).  On this basis the United States ranks first, followed by the United Kingdom and Canada.  The Nordic countries rank highly: Finland and Sweden are equal fourth, Demark is sixth and Norway twelfth.