How can two rankings looking at the international orientation of universities be so far apart?

The question was brought into sharp focus by the publication of two contrasting higher education rankings for ‘internationalisation’ within weeks of each other.

Times Higher Top 100

The first, by the established world rankers at the Times Higher Education, or THE, magazine, came out on 23 January 2015.

It claimed to reveal the 100 most international universities in the world.

Swiss universities grabbed the top places with École Polytechnique Fédérale de Lausanne, or EPF, pictured, gaining first place, followed by ETH Zürich and University of Geneva in third spot.

Ten British universities, led by Royal Holloway, and five Australian institutions made the top 27 THE top performers for internationalisation.

New boy U-Multirank

Then, on 9 February, along comes the new boy on the block, U-Multirank – the consortium backed by the European Commission with the aim of breaking the mould of world rankings.

And they did so in style, with a new ranking of higher education institutions based on four elements of ‘international orientation’.

Instead of an overall winner or Top 100, U-Multirank listed 27 gaining four A-grades.

Only four institutions made the top 27 in both university rankings – EPF de Lausanne in Switzerland, the University of Innsbruck and the University of Vienna, both in Austria, and KTH Royal Institute of Technology in Sweden

Many established commentators and education figures were surprised, if not shocked, to see France grabbing six of the 27 top slots in the U-Multirank rankings.

Britain’s sole representative at the U-Multirank top table was the University of Liverpool and no universities from the United States or Australia achieved four A-grades.

So what’s going on?

Well, for one thing they were looking at different things.

And, secondly, they were looking at different universities.

Let me explain…

Phil Baty of Times Higher Education

Phil Baty of Times Higher Education

The Times Higher’s world rankings editor, Phil Baty, freely admitted to me in an interview for University World News that they just lift international outlook data from their World University Rankings to create their list of the most international universities.

THE focus on research-intensives

Their world rankings are unashamedly focused on research-intensive international institutions and they limit themselves to looking at around 1,000 global universities to compile their ‘Top 400’ World University Rankings.

International outlook is but one factor in the THE table, and only accounts 7.5% of the total scores, with the Times Higher seeing ‘internationalisation’ as a university’s international student numbers, its percentage of international staff and the proportion of its research papers published with a co-author from at least one other country.

Limited U-Multirank database

U-Multirank has its own limitations, with one of the project’s leaders, Frank Ziegele, telling me for the University World News piece they only compared the performance of 237 PhD-awarding institutions with a student body comprising more than 7.5% of international students from their database.

He said this was an attempt to compare ‘like with like’ – but the obvious drawback is that only those universities supplying data to U-Multirank have a chance of gaining top grades.

Currently U-Multirank representation is weak in the UK and the USA in terms of institutions supplying data and many of the big brand names, like Oxford and Harvard, are missing out.

Mobility is a two-way street

The U-Multirank definition of ‘international orientation’ is slightly wider than the Times Higher’s.

As well as the number of international academic staff and international joint publications, it also includes international doctorate degrees. But perhaps the most important difference between the two rankings is when it comes at looking at student mobility.

The THE suffers from what I see as an Anglophone problem of just counting incoming international students – while U-Multirank takes a welcome step-forward and counts both incoming and outgoing students.

This helps to explain why some of the big players participating in U-Multirank didn’t make their top 27 slots. Certainly it was the case of ETH Zürich, which came second in the THE league table for international outlook, but narrowly missed U-Multirank’s top list because it got a ‘B’ grade for student mobility.

Rankings can be very different

Frans van Vught, another U-Multirank project leader, said: “Rankings can be very different. Ours is not a league table like the Times Higher’s.

Frans van Vught of U-Multirank

Frans van Vught of U-Multirank

“We don’t create composite indicators or have different weightings, like 7.5% for international outlook as the THE does in their world university rankings.

“We have four categories and give the top 25% an ‘A’ grade for each indicator, the next 25% a ‘B’ and so on. If you get ‘A’ grades in all four you get top marks for internationalisation.”

U-Multirank does look at publicly available data, such international joint publications, but clearly those participating fully – like many of the French institutions have a better chance of getting the best grades, explained Frans.

“Interestingly, where data is publicly available the British and Americans don’t necessarily do as well as one might expect. For international joint papers, Oxford was the top British university coming in at 35th in that category. Caltech, the California Institute of Technology, tops the THE world university rankings, but came in at 118th for international joint papers in U-Multirank”, said Frans.

Transparency

What the two rankings tell us is that there is more than one way of judging how international university is.

The Times Higher restricts itself to the big comprehensive research-intensives while U-Multirank widens the net to include smaller specialist institutions, many of whom only teach at the postgraduate level.

The reluctance of many UK and US universities to embrace U-Multirank could be seen as an issue of transparency and fear that reputations may be threatened if you look at things in a different light.

____________

For more on U-Multirank, see my report ‘France dominates new rankings for internationalisationin University World News.

_____________