Let’s face it: global rankings of universities are here to stay, despite their poor methodologies. Among them, the ranking by the Shanghai Jiao Tong University University probably deserves the award for the worst methodology. It gives a huge weight — 30 percent — to Nobel prizes won by the faculty and alumni, and a further 20 % for papers published in the journals Nature and Science.
Last year’s ranking, for example, included the University of Calcutta among those ranked between 401 and 500; IIT-Kharagpur too figured in this list, while IISc was among those with ranks between 301 and 400. The inclusion of UCalcutta was definitely because of the Nobel Prizes won by its faculty (presumably, C.V. Raman) and its alumni (Amartya Sen?). Thus, depite scoring poorly on every indicator of its current research activity — it actually scores zero for (a) publications in Nature and Science, and (b) highly cited papers — it was grouped with IIT-Kharagpur — and 98 others! — in the range 401 to 500. 
[Oh, just in case you are wondering, UCalcutta is out of the top 500 in this year’s ranking, while IISc and IIT-Kharagpur retain their last year’s rankings.]
Just because the Shanghai group has been in this business for the longest duration — four years! — its ranking is often quoted without paying any attention to its methodology. For example, Kaushik Basu used it in his recent BBC column:
A recent evaluation of universities and research institutes all over the world, conducted by a Shanghai university, has not a single Indian university in the world’s top 300 – China has six.
The Indian Institute of Science, Bangalore, comes in somewhere in the top 400 and IIT, Kharagpur, makes an appearance after that.
Now, within the US, there’s a number of rankings that (mis)lead to different conclusions about the excellence level of different institutions . In India, too, all the major newsweeklies (India Today, Outlook, The Week) rank our colleges, B-schools, engineering and medical colleges — with poor (and poorly spelt out) methodologies, each pointing to a different conclusion. When the situation is so complex within a country, the ranking exercise should be hopelessly complex at the international level. After all, the higher ed systems in different countries are different; more importantly, their funding patterns are different. How can one compare, for example, a hub-and-spoke and multi-campus university like the University of Delhi and a tech-centric ‘small’ university like CalTech?
Global ranking exercises, of course, don’t want to — and don’t want you to — look at the enormous complexity that arises from regional variations. By claiming to do the hard work on your behalf, they encourage your laziness!
For example, they want you to ignore a key fact about excellence: it’s always the individual research groups that are the repositories of excellence. Thus, even in supposedly moribund Indian universities, there are such islands of excellence in specific subfields. A good example would be the School of Chemistry in the University of Hyderabad, or the Department of Physics in the University of Pune. The rankings also have an inherent bias against small institutions (and India has quite a few of them). For example, places like the Raman Research Institute, the S.N. Bose Natiaonal Centre for Basic Sciences, or the National Center for Biological Sciences are outside the radar screen of most rankings, in spite of their great research groups, simply because they are small and niche players.
Of course, it is certainly a great feat to put together a large university, with a solid academic and research footprint in many fields. Thus, despite their shortcomings, these rankings do do a good job of identifying the top research universities. By the same token, the top 20 (or even the top 50) universities in these lists are not very different. It’s only when you move away from these top universities that you start seeing the effects of the rankings’ individual quirks.
* * *
 In the mid-nineties, there was a ‘ranking’ that took India by storm. Coming from someone (whose name eludes me now) in the University of Maryland, it placed IISc at No. 17 and several IITs in the top 100. It got a lot of play in Indian press for sometime; it was, fortunately, a one-off affair.
 For what it’s worth, a recent issue of Newsweek put out its own list of top 100 ‘most global universities‘. Washington Monthly has a sort of anti-U.S. News listing of U.S. universities based on a measure of ‘excellence’ using a different set of critieria.