An online ranking that compares the performance of
academics across all fields found that Karl Marx is the most influential
scholar and Edward Witten is the most influential scientist
Karl Marx is the most influential scholar ever, according to a discipline-corrected ranking system. Image: Wikimedia Commons/Marxists.org
Is theoretical physicist Ed Witten more influential in his field than the biologist Solomon Snyder is among life scientists? And how do their records of scholarly impact measure up against those of past greats such as Karl Marx among historians and economists, or Sigmund Freud among psychologists?
Performance metrics based on values such as citation rates are heavily biased by field, so most measurement experts shy away from interdisciplinary comparisons. The average biochemist, for example, will always score more highly than the average mathematician, because biochemistry attracts more citations.
But researchers at Indiana University Bloomington think that they have worked out the best way of correcting this disciplinary bias. And they are publishing their scores online, for the first time letting academics compare rankings across all fields.
Their provisional (and constantly updated) ranking of nearly 35,000 researchers relies on queries made through Google Scholar to normalize the popular metric known as the h-index (a scientist with an h-index of 20 has published at least 20 papers with at least 20 citations each, so the measure takes into account quantity and popularity of research). It found that as of 5 November, the most influential scholar was Karl Marx in history, ahead of Sigmund Freud in psychology. Number three was Edward Witten, a physicist at the Institute for Advanced Study in Princeton, New Jersey. The ranking appears on the website Scholarometer, developed by Filippo Menczer, an informatician at Indiana University Bloomington, and his colleagues Jasleen Kaur and Filippo Radicchi.
Universal metrics
“We think there is a hunger for this. Our colleagues use Google Scholar all the time, and yet it only shows the h-index," says Menczer. "We are constantly asking ‘how do we evaluate people in a discipline we don’t understand?’”
In October, Menczer's team published a paper arguing that the best statistical way to remove disciplinary bias is to divide a researcher’s h-index by the average of their scholarly field.
Using this correction, Marx scores more than 22 times the average h-index of other scholars in history (but 11 times that of the average economist). Witten has more than 13 times the average physicist, and so on. The effect is to ensure that those in, say, the top 5% of their discipline also appear in the top 5% of all scholars.
The idea is not new. Metrics experts have invented numerous methods to solve bias, often using averages based on age, journal and scholarly field. Normalized measures are available from commercial information firms such as Thomson Reuters.
First time for everything
But Scholarometer pushes boundaries in two ways. Most importantly, its normalized scores are freely accessible, unlike those of most sites. Thomson Reuters analyses are based on proprietary databases and cannot be made public. Another site, Publish or Perish, does return a variety of age and field-normalized metrics from public queries to Google Scholar — but only to one individual at a time. The problem is that Google Scholar blocks automated computer programs that hit it with multiple queries, making it impossible to collate scores.
The Indiana team’s solution is to create an automated program that does not query Google Scholar itself, but rather scrapes the results of individual Google Scholar queries placed through a Scholarometer browser extension. Over years, they have built up a dynamic public database, with h-indices constantly revised as new Google Scholar queries come in. Menczer says that an age-corrected h-index that allows comparison of scholars at different career stages may follow.
