Bruno Frey's academic utopia

Bruno Frey has fallen into disgrace these days as he has been shown to play dangerous games of self-plagiarism, submissions to multiple journals and hypocrisy in describing the perverse incentives facing researchers. I have argued recently that he is living in a bubble that has now popped.

Apparently not. With this wife Margit Osterloh, he just authored a paper about the impact of rankings on academic publishing. Their argument is that the current emphasis on rankings pushes academicians to privilege publication to science. They want to decouple funding, tenure and promotion from any evaluation metric. Rather, scientist should be carefully selected at their initial appointment and then be given guaranteed funding and only be asked to evaluate themselves. This strikes as a very utopian view of academia, and in particular a view that surprisingly ignores the impact of incentives on motivation. While the Frey and Osterloh utopia may yield in a few cases the expected very innovative researchers willing to take substantial risks along new paths, most would free-ride to a large degree. The best example of this is the French system of "research associates" on the national research foundation, who are appointed right after their doctorate for a lifetime position of researcher with no other significant duties. While this system has yielded some success stories, the research impact of these associates is rather dismal compared to researchers elsewhere who are subject to regular evaluations using publication metrics.

Reading through the paper, I could not resist to see the irony in many of the arguments, where Frey could be really writing about himself. A few quotes:

In academia, examples can be found (e.g. the ‘slicing strategy’) whereby scholars divide their research results into a ‘least publishable unit’ by breaking the results into as many papers as possible in order to enlarge their publication list.

Of course, Bruno Frey and his students are big specialists in slicing.

there is evidence showing that academics with the highest score in publication rankings score only modestly in a ranking based on their contributions in editorial boards (Rost and Frey, 2011).

Note that Frey was kicked off an editorial board for resubmitting a published paper elsewhere.

For example, a broad and international selection of reviewing peers is necessary in order to avoid cronyism.

Frey's colleague Ernst Fehr recently anointed one of his students with the most prestigious prize for an European economists. Frey and Osterloh argue that awards cannot be manipulated, while citations metrics can. I do not think this is true, in fact awards are easier to manipulate because they are determined by small committees. Citations come from the whole research community.

Another example is editors who encourage authors to cite their respective journals in order to raise their impact rankings

Frey requires this for acceptance in the journal he co-edits, Kyklos. At least, the paper does not mention self-plagiarism this time.

Is this paper a mea culpa? It certainly does not read that way. As in his previous writings on the publishing game, he comes across as someone about it all, who tell everyone how things should be while claiming the moral high ground. The paper was completed on August 24, 2011, thus well after all the conflagration about the hypocrisy of Bruno Frey, yet it does not show anything about it. Bruno Frey has not learned a thing. And, do you want to bet that he is submitting this to journal, taking away space that young researchers need to get publications for tenure, as he has argued before? In fact this particular paper would fit much as a blog post than into a journal.

Addendum: And they are doubling up with another paper, entitled "Input Control and Random Choice - Improving the Selection Process for Journal Articles" with the following abstract:

The process by which scholarly papers are selected for publication in a journal is faced with serious problems. The referees rarely agree and often are biased. This paper discusses two alternative measures to evaluate scholars. The first alternative suggests input control. The second one proposes that the referees should decide only whether a paper reaches a minimal level of quality. Within the resulting set, each paper should be chosen randomly. This procedure has advantages but also disadvantages. The more weight that is given to input control and random mechanism, the more likely it is that unconventional and innovative articles are published.

You read it right: instead of peer review, they advocate complete tyranny by editors who randomize over a select set or, alternatively, to decide early who is worthy publishing and then let the chosen few do as they wish. Which is how Bruno Frey and his lackeys have been operating. But at least Frey and Osterloh have had the decency to withdraw that last paper (which is why I print the abstract above).

How to publish prolifically

I dedicated several posts to Bruno Frey and his chronic self-plagiarism. In retrospect, one should have seen that something was fishy from the mere fact that he was simply publishing too much for it to be normal, 600 articles by his own count. It is not possible for an academic, at least in Economics to be that productive. Yet, there are some who seem to be on a similar path.

Take, for example, Michael McAleer. He is an Australian econometrician who had a very respectable career in the 1980's, publishing in the AER with Adrian Pagan (and a homophone of Paul A. Volcker), four Review of Economic Statistics, a Review of Economic Studies, an Economic Journal and plenty of other decent publications. McAleer get elected into the Academy of Social Science in Australia in 1996. Then the quality of the publications dips, as he must be facing the same loss in productivity so many in the profession suffer in their forties. Still a good stream of publications.

Then suddenly, a burst of historic proportions.

Let us first look at working papers. According to his RePEc page (that is all I could find, a 2004 CV has 32 pages despite having no publications listed): 12 in 2008, 45 in 2010, 39 in 2010, and so far 15 in 2011. And these are according to their titles, at least, distinct papers. How can one do this? First McAleer has many co-authors, but he is no Paul Erdős, as his has a small set of regular collaborators. Second, many of the papers are about the same theme, with small variations: journal impact, with applications to neuroscience, tourism studies, econometrics, and economics in general, including one that I discussed. There is nothing wrong with this, except that entire sections are copy-and-pasted from one paper to the next. His other papers, for example on tourism demand in the Far East, are incredibly thin slices of research.

But these are all working papers, and he is free to write all this as long as he does not pretend this is all original and substantially new work when submitting to journals that have such requirements. McAleer is, however, also publishing avidly, although luckily few of the papers mentioned above get placed, and then only poorly. In terms of publishing, he has found another niche, the Journal of Economic Surveys:

• 2011, issue 2: 1 article
  
• 2011, issue 1: 2 articles
  
• 2010, issue 1: 2 articles
  
• 2009, issue 5: 2 articles
  
• 2007, issue 5: 1 article
  
• 2006, issue 4: 3 articles
  
• 2005, issue 5: 1 article

The journal has 5 issues a year, averaging 7 articles in each issue. That is a remarkable publishing success in a generalist journal. It turns out frequent co-author Les Oxley is the editor, who himself does not hesitate to frequently publish in his own journal. I counted 17 articles of non-editorial nature, several over 60 pages long, as well as 7 reports on conferences he attended.

A good number of those articles are titled "The Ten Commandments of ...", which I find rather pretentious. I was curious about The Ten Commandments for Academics, which could reveal some of the motivations of McAleer. They are:

1. choose intellectual reward over money;
2. seek wisdom over tenure;
3. protect freedom of speech and thought vigorously;
4. defend and respect intellectual quests passionately;
5. embrace the challenge of teaching undergraduate students;
6. acknowledge the enjoyment in supervising graduate students;
7. be generous with office hours;
8. use vacation time wisely;
9. attend excellent conferences at great locations;
10. age gracefully like great wine.

What I find interesting here is what was not considered. I think a better alternative, and one that would condemn much of what McAleer is doing, are due to Wesley Shrum:

1. Thou shalt not work for deadlines;
2. Thou shalt not accept prizes or awards;
3. Honor thy forebears and colleagues regardless of status;
4. Thou shalt not compete for recognition;
5. Thou shalt not concern thyself with money;
6. Thou shalt not seek to influence students but to convey your understandings and be honest about your ignorance;
7. Thou shalt not require class attendance or emphasize testing;
8. Thou shalt not worry about thy own intelligence or aspire to display it;
9. Thou shalt not condemn those with different perspectives;
10. SEEK TO UNDERSTAND THE WORLD.

These are principles about integrity, about changing the world and putting the scientific interest ahead of oneself. McAleer, rather, seems keen on clogging journals and working paper series with useless drivel, showing off and self-plagiarizing. At least for the latter part of his career, I do not see a positive externality from his efforts.

To come back to my initial question, to be prolific: find willing co-authors and editors, slice thinly, copy-and-paste, and do not think too hard what academia is about.

Do we need awards in Economics?

I do not like awards. They always create jealousies, and one cannot help that whenever a committee is involved, something may not have gone right. I am thus quite happy that economists give very few awards. It makes their CVs look bad compared to other scientists, but that is the price for a relative peace in the profession.



But we still have some prizes. The Nobel one, which is not really part of the Nobel family but is still attributed much prestige is always under much scrutiny. And in the end, the right people tend to win it. There have been a few controversial cases, Myrdal, Hayek, Buchanan and Ostrom come to mind as example where quite a few eyebrows were raised, but overall this award works well.



The American Economic Association gives an award that is considered to be even more difficult to get than the Nobel Prize: the Clark Medal, given to an American aged under 40. It is difficult to get because only one is awarded every year (no joint winners) and until recently it was given every second year. When comparing to the Nobel Prize, it is relevant to understand that American get a vast majority of them.



Now let us have a look at the past few year for the Clark award:

2011: Jonathan Levin, PhD MIT, Faculty at Stanford

2010: Esther Duflo, PhD MIT, Faculty at MIT

2009: Emmanuel Saez, PhD MIT, Faculty Harvard then Berkeley

2007: Susan Athey, PhD Stanford, Faculty at MIT then Stanford and Harvard

2005: Daron Acemoglu, PhD LSE, Faculty at MIT

2003: Steven Levitt, PhD MIT, Fellow at Harvard then faculty at Chicago

2001: Matthew Rabin, PhD MIT, Faculty at Berkeley

1999: Andrei Shleifer, PhD MIT, Faculty at Princeton, Chicago and Harvard



Do you see a pattern? Well I do, and others have, too. I am not saying these awardees are not bright and promising economists, but is there really no other qualifying economists that could have received it? Of course, John List comes to mind, who has no connection with MIT (or Harvard). But it actually worse than that. The award is given by a small committee, designated by the AEA. The AEA leadership is stacked with people with MIT and Harvard connections, so they also nominate their friends to the various committees, and you see the result.



It is even worse. In 2010, Ester Duflo was considered to be in the pool of strong candidates for the award. Guess who was on the awarding committee? Abhijit Banerjee, her PhD advisor, frequent co-author and colleague at MIT. In such a situation, an ethical person would decline the invitation to serve on the committee. That does not seem to have crossed the mind of Banerjee, who may be used to this cronyism.



There is another award, this time given by the European Economic Association: the Yrjö Jahnsson Award, to an European economist under age 45. It is given every two years, but can have several recipients. This awards has looked much cleaner because the committees and awardees have been distributed all over Europe. Europeans are indeed very sensitive to this. The last one was a shocker, though. Armin Falk won it to the surprise of many. And guess who chaired the awarding committee? His advisor, Ernst Fehr. Again, ethics would have indicated that if Falk had a chance of winning it, Fehr should have recused himself not just from chairing the committee, but from participating in it. In retrospect, this is not Fehr's first wrongdoing: two years earlier he was also on the committee when Fabrizio Zilibotti co-won the award. Zilibotti is a colleague of Fehr in Zurich.



I think we should do away with these two awards. It simply does not work.

On the ethics of research cloning

Even though the Journal of Economic Perspectives recently went open access, a move the American Economic Association should be applauded for, I am still receiving physical copies. It is a nice journal to read while lounging in the garden or on a plane ride. The last issue has as usual a good set of interesting articles, including one I had reported on earlier when it was still a working paper. But while checking what I had said about it, I noticed something rather odd: the paper I discussed was ultimately published in the Journal of Economic Behavior and Organization. I had to investigate.

The two papers are by Bruno Frey, David Savage and Benno Torgler. They both report on the sinking of the Titanic and discuss the characteristics of the passengers who survived versus those who perished. Both papers come to the same conclusions. The texts are different, though, and the published regressions are slightly different, with no explanation why, because there is no reference to the other paper. One has therefore to read in much detail to understand what the contribution of each paper is, if there is any.

All this is very fishy. It really looks like the authors are playing games here, trying to get multiple publications out of the same work. They do not mention the other work to fool editors and referees into thinking these are original contributions, as required for any submission to those journals. They tweak the results and rewrite the text so that they cannot be accused of blatant self-plagiarism. This is unethical behavior, but it is not unheard of in the profession.

But like a late-night infomercial, there is a bonus. Looking at the author's CVs, I notice that they have a third publication with the same topic and results, in the Proceedings of the National Academy of Sciences. Bruno Frey has also published two short pieces in German in magazines prior to the academic publications: 1, 2, both pdf.

Now, who are the authors? David Savage is a PhD student at Queensland University of Technology. He must have been following orders of the more senior authors, either without realizing their unethical behavior or watching in horror and not being able to do something about it. Let us give him the benefit of the doubt. His adviser is Benno Torgler, who has already an impressive track record for someone whose first refereed publication was in 2002. His RePEc profile lists 105 working papers and 52 journal articles. Looking at the published works, it seems to like to revisit previous papers by adding new twists to them. Nothing wrong with that, but it may explain why there is no major hit in the publications. There is simply too much slicing and no single slice is a major contribution worth a good publication. But early in his career, he published a series of articles on tax morale using the World Values Survey. Using the same data and the same methodology, he managed to publish several articles whose distinguishing feature is only that they look at a different set of countries: Asia, transition countries, Canada, Latin America, and possibly more. While I must confess that I have not read the papers in detail, there is simply too much material, and Benno Torgler may be innocent, I still find these patterns very disturbing.

It took me some time to figure out where Benno Torgler earned his doctorate. It is at the University of Basel, under the supervision of René Frey (Basel) and Bruno Frey (Zurich), who are brothers, after undergraduate studies at the University of Zurich. Which bring us to Bruno Frey. He is a researcher of international recognition, mostly for his work on welfare economics, happiness research, and critiques of fundamental assumptions in economic models. He credits himself with over 600 published articles and books, an astounding number in Economics. Of course, if this number comes about by slicing papers or republishing known results as described above, this number is less surprising. Looking at his list of major articles, one can surely suspect something is not quite right. I do not have the time (or the will) to go all of this, but there is indeed a lot of rehashing the same themes, which is OK when one uses new data sets or new approaches. But seeing those quantities, that seem unlikely.

Another aspect that I find disturbing in Bruno Frey's record is that his recent work has been railing against the tendency of academics (and especially their administrators and grant makers) to look for quantifiable evidence of their productivity, what he calls "evaluitis." He writes against the pressure to publish and the prominence of rankings of research output. I have reported about some of this writing myself (1, 2, 3). But again he seems to be repeating himself a lot, even in published articles, essentially criticizing a game that he seems to be excelling at. Either he is sarcastic or hypocritical, I cannot decide.

I realize the accusations I am making here can have severe consequences. But I am only accusing, not condemning. I leave the reader the opportunity to make her own opinion, as I have linked to plenty of evidence. I hope to be proven wrong, that these three individuals are indeed extremely innovative and productive. But from what I have seen so far, my prejudice is strongly negative in this regard.

Update (Sunday): I have been alerted that there is a fourth publication about the same Titanic study, in Rationality and Study.

Further update: A follow-up post.

Fraud cycles

The evolution of crime over time is much studied, and there is a lot of agreement that demographics are very important for many crime categories that are the "specialty" of young adults, like violent crimes. Fraud, however, cannot be tied to a particular age category, yet fraud statistics exhibit a remarkable cyclical pattern, a pattern that is not correlated across fraud categories or with the business cycle. What could give rise to such cycles?

Jiong Gong, Preston McAfee and Michael Williams come up with a theory that can rationalize these cycles. Once a lot of fraud cases make the news, people become more careful and new laws are put in place, which makes fraud more difficult. As fraud then disappears from the picture, people become less careful, and fraudsters find new and innovative ways to make money, and statistics show a comeback. That reminds me of privatization-nationalization cycles.

Of course, fraud statistics are not perfect. Indeed, they only measure fraud arrests, not fraud occurrence. One could argue that more people get arrested for fraud when victims are more vigilant, not less. That would be an entirely different story of fraud cycles.

Ethics in Economics

Ethical behavior in the Economics profession has so far never been codified or even much thought about. Indeed, economists have the reputation to be easy to buy, just ask any lawyer needing a "expert" to testify. But the recent movie Inside Job has brought to the general public the issue of conflict of interest in the profession, and the American Economic Association seems to have finally picked up the ball. As it is set to draw a code of ethical standards for the profession (New York Times article), one has to evaluate the large task ahead given the many ways in which economists have breached basic scientific conduct guidelines, with rather small consequences. Let me quickly go through a serious of examples to show how little unethical behavior has been sanctioned so far.

The prime example is, of course, the case of Andrei Shleifer, Harvard professor who was a major consultant for the Russian government during the massive privatization of its state-owned industry. Shleifer amassed substantial wealth during this process in ways many think where not legitimate, and in particular the US Department of Justice thought so. In the end, Harvard paid much of the fines, Shleifer is still a professor there and probably one of the richest people in the profession (more).

Or Florencio Lopez-de-Silanes, coincidentally student and frequent co-author of Shleifer, who as director of the Yale Center for Corporate Governance ironically double-billed US$150'000 worth of travel expenses. He was fired despite tenure, but landed on his feet, still active in the profession as a professor in France. (more).

A bit too frequently, the issue is that if someone gets caught with the hand in the cookie jar, he just resigns and lands a job elsewhere where no one suspects anything. The next examples are not documented online or in print, so I will not give names.

Professor A has help full-time appointments at two universities, with none of the two knowing about it. Colleagues grew suspicious when Prof. A would have strange schedules, rearrange classes in odd ways and never be available. The scheme was finally discovered after a few years, and Prof A was summarily fired. However, a few years later he was again holding two full-time positions at different universities...

PhD student B gets caught in massive plagiarism, we are talking here about copy-and-pasting a dozen full articles. Administrators get alerted and promise to dismiss the student. Student B still graduated with a PhD and is now a "respected" free-lance analyst.

PhD student C gets caught copy-and-pasting the entirety of a term paper from a published article. During the investigation it turns out Student C is a repeat offender, the teacher in the previous case failed to follow the proper procedure, Summarily dismissed, the student enrolls in another PhD program and is now close to graduate and already has a research job.

Professor D publishes as sole author the term paper of a student. Gets caught and dismissed. Moves to another country and now holds a chair.

Professor E plagiarizes and gets caught. Is asked to leave from a well-respected US department, applies for a well-endowed chair in another country. The university is impressed to see a candidate of such prestige and hires Professor E, oblivious to the baggage.

Is the AEA going to be able to take care of all these cases? Most likely not, as it is going to focus on conflict of interest. For the other cases, it is going to argue that every university has internal procedures to enforce ethics and in particular plagiarism. But as I showed with the cases above, this is of little use when offenders can simply walk away and act like a virgin elsewhere.

RePEc has a new initiative that would take care at least of the plagiarism cases by exposing them. As the blog post explains, the goal is the shame plagiarizing authors (after proper procedures have been followed) as was as shaming administrations in imposing proper sanctions. I applaud this initiative and I hope it will set an example and reduce the staggering among of plagiarism going on, and perhaps instill more ethical behavior into economists.