Bruno Frey's academic utopia

Bruno Frey has fallen into disgrace these days as he has been shown to play dangerous games of self-plagiarism, submissions to multiple journals and hypocrisy in describing the perverse incentives facing researchers. I have argued recently that he is living in a bubble that has now popped.

Apparently not. With this wife Margit Osterloh, he just authored a paper about the impact of rankings on academic publishing. Their argument is that the current emphasis on rankings pushes academicians to privilege publication to science. They want to decouple funding, tenure and promotion from any evaluation metric. Rather, scientist should be carefully selected at their initial appointment and then be given guaranteed funding and only be asked to evaluate themselves. This strikes as a very utopian view of academia, and in particular a view that surprisingly ignores the impact of incentives on motivation. While the Frey and Osterloh utopia may yield in a few cases the expected very innovative researchers willing to take substantial risks along new paths, most would free-ride to a large degree. The best example of this is the French system of "research associates" on the national research foundation, who are appointed right after their doctorate for a lifetime position of researcher with no other significant duties. While this system has yielded some success stories, the research impact of these associates is rather dismal compared to researchers elsewhere who are subject to regular evaluations using publication metrics.

Reading through the paper, I could not resist to see the irony in many of the arguments, where Frey could be really writing about himself. A few quotes:

In academia, examples can be found (e.g. the ‘slicing strategy’) whereby scholars divide their research results into a ‘least publishable unit’ by breaking the results into as many papers as possible in order to enlarge their publication list.

Of course, Bruno Frey and his students are big specialists in slicing.

there is evidence showing that academics with the highest score in publication rankings score only modestly in a ranking based on their contributions in editorial boards (Rost and Frey, 2011).

Note that Frey was kicked off an editorial board for resubmitting a published paper elsewhere.

For example, a broad and international selection of reviewing peers is necessary in order to avoid cronyism.

Frey's colleague Ernst Fehr recently anointed one of his students with the most prestigious prize for an European economists. Frey and Osterloh argue that awards cannot be manipulated, while citations metrics can. I do not think this is true, in fact awards are easier to manipulate because they are determined by small committees. Citations come from the whole research community.

Another example is editors who encourage authors to cite their respective journals in order to raise their impact rankings

Frey requires this for acceptance in the journal he co-edits, Kyklos. At least, the paper does not mention self-plagiarism this time.

Is this paper a mea culpa? It certainly does not read that way. As in his previous writings on the publishing game, he comes across as someone about it all, who tell everyone how things should be while claiming the moral high ground. The paper was completed on August 24, 2011, thus well after all the conflagration about the hypocrisy of Bruno Frey, yet it does not show anything about it. Bruno Frey has not learned a thing. And, do you want to bet that he is submitting this to journal, taking away space that young researchers need to get publications for tenure, as he has argued before? In fact this particular paper would fit much as a blog post than into a journal.

Addendum: And they are doubling up with another paper, entitled "Input Control and Random Choice - Improving the Selection Process for Journal Articles" with the following abstract:

The process by which scholarly papers are selected for publication in a journal is faced with serious problems. The referees rarely agree and often are biased. This paper discusses two alternative measures to evaluate scholars. The first alternative suggests input control. The second one proposes that the referees should decide only whether a paper reaches a minimal level of quality. Within the resulting set, each paper should be chosen randomly. This procedure has advantages but also disadvantages. The more weight that is given to input control and random mechanism, the more likely it is that unconventional and innovative articles are published.

You read it right: instead of peer review, they advocate complete tyranny by editors who randomize over a select set or, alternatively, to decide early who is worthy publishing and then let the chosen few do as they wish. Which is how Bruno Frey and his lackeys have been operating. But at least Frey and Osterloh have had the decency to withdraw that last paper (which is why I print the abstract above).

Progesa: a success story thanks to academics

I have written a few times about the frustration when policy makers ignore the advice of economists. Yet, there are a few cases where economists were given free reign over the design of policy interventions, which not only allowed to obtain positive outcomes but also useful information for further study.

Nora Lustig reports about Progresa, the Mexican cash transfers program designed to elicit parents to send their kids to school and make sure necessary health check-ups were attended. From the start, the program was designed and administrated by people with an academic background. Progresa has worked remarkably well, to the point that it was not only not scrapped, as is usual, with presidential changes, its coverage also kept increasing. The only setback was a name change to Oportunidades. The critical ingredient to this success was the scholarly involvement, that not only designed it for success, but also provided the tools to measure this. And along with that a wealth of data that has allowed to understand even better what makes good intervention in practice.

The Bruno Frey Bubble

About four months ago, I reported about the apparent self-plagiarism by Bruno Frey, David Savage and Benno Torgler. I found the case particularly ironic, as Bruno Frey repeatedly wrote about the fact that the pressure to publish to get tenure can lead to scholar to unethical behavior, and about the lack of space in journals for young scholars to publish the articles needed for tenure.



The case has taken a much larger dimension now, as many more cases of self-plagiarism by Bruno Frey and his students have appeared (see many links in the comments on the post mentioned above). This raises two very important questions: 1) how could such a culture of self-plagiarism arise? 2) How could they get away with it for so long?



To answer the first question, I think we need to put Bruno Frey is the context of the German(-speaking) academic environment. At least in Economics and Business, the typical German professor publishes a lot of rather insignificant articles, in particular book chapters and "Festschrifts." These works are rarely original, and are not expected to be so. There is also a tradition of writing "educational" pieces that explain economic concepts, say the Edgeworth box or voluntary export restraints, for journals targeted towards professionals in industry and government (as well as students). Again, there is nothing original in there, except maybe the way something is explained.



Bruno Frey works within this paradigm. His work lacks creativity in the sense that he recycles a lot of his ideas for multiple publications, often copying extensively his own words. The differences is that he does that at a higher level than his German colleagues, in international journals that are actually read. And many of his original papers are in fact not that original it appears. If we take the Titanic paper as an example, the empirical exercise he performs is routinely done in undergraduate statistics classes with the same dataset. His contribution is pedagogical, he found a good and interesting way to explain something already present in the body of knowledge.



Like a bubble that keeps getting fed by self-fulfilling expectations, Bruno Frey built on his initial success and continued with this strategy and encouraged his students to do the same. And several of them have assembled remarkable portfolios that way. I mentioned that of Benno Torgler in my original post, but there are several others who got into positions that seem beyond the usual reach of a Swiss doctoral program.



There is another way in which this resembles a bubble. The Economics department at the University of Zurich has made considerable efforts over the past decade or so to become a program that can compete with the better departments in the world. It is certainly among the best in Europe. It did so by americanizing itself: dropping to a large extend the rigid chair structure so prevalent in German speaking universities, hiring internationally respect scholars and creating a proper PhD program with courses and exams. Bruno Frey has not followed this trend at all. In fact, he insisted on exempting his students from the course and exam requirements. The Frey group lives in a cocoon apart from the rest of the department, and lives entirely following the role model of Bruno Frey. Call this living in a bubble.



Or a cult. The interaction of Bruno Frey and his students is reminiscent of a prophet and his disciples who follow him everywhere and write down every word he utters. Well, I exaggerate somewhat, but this does definitely not look like a standard interaction between a mentor and his students. It looks like they follow him blindly, and with his everlasting confidence, he makes them follow his example in publishing.



But this bubble is now popping under the assault of widespread scrutiny from editors, the Economics community and an internal investigation at the University of Zurich. The second question of course is how it was possible for Bruno Frey to act so unethically for so long (he is 70). It appears that he has been caught in the past, but it never became public, or at least explicitly. For example, he has been booted out of an editorial board, but there was no mention of why, his name just disappeared from the list. Also, the journals he has been publishing in are often not prominent and thus not that well read. In fact, it looks like he targeted them so that the audience would not overlap, including editors and referees (the added bonus of this strategy that it satisfies the goal of increasing the pedagogical reach by reaching very different audiences).



Hiding this unethical may have been helped by the fact that Bruno Frey actually tried to present himself as an expert on publishing ethics in Economics. He has written about the perils of publication pressure and how this can lead to slicing papers into insignificant bits, to self-plagiarizing and other unethical behavior. He has complained loudly about the ranking craze which he has been so adept to exploit, both with his self-plagiarism and by requiring authors to cite other works in Kyklos to increase its impact factor. While he is certainly not the only editor to do so, it is ironic that he openly campaigned against such practices. Bruno Frey abused the moral high ground in which he pictured himself.



But as every lie that grows too big over time, this is unsustainable. And it will be less likely to happen in the future with initiatives like this one. Making this unethical behavior more visible will prevent it.



That said, self-plagiarism is not limited to the Bruno Frey group or German speaking economists. I will discuss soon another case that I find particularly enraging.

Do we need awards in Economics?

I do not like awards. They always create jealousies, and one cannot help that whenever a committee is involved, something may not have gone right. I am thus quite happy that economists give very few awards. It makes their CVs look bad compared to other scientists, but that is the price for a relative peace in the profession.



But we still have some prizes. The Nobel one, which is not really part of the Nobel family but is still attributed much prestige is always under much scrutiny. And in the end, the right people tend to win it. There have been a few controversial cases, Myrdal, Hayek, Buchanan and Ostrom come to mind as example where quite a few eyebrows were raised, but overall this award works well.



The American Economic Association gives an award that is considered to be even more difficult to get than the Nobel Prize: the Clark Medal, given to an American aged under 40. It is difficult to get because only one is awarded every year (no joint winners) and until recently it was given every second year. When comparing to the Nobel Prize, it is relevant to understand that American get a vast majority of them.



Now let us have a look at the past few year for the Clark award:

2011: Jonathan Levin, PhD MIT, Faculty at Stanford

2010: Esther Duflo, PhD MIT, Faculty at MIT

2009: Emmanuel Saez, PhD MIT, Faculty Harvard then Berkeley

2007: Susan Athey, PhD Stanford, Faculty at MIT then Stanford and Harvard

2005: Daron Acemoglu, PhD LSE, Faculty at MIT

2003: Steven Levitt, PhD MIT, Fellow at Harvard then faculty at Chicago

2001: Matthew Rabin, PhD MIT, Faculty at Berkeley

1999: Andrei Shleifer, PhD MIT, Faculty at Princeton, Chicago and Harvard



Do you see a pattern? Well I do, and others have, too. I am not saying these awardees are not bright and promising economists, but is there really no other qualifying economists that could have received it? Of course, John List comes to mind, who has no connection with MIT (or Harvard). But it actually worse than that. The award is given by a small committee, designated by the AEA. The AEA leadership is stacked with people with MIT and Harvard connections, so they also nominate their friends to the various committees, and you see the result.



It is even worse. In 2010, Ester Duflo was considered to be in the pool of strong candidates for the award. Guess who was on the awarding committee? Abhijit Banerjee, her PhD advisor, frequent co-author and colleague at MIT. In such a situation, an ethical person would decline the invitation to serve on the committee. That does not seem to have crossed the mind of Banerjee, who may be used to this cronyism.



There is another award, this time given by the European Economic Association: the Yrjö Jahnsson Award, to an European economist under age 45. It is given every two years, but can have several recipients. This awards has looked much cleaner because the committees and awardees have been distributed all over Europe. Europeans are indeed very sensitive to this. The last one was a shocker, though. Armin Falk won it to the surprise of many. And guess who chaired the awarding committee? His advisor, Ernst Fehr. Again, ethics would have indicated that if Falk had a chance of winning it, Fehr should have recused himself not just from chairing the committee, but from participating in it. In retrospect, this is not Fehr's first wrongdoing: two years earlier he was also on the committee when Fabrizio Zilibotti co-won the award. Zilibotti is a colleague of Fehr in Zurich.



I think we should do away with these two awards. It simply does not work.

Are PhD dissertations lagging the research frontier?

A doctoral or PhD dissertation is supposed to be work that pushes the research frontier further. Obviously, not all dissertations are created equal and it is to be expected that some will push more that others. But they are all supposed to push. Well, do they? This is something that is quite difficult to measure as one needs to know where the research frontier lies and what the contribution of a dissertation is. For each dissertation, only few people can do this, and it is thus impossible to have an aggregate picture, unless you use a clever trick.

Sheng Guo and Jungmin Lee take publications in top Economics journals as the research frontier and look in which JEL categories they fall. They compare this to the JEL codes for US Economics dissertations and find a strong correlation controlling for the number of jobs available in the field. If you lag the publications by two years, the regression is just as good, which hints that dissertations react to the research frontier rather than the opposite (which is unfortunately undocumented), especially when you consider that with the long publication delays in Economics, the journals are in fact a few years behind the research frontier.

My interpretation here is not quite that of the authors, who really want to understand how students choose their field of study, given that they want to be on the job market with research on a hot topic. But when they start working on it, they do not know yet what will be hot. I am not sure this is quite such a conundrum, as seminars and conferences already give quite a good picture, and working papers as well. But it still looks like dissertations follow the trends instead of creating them.

On the ethics of research cloning

Even though the Journal of Economic Perspectives recently went open access, a move the American Economic Association should be applauded for, I am still receiving physical copies. It is a nice journal to read while lounging in the garden or on a plane ride. The last issue has as usual a good set of interesting articles, including one I had reported on earlier when it was still a working paper. But while checking what I had said about it, I noticed something rather odd: the paper I discussed was ultimately published in the Journal of Economic Behavior and Organization. I had to investigate.

The two papers are by Bruno Frey, David Savage and Benno Torgler. They both report on the sinking of the Titanic and discuss the characteristics of the passengers who survived versus those who perished. Both papers come to the same conclusions. The texts are different, though, and the published regressions are slightly different, with no explanation why, because there is no reference to the other paper. One has therefore to read in much detail to understand what the contribution of each paper is, if there is any.

All this is very fishy. It really looks like the authors are playing games here, trying to get multiple publications out of the same work. They do not mention the other work to fool editors and referees into thinking these are original contributions, as required for any submission to those journals. They tweak the results and rewrite the text so that they cannot be accused of blatant self-plagiarism. This is unethical behavior, but it is not unheard of in the profession.

But like a late-night infomercial, there is a bonus. Looking at the author's CVs, I notice that they have a third publication with the same topic and results, in the Proceedings of the National Academy of Sciences. Bruno Frey has also published two short pieces in German in magazines prior to the academic publications: 1, 2, both pdf.

Now, who are the authors? David Savage is a PhD student at Queensland University of Technology. He must have been following orders of the more senior authors, either without realizing their unethical behavior or watching in horror and not being able to do something about it. Let us give him the benefit of the doubt. His adviser is Benno Torgler, who has already an impressive track record for someone whose first refereed publication was in 2002. His RePEc profile lists 105 working papers and 52 journal articles. Looking at the published works, it seems to like to revisit previous papers by adding new twists to them. Nothing wrong with that, but it may explain why there is no major hit in the publications. There is simply too much slicing and no single slice is a major contribution worth a good publication. But early in his career, he published a series of articles on tax morale using the World Values Survey. Using the same data and the same methodology, he managed to publish several articles whose distinguishing feature is only that they look at a different set of countries: Asia, transition countries, Canada, Latin America, and possibly more. While I must confess that I have not read the papers in detail, there is simply too much material, and Benno Torgler may be innocent, I still find these patterns very disturbing.

It took me some time to figure out where Benno Torgler earned his doctorate. It is at the University of Basel, under the supervision of René Frey (Basel) and Bruno Frey (Zurich), who are brothers, after undergraduate studies at the University of Zurich. Which bring us to Bruno Frey. He is a researcher of international recognition, mostly for his work on welfare economics, happiness research, and critiques of fundamental assumptions in economic models. He credits himself with over 600 published articles and books, an astounding number in Economics. Of course, if this number comes about by slicing papers or republishing known results as described above, this number is less surprising. Looking at his list of major articles, one can surely suspect something is not quite right. I do not have the time (or the will) to go all of this, but there is indeed a lot of rehashing the same themes, which is OK when one uses new data sets or new approaches. But seeing those quantities, that seem unlikely.

Another aspect that I find disturbing in Bruno Frey's record is that his recent work has been railing against the tendency of academics (and especially their administrators and grant makers) to look for quantifiable evidence of their productivity, what he calls "evaluitis." He writes against the pressure to publish and the prominence of rankings of research output. I have reported about some of this writing myself (1, 2, 3). But again he seems to be repeating himself a lot, even in published articles, essentially criticizing a game that he seems to be excelling at. Either he is sarcastic or hypocritical, I cannot decide.

I realize the accusations I am making here can have severe consequences. But I am only accusing, not condemning. I leave the reader the opportunity to make her own opinion, as I have linked to plenty of evidence. I hope to be proven wrong, that these three individuals are indeed extremely innovative and productive. But from what I have seen so far, my prejudice is strongly negative in this regard.

Update (Sunday): I have been alerted that there is a fourth publication about the same Titanic study, in Rationality and Study.

Further update: A follow-up post.

Economists did see the bubble coming

Economists have been lambasted for not alerting the public that a bubble was in the making in US real state, except for a few oddballs. Of course everyone is wiser in hindsight, but what did economists actually say? It never hurts to look at the facts.

Martha Starr analyzes statements in 24 California newspapers from 2002 to 2007. From 1998 to 2005, the state's house prices increased by more than 10% each year. This prompted the newspaper to run 379 stories with 688 statements by economists on house prices. Academics were clearly warning that house prices were not sustainable. Economists employed in the real-industry, however, were resolutely optimistic. What emerges is a mixed message that gave no guidance to the public, which was even reassured by positive messages from the Federal Reserve.

It is entirely possible opinions could have diverged based on the same evidence. But it seems more likely the professionals were not acting in good faith. They had everything to lose from predicting an end of house price growth. The media should have learned not to trust such biased speakers, yet they continue to be interviewed. Now as to why Greenspan and then Bernanke were so optimistic is beyond me. There speeches are definitively strategic and while they may have realized there was a problem, they may have tried to prevent a bubble from bursting too brutally. Then all the credit to them for trying. But one cannot postpone indefinitely a bubble from bursting, and they knew that.

Are hot teachers better teachers?

It is well known that beautiful and tall people have better lives and are better paid. This is especially thought to be true in activities where skills are relatively unimportant. What about economics professors?

Anindya Sen, Marcel Voia and Frances Woolley use the hotness indicators from student evaluations at ratemyprofessor.com in Ontario and find hot economics university professors are paid a whooping 10% more than their less attractive counterparts. Not only is this a large number, it also runs counter to previous results that such effects are limited to unskilled professions. This effect is especially strong for men and not present for women, Indeed women who negotiate hard are not deemed attractive.

In addition, hotter teachers also get better student evaluations, even after controlling for all what the authors could put their hands on. For other indicators of professor productivity, it turns out that hotness affects positively women for citations, although this could be due to a few highly cited women (citation counts are always very skewed). But neither men nor women publish significantly more when hot, but they tend to attract more co-authors. I really need to be careful with my appearance.

Ethics in Economics

Ethical behavior in the Economics profession has so far never been codified or even much thought about. Indeed, economists have the reputation to be easy to buy, just ask any lawyer needing a "expert" to testify. But the recent movie Inside Job has brought to the general public the issue of conflict of interest in the profession, and the American Economic Association seems to have finally picked up the ball. As it is set to draw a code of ethical standards for the profession (New York Times article), one has to evaluate the large task ahead given the many ways in which economists have breached basic scientific conduct guidelines, with rather small consequences. Let me quickly go through a serious of examples to show how little unethical behavior has been sanctioned so far.

The prime example is, of course, the case of Andrei Shleifer, Harvard professor who was a major consultant for the Russian government during the massive privatization of its state-owned industry. Shleifer amassed substantial wealth during this process in ways many think where not legitimate, and in particular the US Department of Justice thought so. In the end, Harvard paid much of the fines, Shleifer is still a professor there and probably one of the richest people in the profession (more).

Or Florencio Lopez-de-Silanes, coincidentally student and frequent co-author of Shleifer, who as director of the Yale Center for Corporate Governance ironically double-billed US$150'000 worth of travel expenses. He was fired despite tenure, but landed on his feet, still active in the profession as a professor in France. (more).

A bit too frequently, the issue is that if someone gets caught with the hand in the cookie jar, he just resigns and lands a job elsewhere where no one suspects anything. The next examples are not documented online or in print, so I will not give names.

Professor A has help full-time appointments at two universities, with none of the two knowing about it. Colleagues grew suspicious when Prof. A would have strange schedules, rearrange classes in odd ways and never be available. The scheme was finally discovered after a few years, and Prof A was summarily fired. However, a few years later he was again holding two full-time positions at different universities...

PhD student B gets caught in massive plagiarism, we are talking here about copy-and-pasting a dozen full articles. Administrators get alerted and promise to dismiss the student. Student B still graduated with a PhD and is now a "respected" free-lance analyst.

PhD student C gets caught copy-and-pasting the entirety of a term paper from a published article. During the investigation it turns out Student C is a repeat offender, the teacher in the previous case failed to follow the proper procedure, Summarily dismissed, the student enrolls in another PhD program and is now close to graduate and already has a research job.

Professor D publishes as sole author the term paper of a student. Gets caught and dismissed. Moves to another country and now holds a chair.

Professor E plagiarizes and gets caught. Is asked to leave from a well-respected US department, applies for a well-endowed chair in another country. The university is impressed to see a candidate of such prestige and hires Professor E, oblivious to the baggage.

Is the AEA going to be able to take care of all these cases? Most likely not, as it is going to focus on conflict of interest. For the other cases, it is going to argue that every university has internal procedures to enforce ethics and in particular plagiarism. But as I showed with the cases above, this is of little use when offenders can simply walk away and act like a virgin elsewhere.

RePEc has a new initiative that would take care at least of the plagiarism cases by exposing them. As the blog post explains, the goal is the shame plagiarizing authors (after proper procedures have been followed) as was as shaming administrations in imposing proper sanctions. I applaud this initiative and I hope it will set an example and reduce the staggering among of plagiarism going on, and perhaps instill more ethical behavior into economists.