For several month, Bruno Frey (University of Zurich) has had to defend himself against accusations of self-plagiarism. Now, several other dodgy cases have been revealed by a website named “FreyPlag Wiki”. I took a close look at the cases and think most of them are justified.
It was a caustic rant against the scientific publication process written by Bruno Frey, an economist with the . According to Frey, the editors and referees of academic journals force researchers to act like prostitutes.
No editor would dare to print this vitriolic criticism, Frey wrote. “This paper will never be published in a (refereed) economics journal”, was the first sentence of the introduction.
In reality, however, Frey managed to get his rant published twice.
In 2003, the article appeared in “Public Choice” entitled “Publishing as Prostitution”. Two years later, the same piece was printed in the “European Journal of Law and Economics” (EJLE). The title (“Problems with Publishing”) and the introduction was slightly different but the biggest part of the text was word-for-word identical.
[googleapps domain="docs" dir="spreadsheet/pub" query="hl=de&hl=de&key=0AuEtgCUuVBDUdFlVc3ZIR2dsRFd1d29iWndjNVdVSFE&single=true&gid=1&output=html&widget=true" width="500" height="300" /]
(A detailed comparision of the complete content of both papers is available here.)
However, there was no reference to the “Public Choice” article in the EJLE.
Publishing a similar academic article two times without cross-references is an academic no go. This conduct is labeled self plagiarism and violates ethical standards that are generally accepted within the scientific community.
Bruno Frey is one of the most internationally renowned German speaking economists. He tops the Handelsblatt ranking of academic economists with regard to the lifetime achievement.
However, for several months Frey’s scientific conducts has been heavily criticised. Several scholars accused him of self-plagiarism. (Here’s a summary.)
The debate was triggered by a series of articles about the sinking of the Titanic published by Frey as well as his co-authors Benno Torgler and David Savage (both: Queensland University of Technology). In 2010 and 2011, Frey, Torgler and Savage published a series of very similar articles in four different academic journals without cross referencing their work. This was a clear violation of the submission guidelines of the affected journals.
After this case was picked up in economic blogs by Andrew Gelman and an anonymous blogger calling himself “Economic Logician”, several affected journals severely criticised Frey. The “Journal of Economic Behavior and Organisation” told Frey that it won’t publish any of his articles anymore.
Among other things, the JEP editor David Autor (MIT) wrote to Frey:
“We view your publication of this substantive material in multiple journals simultaneously as a violation of the spirit of the editorial agreement with American Economic Association that you signed in the winter 2010. (….)
[We] find your conduct in this matter ethically dubious and disrespectful to the American Economic Association, the Journal of Economic Perspectives and the and the JEP ’s readers.”
New evidence collected by internet users on a website called “FreyPlag Wiki” now suggests that the Titanic case by far wasn’t an isolated incident. Apparently, Bruno Frey has repeatedly published very similar articles in different academic journals without citing his other works. Freyplag Wiki lists five suspicious cases. (There is also a link to a backup version of an old Wikipedia entry listing 16 alleged cases. However, I did not have the time to look at them all.)
Over the last days, I took a close look at those papers mentioned on Freyplag Wiki and compared their content meticulously.
From my point of view, three examples lists as “suspicious articles” indeed are very similar to the Titanic case.
One major case (two papers published by Frey and Stephan Meier in 2004 in the American Economic Review and JEBO) probably does not hold the water with regard to self-plagiarism.
However, I had a look at another clear example (papers on World War II) that has not been added to the list of suspicious publications on the original FreyPlag Wiki. (I wasn’t aware that these papers were already mentioned in an old Wikipedia entry that is backed up on FreyPlag, though)
Some cases affect Top 5 journals and co-authors that are currently affiliated to leading US faculties. (I discuss every case in detail below.)
For two reasons, the new evidence is potentially explosive.
Firstly, the University of Zurich has started an investigation of Frey’s conduct after my reports here and in Handelsblatt. External experts are having a look at the Titanic case. The names of those experts are secret and I have no clue who was asked. However, I was assured that those researchers were truly independent and not linked to Frey whatsoever.
Secondly, the new examples contradict Bruno Frey’s previous arguments. In an interview with the Zurich daily “20 Minutes” in July, Frey argued that the omission of the cross-references in the Titanic-papers were accidental misfortune. Asked why he did not cite himself, he answered:
“Because I just overlooked it. My two coauthors and I wanted to reach the broadest possible audience. Hence, we published our research on the Titanic in four different journals. While doing this, I lost track. (…) We went through a long production process, and in life, you sometimes just make mistakes.”
Additionally, he and Benno Torger replied to David Autor:
“we well understand your very serious complaint and we both agree that you are right. It was a grave mistake on our part for which we deeply apologize. It should never have happened. This is deplorable.”
However, the new evidence which shows that Frey apparently frequently published the same research findings several times makes his way of reasoning very hard to believe. Additionally, his profuse apologies to the journals affected in the Titanic case appear very shallow, from my point of view.
The most blatant cases of self-plagiarism both affect the “European Journal of Law and Economics” (EJLE). Twice, Frey published papers containing a number of paragraphs that were word-by-word identical to previously published work but lacks any reference to them.
Ironically, Bruno Frey himself is a member of the editorial board of this journal, and the editor-in-chief is one of this former students, the German economics professor Jürgen Backhaus (University of Erfurt) (Here’s his German Wikipedia entry.)
What follows is a detailed discussion of the individual cases.
“Publishing as ” (Public Choice, 2003) / “Problems with Publishing” (EJLE, 2005)
This is a single authored paper by Bruno Frey. The abstracts in both journals are completely identical.
The first two sentences of the Public Choice paper are slightly different, but starting with the second paragraphs both articles are almost completely identical:
“The author knows that, normally, he would be lucky if, after something like a year or so, he gets an invitation to resubmit the paper according to the demands exactly spelled out by the two to three referees and the editor(s).”
“The author knows that if he submitted it, he would be lucky if, after something like a year or so, he got an invitation to resubmit the paper according to the demands exactly spelled out by the two to three referees and the editor(s).”
All chapter headlines are identical, as are most of the subsequent paragraphs. The only additional piece of information the EJLE version offers is a graphical description of an argument that Frey only makes verbally in the Public Choice edition. On the other hand, the EJLE version lacks the lengthy conclusion given in Public Choice.
There is no reference to the Public Choice paper in the EJLE version.
This is a joint paper by Bruno Frey and Stephan Meier, who joined Columbia Business School in 2008.
[googleapps domain="docs" dir="spreadsheet/pub" query="hl=de&hl=de&key=0AuEtgCUuVBDUdFlVc3ZIR2dsRFd1d29iWndjNVdVSFE&single=true&gid=5&output=html&widget=true" width="500" height="300" /]
There are slight semantic differences between the abstracts of both articles. For example, the first sentence of the abstract Economic Inquiry says:
“Many people believe that economists in general are more selfish than other people and that this greater selfishness is due to economics education. “
The first sentence of the EJLE version states:
“Most professional economists believe that economists in general are more selfish than other people and that this increased selfishness is due to economics education.”
Apart from one additional sentence in the EJLE, the rest of the abstracts are completely identical. One noteworthy difference between both articles is that the EI version is written in American English (“behavior”), while the EJLE versions was transformed into British English (“behaviour”). There are a number of sections that are almost identical in the main text.
However, again, there is no reference to the EI article in the EJLE. Two related articles using similar data and addressing a different but related research question, that were published by Frey and Meier in 2004 in the “American Economic Review” and the “Journal of Economic Behavior” aren’t cited as well.
The submission guidelines of the EJLE clearly prohibit the submission of articles that were previously published elsewhere. They state:
“Submission of a manuscript implies: that the work described has not been published before; that it is not under consideration for publication anywhere else; that its publication has been approved by all co-authors (…) “
The publisher, Germany’s Springer Verlag, told me that they will investigate the incident
“according to our normal processes, which follow COPE guidelines”.
“COPE” stands for the “Committee of Publication Ethics” and is an interdisciplinary forum “for editors and publishers of peer-reviewed journals to discuss all aspects of publication ethics”.
Their procedures regarding “redundant publications” are rather straightforward: redundant articles should be retracted. The COPE “Retraction Guidelines” state (among other things):
“If redundant publication has occurred (i.e. authors have published the same data or article in more than one journal without appropriate justification, permission or crossreferencing) the journal that first published the article may issue a notice of redundant publication but should not retract the article unless the findings are unreliable.
Any journals that subsequently publish a redundant article should retract it and state the reason for the retraction.”
However, when I talked to Jürgen Backhaus on Sunday, the editor-in-chief was strongly backing Bruno Frey. Backhaus argued that Frey is known for his new and unconventional ideas. According to Backhaus, it was necessary to repeat them again and again to get them through to a reluctant audience. Backhaus told me:
“It is well known in the profession that Bruno Frey works like this.”
He said that it was an honour to be able to publish an article by Frey:
“He is an internationally renown academic who is a candidate for the Nobel prize.”
According to Backhaus, publishing an article by Frey enhances the attention for other articles in the journal. I asked him how he would explain to a PhD student that the official submission guidelines of the journals apparently are not applicable to Frey. His answer was:
“Bruno Frey is a trademark. The PhD student still has to build one.”
I was really stunned by these remarks. I emailed those quotes (in German) to Backhaus prior to publication. He confirmed that I quote him correctly. (Translations from German into English were done by me, however.)
(There is a particular irony in the statement by Backhaus, by the way. Frey accused me in his “20 Minutes” interview of exaggerating his academic achievements because I described him as an economic bigshot. Frey: “This exaggeration was necessary for making the article relevant.” Now, Backhaus defends Frey with the argument that he is such a great economist…)
From my personal point of view, this stance is completely unacceptable.
If the EJLE wants to retain any credibility and if Springer takes the COPE guidelines seriously, they won’t have any choice but to officially retract both articles. Additionally, I don’t see how Frey can stay on the editorial board of a journal which submission guidelines he repeatedly has clearly violated.
These are joint papers by Bruno Frey and Marcel Kucher, who apparently has left academia. This example has not yet been entered to FreyPlag Wiki. I found it accidentally when I had a look at Frey’s publication record.
Both papers discuss exactly the same research question, use the same data and methodology and come to the same conclusion. However, there are no cross-references. The Economics Letters version seems to be a briefer and more concise version of the article published in the “Journal of Economic History”.
Plenty of paragraphs are very similar, as you can see in this table. Here’s one example:
Journal of Economic History:
“As can be seen in Figure 1, there is a strong downturn in the index of all government bonds traded in Switzerland from late 1933 up to the outbreak of World War II. During the war, the index remained relatively stable at around 40 percent of par. One interesting feature is the peak in mid-1944, just about when Allied forces invaded Normandy.”
“Fig. 1 shows a strong downturn in the market index before the outbreak of WWII. During the war, the index remained stable at around 40 percent of par. One interesting feature is the peak in 1944, just about the time when the allied forces invaded Normandy.”
“Economics Letters” is published by Elsevier, which asserts in its ethical guidelines:
“An author should not in general publish manuscripts describing essentially the same research in more than one journal or primary publication. Submitting the same manuscript to more than one journal concurrently constitutes unethical publishing behavior and is unacceptable. “
“Articles on economic history and related aspects of history or economics will be considered for publication by the Editors on the understanding that the articles have not previously been published and are not under consideration elsewhere.”
Update: According to an old Wikipedia entry on alleged cases of self-plagiarism by Frey, similar papers were also published 1999 in Empirica (“Asset Prices and History: The Case of Austria”) and in Economics in 2001 (“Wars and Markets: How Bond Values Reflect the Second World War“). The entry states that the cross references are also missing. Please be aware that I did not double-check this claim. I wasn’t aware of this old Wikipeda entry when I compared the papers and wrote the first version of this post.
“The Old Lady Visits Your Backyard: A Tale of Morals and Markets” (Journal of Political Economy, 1996) / “The Cost of Price Incentives: An Empirical Analysis of Motivation Crowding- Out” (American Ecnomic Review, 1997)
From an American perspective, this is potentially the most interesting case, because it affects two Top 5 journals as well as a co-author who is affiliated with Harvard Business School – Felix Oberholzer-Gee. (Coincidentally, Oberholzer-Gee was involved in another quarrel about a very controversial paper made public by my colleague Norbert Häring a few years ago that also involved Steven Levitt.)
However, this case is more complicated than the previous three examples. Without any question, both articles address a similar research question, apparently use the same data and do not cite each other.
Both papers deal with a problem called “Not In My Backyard” (NIMBY): Facilities that are deemed socially necessary or useful but meet strong local resistance in the regions where they are supposed to be build. Examples are high security prisons, power plants or airports. Frey and his co-authors investigate empirically if this resistance can be overcome by monetary compensation.
[googleapps domain="docs" dir="spreadsheet/pub" query="hl=de&hl=de&key=0AuEtgCUuVBDUdFlVc3ZIR2dsRFd1d29iWndjNVdVSFE&single=true&gid=3&output=html&widget=true" width="500" height="300" /]
At the core of both papers is data collected in 305 in-person interviews in rural Switerland in an area taken into consideration for a nuclear waste facility.
However, several things are pretty strange. First of all, the JPE paper (published in December 1996 ) lists three authos (Bruno Frey, Felix Oberholzer-Gee and Reiner Eichenberger) while the AER paper only lists Frey and Oberholzer-Gee.
The results of the interviews are exactly similar in both papers. For example, both papers state that 50.8 percent of the of the citizens living in the host community indicated that they would support this siting decision.
(Interestingly, this result is framed differently in both articles. In the JPE it is described as “a bare majority”, while in the AER it is describes more positively as “more than half of the respondents”.)
Additionally, several survey questions are identical in both papers.
For example, the JPE states:
“To test our hypotheses, we repeated the exact same question, asking our respondents whether they were willing to accept the construction of a nuclear waste repository if the Swiss parliament decided to compensate all residents of the host community. The amount offered for the lifetime of the facility was varied from $2,175 per individual and year (N = 117) to $4,350 (N = 102) and $6,525 (N = 86).”
In the AER it reads:
“To test the effect of external compensation, we repeated the exact same question asking our respondents whether they were willing to accept the construction of a nuclear waste repository when the Swiss parliament had decided to compensate all residents of the host community (Question 2, Appendix). The amount offered varied from $2,175 per individual and year (N = 117) to $4,350 (N = 102) and $6,525 (N= 86).”
Strangely enough, however, the survey is described differently in both articles. In the JPE the authors write that the interviews were conducted in June 1993 a in Wolfenschiessen, one of four communities that were under consideration as possible sites. In the AER they claim that the interviews were done in two adjacent communities located in central Switzerland in the Spring of 1993.
Another peculiarity is that the the results of boths papers appear to be contradictory, from my point of view. In the JPE paper (published 10 months earlier) the authors compare the results of their survey (monetary compensation does not resolve the NIMBY problem) with an actual poll in the area that was conducted in 1994.
The region was offered monetary compensation and the project was given a go-ahead by the citizens. The authors develop a theory that tries to explain this observation (that I, frankly, do not fully understand their argument). In the AER paper, however, there is neither a reference to the poll conducted in 1994 nor, as mentioned above, to the JPE paper. Hence, based solely on the survey, this paper (which was published later) concludes:
“ where public spirit prevails, using price incentives to muster support for the construction of a socially desirable, but locally unwanted, facility comes at a higher price than suggested by standard economic theory because these incentives tend to crowd out civic duty”.
I wrote an email to the current AER editor-in-chief, Pinelopi Koujianou Goldberg with Yale University. This is what she replied:
“As you pointed out, the AER papers were published much before I took over. I am not comfortable commenting on cases I did not handle. This is a very serious matter, and there are people’s professional reputations at stake; I wouldn’t want to pass judgment without carefully reading the papers involved first and without having more context regarding the submissions and decisions. (…)
Regarding your question about our submission guidelines, of course publishing an identical paper in a different journal would violate our guidelines.
But the reason I want to be cautious is that there are many cases that may not be black and white. For example, there may by overlap in the phrasing of particular paragraphs and methods used, but it is possible that two papers address different questions or that their analyses differ in other substantive ways.
Or, sometimes, people submit the first version of a model or the first version of an empirical paper that uses incomplete data to a field journal, and the referees write “This paper would be great if you could extend the analysis in this direction, or if you could provide this additional piece of evidence based on additional data…. With this additional information, the paper would be a good match for a top-five journal, but without it, it is ok for a field journal…”
The authors then go ahead, publish this first version in the field journal, and later on they do the additional analysis suggested by the referees and submit to top journals. In such cases, there may be great overlap between the two submission, but the authors would not be in violation of our submission guidelines; it is up to the editor to determine whether the new submission makes a significant enough contribution relative to the authors’ earlier work for it to be published.
Still, in these cases I would expect authors to cite their own earlier work – if anything, people tend to self-promote and cite themselves excessively – it is unusual not to have references to one’s own earlier papers that are related.
Update: I just got a response by Orley Ashenfelter, the current president of the American Economic Association who was the editor-in-chief of the AER when the paper by Frey and Oberholzer-Gee was published. This is what he wrote me:
“When I edited the AER the editorial decision making was decentralized (as I’m sure is the case today). That is, papers were assigned by me as the Editor to a Co-editor for handling and a decision.
I do not recall who handled the paper you note, and I have no access to that information now. (It most likely was not me, based on what the subject matter appears to be. ) The archives of the Review become public after some point in time, but I believe this paper does not yet fall in that window. No doubt the current editor could reveal the co-editor’s name should that be within current editorial guidelines.
On the more general issue, this episode raises a very difficult set of questions about the publication process. Needless to say, as an editorial matter I would have found it inappropriate to publish material that was not original.(Highlighted by me)
Indeed, the Review had then (and no doubt has now) a requirement that copyright for the material in the journal was held by the AEA, so dual publication could, in principle, lead to legal action.
As to the specific case you mention, I have not looked closely enough at the papers in question to form a judgment about the extent of overlap, nor do I have the relevant information to respond to your other questions.”
I also tried to get in touch with all three authors of the JPE paper.
The only one who replied was Reiner Eichenberger, a Professor of Economics at the University of Fribourg in Switzerland. (I also wrote an email to Stephan Meier, who did not reply as well.)
Eichenberger did neither explain why his name was missing on the AER paper nor pointed out if the survey data used in both articles actually was identical and why the survey was described differently in both papers.
However, he stated that both papers were “not identical at all” but in fact were were addressing very different aspects. He argued:
“Another evidence that the articles are very different is that both of them are cited quite often. However, in the 15 years since their publication nobody ever made the slightest suggestion that they were too similar.”
Additionally, he tried to defend the lack of cross references. According to Eichenberger, I have a wrong understanding of the academic publication process. He argues that both articles were part of a broader research agenda and were written at a rather similar point of time.
“When we submitted the papers at the different journals, some of the other articles were not written at all, some were written but not submitted and some were submitted but not accepted. It is well known that the publication process usually takes two to four years. Hence we did not have any opportunity to cite the other papers. At the point of submission, it was just not known where they were going to be published.”
In my personal point of view, for a number of reasons this is a weak argument.
First of all, prior to the final publication the authors usually get a proof of their article and are able to add final changes. Hence, I find it rather unconvincing that the JPE paper was not cited in the AER paper that came out ten months later.
Additionally, if you want you can also cite working papers in Top 5 publications, as the next expample shows.
These are again joint papers by Frey and Stephan Meier (Columbia Business School), and they are also listed on FreyPlag Wiki in the section “suspicious publication”.
However, I don’t think this example constitutes another example of self-plagiarism. Hence, these papers should be moved to the section “unfounded suspicions”.
As it is already noted in the Wiki, both paper refer to each other. They don’t cite the final publication (probably due to the restrictions pointed out to my by Reiner Eichenberg earlier) but each list a working paper version of the other article.
Additionally, they are not identical. On the one hand side, both papers do address a similar research question and employ similar data. On the other hand, the AER article adds another very important observation gained in a field experiment. As somebody wrote on the FreyPlag:
“This is of course an interesting extension of the JEBO paper.”
Update: In an earlier version of this post, I erroneously wrote that Reiner Eichenberger is at the University of Lausanne. In fact, however, he is at the University of Fribourg in Switzerland. Apologies!
Update II: In an earlier version I also wrote that I stumbled upon the WW II papers which were not mentioned on FreyPlag Wiki. In fact, I found them acidentally when I was scrolling through Frey’s literature list. However, I wasn’t aware that this example was already known as well – the WW II incident was mentioned in the old Wikipedia entry that is backed up on FreyPlag Wiki. (I did not have a close look at that list.) However, I dont’t want to give the impression that I adron myself with borrowed plumes. Apologies to the guys who compiled the original list on Wikipedia!