A summary of the Bruno Frey affair

Titanic's sinking on a popular but factually i...

The sinking of the Titanic was the topic of the controversial Frey et al papers (Image via Wikipedia)

One of the most senior economists of the German speaking world faces serious questions about his scientific modus operandi. Bruno Frey and his research team are accused of self-plagiarism. Additionally, they at least showed an amazing degree of sloppiness with regard to literature research. Five older publications from different authors on exactly the same research question are missing from the references.

This blog (among others, especially Andrew Gelman’s as well as“Economic Logic” and the EJMR forum) has played a role in making the whole thing public. On Wednesday, 6 July the University of Zurich has started a formal investigation against Frey, based on the “suspicion of unethical scientific conduct”.

Since the information about the affair and the views of the people involved is scattered among  a number of posts, I thought it might be a good idea to publish a summary.

What it is all about?

Bruno Frey (University of Zurich), Benno Torgler (Queensland University of Technology) and Torgler’s Ph.D. student David Savage simultaneously published a series of papers dealing with the sinking of the Titanic, but neither cross-reference their own work nor  cite a number of older papers by other researcher addressing exactly the same topic.

The articles by Frey, Torgler and Savage appeared in the “Journal of Economic Behaviour and Organization”, “PNAS”, “Rationality and Society” and the “Journal of Economic Perspectives” in 2010 and 2011. They used individual-level passenger data showing the age, gender, ticket class and nationality of 2207 people sailing on the Titanic and employed an econometrical analysis on the determinants of survival.

For several months the authors have been criticised because they simultaneously published nearly identical papers in four different journals without mentioning their other work on the same topic to the editors. This is a clear violation of the submission guidelines of the journals and economists consider this being unethical. The “Journal of Economic Perspectives” rebuked the authors publicly and will publish the conversation with Frey in its forthcoming issue. The editor of the JEBO apparently black-listed the authors with regard to further publications. (see comment by Barkley Rosser).

However, it has become increasingly clear that Frey, Torgler and Savage by no means have been the first economists who econometrically address the survival probabilities of the people sailing in the Titanic. At least five older publications with a very similar research outline and similar results exist. They have all been published before the first working papers of Frey, Torgler and Savage came out and most of them are rather easy to find using the query “Titanic” or “Titanic survivor” on databases like Repec, EconBiz and Google Scholar. However, none of these publications is cited in any of Frey’s et al papers.

The oldest paper on the topic I could get hold of has been published in 1986 by Wayne Hall (Social class and survival on the S.S. Titanic”) in “Social Science & Medicine”. Hall also related passengers’ chances of surviving the sinking of the Titatic to their sex and their social status: He came to the conclusion that “females were more likely to survive than males, and the chances of survival declined with social class as measured by the class in which the passenger travelled”. If you type in the word “Titanic” into Repec, this paper is the fifth result.

Oddly enough, the author of the 1986 paper is a Professor at the University of Queensland, Brisbane while Frey’s co-authors Benno Torgler and David Savage are both affiliated with the Queensland University of Technology in Brisbane. That’s quite a coincidence, isn’t it?

David Autor (MIT) and currently the editor of the “Journal of Economic Perspectives” (JEP), was furious. The JEP, where one version of the papers by Frey, Torgler and Savage was published, has been dealing with the issue of “self-plagiarism” by Frey et al. for several months. However, the JEP-editors were not aware of the paper by Hall. That “comes as unhappy news to my coeditors and me”, Autor wrote me:

“My reading is that there is considerable overlap between Frey et al. and this 1986 paper.”

However, Autor stresses that he does not accuse them of plagiarism:

“It would be extremely difficult to establish that Frey et al. knew of the Hall paper and had borrowed from it. Absent such proof, I would not make such an allegation. However, Frey et al. should have known about this article. It is their responsibility as scholars to fully research the literature in their field of study, and to acknowledge prior contributions accurately and fairly.”

Bruno Frey and Benno Torgler assure that they neither knew the journal nor the paper nor the author and argue that the connection to Brisbane is just happenstance. In an email, Torgler wrote me:

“Thanks for referring us to other studies. We were not aware of them. We haven’t seen the Hall paper published in Social Science & Medicine beforehand, otherwise we would have cited it.”

Torgler pointed out that they had done a literature research on various issues related to the topic and refers to the numerous articles cited in their papers. He asserted:

“However, it seems that we have missed the ones you have mentioned which is a substantial shortcoming from our part.“

A few hours later Torgler wrote me that they were “puzzled by the fact that we were not able to find the Hall paper” and therefore took a closer look at Repec. He pointed out that the journal “ Social Science & Medicine” apparently was only added to Repec in April 2010 “and therefore after we had done our literature search as the access statistics start then”. He might in fact have a point with regard to his, as the access statistics for the journal and the Hall paper published by RepEc suggest.

Wayne Hall, the author of the 1986 paper, was taken by surprise by the sudden spark of interested in his work. He wrote me:

It is clear that they [Frey et al.] did not look very hard for previous publications on this topic. My paper pops up on the top of lists on Google Scholar and PubMed when you use “Titanic” and “survival” as search terms. My paper has also been cited in one of Michael Marmot’s books on social class differences in mortality and in a widely used as an introductory sociology text. Their analysis of the reasons for the social class differentials on the Titanic shows little evidence that they have examined previous scholarship on this issue. That would fit with their failure to do any searches that located my paper. As my article shows, the mortality difference between the three passenger classes was discussed at the Mersey Inquiry, in the newspapers of the period, and in the popular literature on the sinking such as Lord’s ,A Night to Remember’.” 

In fact, Hall’s paper is by far not the only similar paper Frey, Torgler and Savage missed. They managed to overlook an entire class of statistics publications, too.

The statistical analysis of the passenger data of the Titanic and the determinants of survival have been a favoured case study for statistics courses. In the last 15 years at least four different publications discussed how the passenger data can be used for teaching statistics. It seems to have started with two oddly entitled papers published in the Journal of Statistics Education in 1995 by Robert Dawson (“The “Unusual Episode” Data Revisited”) and in 1997 by Jeffrey S. Simonoff  (“The “Unusual Episode” and a Second Statistics Course”).

With regard to these papers, the credit for discovery goes to an anonymous writer who wrote a comment on the blog “Economic Logic and/or to people discussing the subject on the forum “Economic Job Market Rumours”.

Robert Dawson, who I asked to compare his 1995 paper with the work by Frey et al wrote me at he contributed neither data nor methodology and considers it a “minor omission” that his paper isn’t mentioned in the bibliography. However, Dawson stresses:

“I am more concerned by the absence of Jeffry Simonoff’s name from the bibliographies. His paper “The ‘Unusual Episode’ and a Second Statistics Course” uses logistic regression on “my” data in much the same way that Frey, Torgler, Savage use probit regression (though he does not explicitly interpret the results from an economist’s standpoint); these techniques are close enough that the omission of his article is at the very least careless and unfortunate.”

A quick search on Google Scholar for “Titanic survivor” reveals something even more striking. The first result refers to the “Handbook of Statistical Analyses using SPSS”, published in 2004 by Sabine Landau and Brian S. Everitt. They wrote an entire chapter on the topic “Logistic Regression: Who Survived the Sinking of the Titanic?”. On 23 pages, the authors use passenger data from the Titanic as a case study for the question how to do regressions with SPSS.

Compared to Frey et. al., Landau and Everitt as well as Hall used different datasets: For 1309 passengers they have information about survival, age, gender, ticket class and the number of family members accompanying. Frey, Torgler and Savage also look at crew members and in total have information for 2207 people. The basic research question, however, is exactly the same: “Which of the explanatory variables are predictive of the response, survived or died?”

The Titanic example seems to be so popular in statistics, that a dataset of the passengers and the survivors has been available as a package that’s part of the default distribution of the popular statistics package R, as “Tobias” pointed out in a comment on my blog and somebody also noted on Twitter.

In 2006, another publication using the passenger data as a case study for teaching statistics came out. This time, it appeared in the Journal of Economic Education, was written by Robert Dixon and William Griffiths (both, interestingly, are affiliated with universities in Australia, as well, as Hall, Torgler and Savage are) and was entitled “Survival on the Titanic: Illustrating Wald and Lagrange Multiplier Tests for Proportions and Logits”.

The abstract (here’s a free working paper version) says:

“Students are interested in lecture examples and class exercises involving data connected to the maiden voyage and the sinking of the liner Titanic. Information on the passengers and their fate can be used to explore relationships between various tests for differences in survival rates between different groups of passengers. Among the concepts the authors examined are…”

While it is true that “Social Science & Medicine“ is not part of EconLit (an influential database set up by the American Economic Assoication) and was probably added to RepEc and EconBiz (a database run by the German National Library of Economics) in 2010, the Journal of Economic Education is listed in EconLit. Currently, a EconLit search for “Titanic” lists the paper by Dixon and Griffiths as the 15th search result – seven of the previous 14 results refer to different versions of the paper by Frey, Torgler and Savage.

I’m really stunned. How is it possible that a parcel-learned economic journo like me finds more academic literature about a topic than a research team of three economists, among them two full professors? And how could this stuff pass the hurdle of four different peer-reviewed academic journals? From my point of view, the journals should really rethink their procedures.

David Autor, the editor of the Journal of Economic Perspectives, sees it a differently: “I disagree that this shows a weakness in the peer review process”, he wrote me.

“Referees are not in the business of policing for plagiarism (of self or others). The scientific publishing process relies on the fact that researchers face severe sanctions for fraud or plagiarism. Generally, one such incident can end a noteworthy career, even one decades in the making.”

So let’s wrap the main points up. Which contribution did Frey, Torgler and Savage make, what are their shortcomings and what are the open questions remaining?

  • The basic idea to do a statistical analysis of survival probabilities of passengers on the Titanic is not new at all. It is possible that Frey, Torgler and Savage had the idea independently from the older literature. One has to apply the presumption of innocence here.
  • However, it is striking that Frey, Torgler and Savage did a literature research which completely overlooked the body of existing literature dealing with the same topic. At least, this shows an egregious degree of sloppiness, especially given that Frey is such a senior and experienced figure.
  • Given the eagerness the authors had with regard to publishing their results in four different journals simultaneously, this sloppiness appears even more outrageous.
  • No matter why the older literature on the topic is missing, the lack of cross-references between their own papers and the submission of very similar papers at the same time to different journals is a clear violation of the submission guidelines of the journals that are involved and constitute some kind of “self-plagiarism” that is considered unethical.
  • This unethical behaviour with regard to their own work tarnishes the credibility with regard to the missing papers from other authors.
  • This kind of modus operandi is outrageous given the fact that the third author (David Savage) is a Ph.D student who ought to have been protected better by his advisor Benno Torgler.
  • Frey’s conduct is even more amazing given his frequent statements criticising the publishing process in economics and his own work on fraud in academia (Ironically, Frey was working on a big research project dealing with fraud in academia, which currently is put on hold, as Frey’s coauthor on that study told me yesterday). To me, with the benefit of hindsight, his criticism sounds quite hypocritical.
  • The fact that the author of the initial paper from 1986 works in the same city (albeit at a different university) as Benno Torgler and David Savage is at least a weird coincidence. Even more stunning is the fact that the authors of the 2006 paper are affiliated with Australian Universities as well. Of course, this can also be pure happenstance, but it is really odd.
  • One unique contribution of Frey, Torgler and Savage is the comparison between the sinking of the Titanic and the Lousitania. (Interestingly, this also is the only point where they explicitly claim to be the first ones …)
  • A second unique contribution is to tell an economic story around the results (the stuff about norms, selfishness and so on)
  • Apart from these two contributions, their work basically is a replication of the older papers using slightly different methods and better data. Replication is very important, and the results of Hall et al are confirmed, which is a good thing too. However, I have serious doubts the four journals would have accepted the papers if the editors had been aware of the true character of the papers.

Remark on my own behalf: Some commets were questioning my own intentions and accused me of “exploiting” this issue. I don’t really have dog in this fight. I’ve liked Bruno Frey’s work, respect him personally and I frequently wrote about him and his papers (see here, for example)  favourably.

I still think he has done a lot for Economics, especially with regard to ending the narrow-mindedness of the discipline.

However, his conduct described here  really irritates and disappoints me. I think it’s my duty as a journalist writing about Economics Science to make it public. I try hard to present the facts as accurately as possible and give everyone involved the opportunity to make his view public.

If there are any mistakes, distortions or omissions in my posts, please let me know. I’m happy to correct them.


If you happen to like this blog, why don’t you like me on Facebook as well?


Filed under Bruno Frey, General Economics

23 Responses to A summary of the Bruno Frey affair

  1. Christian M.

    Good post, and thanks for doing your homework as a journalist.

  2. 5 posts on Bruno Frey in a row and still counting. A little bit too much for a storm in a teacup. Don’t you think so?

    • No, I don’t think so, and the editors of the journals affected by the self-plagiarism don’t, either. I don’t consider this as a mere storm in a teacup. Anyway, that’s only as many posts as Frey et al did versions of their paper (including the German language publication)….

  3. Prof Anonymous

    This problem with Frey is pretty general. In the 1990s he was writing papers that relied heavily on mine, and either did not cite mine or did so in a way to mis-represent the relationship between his and mine.

    That is, readers of his papers would think he was a pioneer; he was not.

    One reason this can happen is that the affected authors are reluctant to bring it up. It sounds like whining. And leads them to create new email addresses…

  4. Pingback: Bruno Frey is in trouble « Matthias Greiff

  5. Hans Wurst

    Bruno Frey “verteidigt” sich:
    Welche Lehre ziehen Sie aus Ihrem Fall?
    Ich muss den Publikationsprozess enger begleiten.

    Finden Sie die Aufregung übertrieben?
    Ja. Der Journalist des deutschen Handelsblatts, der gestern breit darüber berichtete, schreibt viel Falsches. Und er bauscht mich zum Star-Ökonomen auf, damit er die nötige Fallhöhe für einen reisserischen Artikel hinkriegt.

    Fühlen Sie sich unfair behandelt?
    Ich frage mich, ob bei Deutschen Neid eine Rolle spielt. Dass ein Schweizer wie ich oft die Beststeller-Listen anführt, mag einigen im Nachbarstaat nicht in den Kram passen.
    Weiter hier (click)

  6. ec prof

    the interview shows how much of a coward frey is. “the journalist writes a lot wrong”. maybe frey would like his name on the 1986 hall paper also (is that the error?). there’s something seriously wrong with this guy.
    btw no one envies the Swiss economists- many academics do not respect them due to frey and his group. We tell our students to stay away from such work. frey, you should man up and admit to your deliberate intent to miss lead others.
    Olaf, you have done a great job and this takes some time and courage. You also try to be fair and give frey some accolades. I appreciate that, however from an academic point of view, he is really not that good. I would not pay too much attention to the rankings.

  7. Pingback: Bruno Frey fights back | Economics Intelligence

  8. Pingback: Die Welt – ökonomisch gesehen » Du sollst nicht abschreiben…

  9. SSam

    Nice work Olaf! You really covered this unpleasant event in detail. Is there a chance that the journal articles will be retracted from any of the journals given that the event and behavior has been covered and analyzed by someone like Hall (1986) and the other papers during the late 1990s?
    I see no point to publishing something that has been done before.

    • Many Thanks, Sam . I don’t think the articles will get retracted. However, the JEP has added a “watermark” to the PDF version of the article, stating that it is “substantially similar” to the other one’s. That’s quite an embarrassment for the authors….

  10. My apologies for spoiling the party. But I get these automatic emails from WordPress and I’m only wondering. This is a complete sideshow. Sort of academic navel gazing. Luckily the dreaded WSJ asked me for my opinion: it is a tempest in the teapot. Where’s the harm beside Frey collecting some journal publications? His argument as such seems quite reasonable to me.

    I wish brilliant economic journalists like Olaf would turn their attention to publications in journals and the media which peddle mainstream economics nonsense because the authors missed the fact that we are not on the gold standard anymore (except for the stupid Eurozone). My suggestion: Read this BIS paper “The bank lending channel revisited” and then follow your journalistic instincts. You will be busy for the next years. Promised!

    • Stephan, many thanks for joining the debate. However, I respectfully disagree. For several reasons, I don’t think this was just a storm in a tea cup, and the feedback I received here and in some private mails from academics confirms this impression. Here’s an email I received from an economics professor affliated with a highly prestigious UK university who asked not to be namend:

      “Dear Olaf

      (…) I’ve also seen examples of the kind of practice you have reported during my own time as an editor. (I edit the [***prestigious economic journal, name removed***], amongst other things.) Sadly, my co-editor colleagues and I come across it all too often. One of us in particular is particular angered by instances of dubious ethical practice.

      Personally, I am concerned by the excess resources used in the review process and how such resource burn is exacerbated by duplicate submissions. I spend 40% of my time on editing work and recognise that others do too.

      In turn, I impose massive burdens on excellent researchers when I ask them to evaluate papers; a proper evaluation can easily be a day or two of work. Each academic paper can sometimes generate a dozen reports, as authors move through different journals as they seek an outlet for their work. Having a single piece of work simultaneously sent to many outlets (the subject of your posts) obviously multiplies this. The wasteful duplication is so demoralising.

      I am sympathetic to the reach-different-audiences aim. But (perhaps sadly) that’s not what economics journals are now about; they also act as accreditation agencies, and that means papers do attract much deeper attention from referees. This accreditation is eased, and with no real loss to authors, by simple open honesty about related work from writers. On the communication front, as an editor, I can still decide whether it is appropriate to publish minor variations based on the desire to reach readers in different disciplines once the right citations are in a paper. I see no reason to hide things.

      Overall, the activities you are highlighting are instances of plain selfishness. It’s such a shame.

      Anyway, this email is simply a private note of thanks (this is not an official message in my capacity as a journal editor) to say “good work” and “thank you” for following this up.

      With my best wishes,

  11. Pingback: Tyska nationalekonomer anklagade för plagiat « Ekonomistas

  12. Pingback: “Ethically dubious and disrespectful” – JEP publicly lambats Bruno Frey | Economics Intelligence

  13. Pingback: Skandale i forskningsverdenen kommer til en foreløbig ende @ Punditokraterne

  14. Pingback: Bruno Frey: More cases of self-plagiarism unveiled | Economics Intelligence

  15. Pingback: Freygate | eco.nomie.nl

  16. Pingback: Bruno Frey and his habitual self-plagiarising by the “cloning” of papers « The k2p blog

  17. Pingback: On the merits of repeating oneself – A conference in defense of Bruno Frey | Economics Intelligence

  18. Pingback: University of Zurich: Forced retirement for Bruno Frey | Economics Intelligence

  19. Pingback: Sexual selection and inequality - Evolving Economics