Are Credit Ratings Massively Overrated?

A striking new paper by two economists with Brandeis University and Oxford fundamentally questions the informational content of credit ratings. How “incomprehensible and irrational” are the assessments of Standard & Poor’s, Moody’s and Fitch?

Try your luck at the big rating gamble. (c) Klaus Meinhardt, 2012

Christian Noyer is usually a calm and sober-minded central banker. Last December, however, the president of the Banque de France was unusually outspoken and indulged himself in a vitriolic rant against credit rating agencies. According to Noyer, their assessments had become nothing less than “incomprehensible and irrational”. Some decisions “do not seem to be justified on the basis of economic fundamentals,” Noyer complained.

A new study by two economists now provides empirical evidence for this drastic view. Jens Hilscher (Brandeis University International Business School) and Mungo WilsonSaïd Business School, University of Oxford) come to the conclusion that credit ratings do not contain much information about actual default probabilities. “The informational value of credit ratings is surprisingly low,” says Hilscher in summarising the key findings of the paper entitled “Credit Ratings and Credit Risk.”

More often than not a rather simple analysis of publicly available data easily beats the assessments of the credit rating agencies and gives a much better picture about the actual default risks, the researchers observe. “The judgements of the credit rating agencies are significantly over-rated ,“ concludes Hilscher.

These results are directly challenging the pivotal role credit agencies currently play on world financial markets. The agencies largely enjoy their dominant position thanks to the decisions of financial regulators. For instance, a lot of investment funds are only allowed to hold assets with a certain minimum rating. The equity buffers that banks have to put aside to protect themselves against potential losses in the current regulatory framework also hinge on the assessment of S&P, Moody’s and Fitch.

Since the financial crisis broke out, these privileges have been under attack. Critical researchers take the new paper by Hilscher and Wilson as a confirmation of their views. “My personal view is that credit ratings have many problems, and should not be used for regulatory policy,“ says Robert Jarrow, Professor of Investment Management at the Johnson Graduate School of Management, Cornell University. Jarrow feels vindicated by the Hilscher/Wilson paper. „I think the paper’s results are correct and should be taken very seriously.“

John Campbell, professor of economics at Harvard, also has a high opinion about the paper: “I do think Jens and Mungo’s work is convincing and methodologically sound.”

At the core of the paper are corporate bond ratings by Standard & Poor’s. Hilscher and Wilson look at all assessments that have been given between 1986 and 2008. Additionally, they constructed an alternative indicator that is meant to gauge the default risk of the bond issuers. The economists only use publicly available information for this “failure score”, mainly balance sheet data like profitability, leverage and cash holdings.

Hilscher and Wilson compare the S&P ratings and their own failure score with actual defaults and come to a straightforward conclusion: Their simple indicator delivers much better predictions about expected defaults than the verdicts of the rating agencies. According to the paper, the failure score is almost twice as reliable:

“We find that this measure (…) is substantially more accurate than rating at predicting failure at horizons of 1 to 10 years. The higher accuracy in predicting the cumulative failure probability is driven by a much higher ability of failure score at predicting marginal default probabilities at horizons of up to 2 years and the fact that credit rating adds little information to marginal default prediction at horizons up to 5 years.”

This leads Hilscher and Wilson to a straightforward conlusion that is rather inconvenient for the suppliers and users of credit ratings: “ratings are in fact a poor predictor of corporate failure.”

As Jens Hilscher explains, those findings probably also apply to sovereign bonds. As a piece of evidence, Hilscher refers to a paper co-authored with Yves Nosbusch in 2010. Both economists analysed the bond spreads of emerging market sovereigns and show that publicly available data about the macroeconomic situation of the borrowers can explain them much better than the verdicts of rating agencies.

The rating agencies rebuff the findings of Hilscher and Wilson. “The research is criticising ratings for missing goals that we do not try to achieve,” asserts Albert Metz, Managing Director Credit Policy Research with Moody’s:

“The objective of our ratings is not to predict absolute default probabilities of individual firms. Such ratings would be more volatile and highly cyclical. The purpose of Moody’s ratings is to produce a stable indicator of relative credit risk.”

At a given point in time, two companies that have a similar rating should have a similar expected loss profile. “That’s our aspiration,” says Metz.

However, according to the paper by Hilscher and Wilson, even this goal is frequently missed: According to their findings, the ratings for different firms prove to be quite inconsistent at a given point in time:

“Firms with the same rating often have widely different default probabilities and firms with very different ratings often have very similar default probabilities. (…) In fact, the 75th percentile AA-rated issuer (two notches below AAA) is more likely to default than the median issuer rated BBB-, the last rating before reaching junk status.”

S&P fundamentally questions the design and the approach of the paper. “If the intention of this study is to assess the track record of credit ratings, then its methodology is flawed and many of the assumptions it makes about the meaning and basis of ratings are wrong”, says Martin Winn, Vice President Communication with S&P.

Winn asserts that a close correlation between ratings and defaults in fact exits. As an example, he refers to the track record of the company since 1980. Only around 1% of corporates that were rated investment grade (BBB- or above) defaulted within 5 years, compared with around 20% of companies rated speculative grade (BB+ or below).

Hilscher and Wilson reply that they do not claim that ratings have no informational content whatsoever. Having a S&P rating was certainly better than no information at all, stresses Hilscher.

However, according to their findings ratings only prove meaningful with regard to the question how a company will fare in bad economic conditions. Higher  rated firms are more likely to survive an economic downturn, the researchers observe: “Ratings reflect sensitivity of credit risk to bad times and, therefore, (..) rating may at least partly capture systematic risk of default.”

However, this does not change the basic findings that a simple failure score based on publicly available information easily beats the ratings when it comes to the assessment of absolute and relative default risks of borrowers. The key question according to Hilscher is: “Why are their measures not more accurate if more accurate information is easily available?”

12 Comments

Filed under Financial Crisis, General Economics

12 Responses to Are Credit Ratings Massively Overrated?

  1. Pingback: Credit Ratings and Credit Risks « Global Index Group

  2. Pingback: Tuesday links: competitive benefits | Abnormal Returns

  3. Chris

    “Ratings reflect sensitivity of credit risk to bad times and, therefore, (..) rating may at least partly capture systematic risk of default.”

    Given the importance of ratings in the financial markets a higher rating should imply better credit terms. Instead of a higher rating capturing lower risk of default might not a higher rating mean easier terms of credit therefore causing a lower risk of default? So even Hilscher and Wilson’s assessment of the informational content of ratings may be overly optimistic, no?

  4. The credit rating agencies are incompetent, which they’ve repeatedly proven. But they are also irrelevant. Look at the market’s reaction after the U.S was downgraded. U.S. debt rallied. That is because the markets already set the credit worthiness of any borrower that has outstanding paper. The price is out there. The price of credit default swaps is out there. That is the ‘real’ credit rating of a sovereign or corporate borrower. Ratings are both redundant and dumber than the crowd-sourced price that already exists. I talk about the failure of the credit rating agencies in my book “Jackass Investing.” I’m pleased to provide readers with a complimentary link to the chapter in which they’re discussed (Myth #13: It’s Best to Follow Expert Advice):
    http://bit.ly/ugSnBf

    Mike Dever
    Author, Jackass Investing: Don’t do it. Profit from it.

  5. Pingback: Tuesday links: competitive benefits

  6. Olive73

    May credit ratings issued by crowdsourcing be an alternative?
    I’ve just found this site: http://www.wikirating.org.
    Interesting idea, need to be improved though.

  7. Are credit ratings massively overrated?

    Yes but not so much because of the reasons here described, which have more to do on whether the credit rating contain good information.

    They are overrated because even though banks and markets give consideration to the ratings when they set the interest rates, amounts and other terms of loans and investment, the regulators also use those same ratings to set the capital requirements for the banks.

    Even though the credit ratings were perfect, the current double counting, allowing banks to earn more risk-adjusted return on bank equity when ratings are good that when bad, condemns the banks to an overexposure to good ratings and an underexposure to bad.

    Let us never forget that even the safest of all harbors can end up dangerously overcrowded, and even the most dangerous of the bays can contain the riches that can save us.

    Stop the nannies from making our economies sissy! Occupy Basel! http://bit.ly/dFRiMs

  8. The truth is that in fact bad rating agencies are systemically less dangerous than good, because if they are very good, the higher will they be able to build their reputational tower, from which we will fall, sooner or later fall… no matter what… when we give their opinions too much voice.

    One of the current problems is that if a credit rating has already been awarded, and especially if it is good, that there is absolutely no incentive for a second opinion, since no one is going to pay the price of a second opinion that might differ from the first opinion only once in awhile. Especially if the First and Official Opinionater, the credit rating agency, has had access to special information on the borrower, as they do have.

  9. Amid

    I do think this article comes in time. It simply confirms the debatability of the ratings in general and the power given to the rating agencies.
    Are investors more equipped without the credit agencies to do their job in a timely fashion? What are the alternatives to them ready to be widely available et having the credentials they have acquired through decades of existence and experience?

    Definitely, Jens Hilscher and Mungo Wilson have made a very interesting case.

  10. If there were no credit ratings, or as used to be the case these were not too trusted, then a lot more individual analysis and opinions would at work. As is, the world seems to have settled for using few presumably qualified eyes. That is sad. Soon there is going to be more how-to-analyze-the-possibilities-the-credit-rating-of-a-company-being-changed courses, that there are going to be courses about how to analyze the company itself. We’ve placed us in the hands of the messengers!

  11. Pingback: Credit rating agencies beaten by a spreadsheet | Left Foot Forward

  12. Pingback: Downgrade the ratings agencies « Emergent Economics