Banks' Risk Measurements Rarely Off By Much More Than A Factor Of Ten

Banks are opaque, or so I hear, and so the only way many people can stand to be around them is if they can have some sort of number to serve as a flashlight into all that opacity. One of the big numbers is Basel III risk-weighted assets, which are intended to, as the name says, measure how much stuff a bank holds, weighted by the riskiness of the stuff. RWAs are related to another popular number - they are in many cases calculated based on value-at-risk models - and they determine capital requirements: the more risky assets you have, the more long-term loss-absorbing funding you need to have, to insulate you from loss if the risks come true.

All numbers deceive, though, and manypeople have noted that RWAs vary wildly across banks, with some banks reporting much lower RWAs per total account assets than others without any obvious decrease in risk. And since RWAs are based in large part on internal models, there is some suggestion that banks optimize their models to reduce RWAs. So risk-weighted assets don't have any obvious correlation with (1) risk or (2) assets.

Which seems bad, as a first cut, but maybe isn't: not having an obvious correlation doesn't mean there's no correlation. RWA critics are just looking at public information, and public information is, as noted, opaque. Maybe all the banks really are consistently and conscientiously measuring the riskiness of their assets, and just happen to hold different assets. Which is why it's nice that the confab of bank regulators at the Bank of International Settlements went and issued a report on consistency of risk-weighted assets for market risk.1 The authors of this report, between them, regulate pretty much every big bank in the world. So they could go look at each bank's trading book, see what assets they have, see how they risk weight them, and compare that to how other banks weight them, done.2

One kind of immediate thing to notice is that that didn't happen. They're as confused as you are:

A preliminary review suggests that differences in the composition of trading assets can only partially explain variation in mRWAs. For example, upon examining the US banks in the sample, some with higher mRWAs have a greater proportion of illiquid trading assets, including distressed debt and illiquid equity. However, the relationship appears weak and some banks with illiquid equity holdings report relatively low mRWAs. One reason may be that these banks are able to hedge their investments in risky trading assets, lowering their mRWAs. Overall, the correlation of the composition of trading assets to mRWAs appears low. More generally, there is insufficient public information available to allow a comprehensive study across banks regarding the effect of the composition of trading assets on the variation in mRWAs.

Why not look at, y'know, non-public information? I think part of the answer is - reasonably! - that the regulators didn't want to do that much work for what is essentially an exercise in curiosity. But another part is this:

Supervisory data collection while consistent within jurisdictions is not consistent across jurisdictions. In addition, as mRWAs is for most banks only a small share of total RWAs (less than 10%), some jurisdictions collect only limited additional data for supervisory purposes. The Committee surveyed each jurisdiction within the sample of banks to gain an understanding of the level and extent of supervisory information related to the regulatory reporting of mRWAs and its components. For purposes of this report it was determined that the utility of supervisory data to perform a meaningful cross jurisdictional comparison of mRWAs and the underlying drivers relative to the composition of a trading portfolio is limited.

That's footnote 16, btw. All the worst things are in footnotes.

After that look at public information, which tells you what people knew already, that banks appear to be inconsistent in their aggressiveness in risk-weighting assets but it's hard to tell why or how or whether it's a big deal, the report turns to a sample portfolio exercise. Here the regulators gave the banks a bunch of different portfolios and asked them to risk-weight them, both alone and aggregated to include diversification benefit. The results are ... the results pretty much confirm that bank risk-weighting is very inconsistent:

The X axis here shows the various portfolios, and the Y axis shows the VaR of those portfolios - which more or less linearly determines RWAs - calculated by different banks, normalized to 100%. Take portfolio 1, on the left hand side. The different banks computed a VaR for this portfolio ranging from, say, 0.8x to 2.0x - the top bank showed two and a half times more risk for this portfolio than the bottom bank. This portfolio was: long a stock index.3 How much capital should you have to support a simple portfolio of long equities? I dunno, and it seems like no one else does either. This is one of the least dispersed measurements; portfolios 13 (a relative value credit trade, long GSK bonds and short ROSW bonds) and 14 (covered FX calls, long EUR/USD forwards and short EUR/USD calls) are pretty much total guesswork, with order-of-magnitude type ranges of risk weights.

The problem is that banks have a lot of leeway to set up their own internal models, rather than using standardized risk weightings. One might expect each bank's internal models to be optimized to require as little capital as possible for its particular portfolio of assets. One would appear to be right:

The more you use your own models, the more favorable your results, and the less capital you have to have. I submit to you that if it were otherwise, the internal modelers would not be doing their jobs.4

The BIS folks have some suggested fixes, along the lines of:

• improved public disclosures of modelling choices and the drivers of RWA numbers,
• narrowing modelling choices to reduce flexibility to, for instance, choose different historical periods to calibrate VaR models, and
• "additional supervisory guidance for upholding consistent model standards."

So I guess? The improved disclosure thing is hard to argue with. But for the rest ... one sort of pleasing aspect of this exercise is that it shows dispersions but not the "right" number. Some banks think equity indices aren't that risky, others think that they're, um, 2.5x more risky, but nobody actually knows. Why would Basel?

Maybe it's a good thing that banks are all over the place on how they risk-weight assets, and that their decisionmaking process seems to be more (1) decide what to invest in and (2) optimize RWA models to make that look good, rather than the reverse. Maybe that's better than having a monoculture where everyone assigns the same risk-weight to each asset and thus everyone flocks to the same assets, which are less risky over whatever period Basel wants to use to calibrate VaR models.5 Perhaps the BIS should be celebrating the fact that nobody has any idea how to calculate risk-weighted assets. That certainly sounds like more fun than going and figuring out the right answer.

Regulatory consistency assessment programme (RCAP) - Analysis of risk-weighted assets for market risk [BIS, and pdf]
Reforming RWAs, some reading [FTAV]

1.That is, RWAS in the trading book, as opposed to loans etc. in the banking book. The market risk-weighted assets are sometimes abbreviated "mRWAs" in the report.

2.Oversimplified: much of the trick is in how you aggregate the book as a whole rather than individual assets. Still, tot up all the assets, look at the risk weights, run a regression, etc.

3.Well, an index future. From the appendix, it's "Long 10 contracts ATM 3-month FTSE 100 index futures."

4.To be fair that's the 2010 chart which I'm using because it's more suggestive. There's a 2011 chart where the trend is perhaps less apparent:

5.But, sort of ipso facto, more risky over the next period.