How should one read JPMorgan’s Whale Report? I suppose “not” is an acceptable answer; the Whale’s credit derivatives losses at JPMorgan’s Chief Investment Office are old news by now, though perhaps his bones point us to the future. One way to read it is as a depressing story about measurement. There were some people and whales, and there was a pot of stuff, and the people and whales sat around looking at the stuff and asking themselves, and each other, “what is up with that stuff?” The stuff was in some important ways unknowable: you could list what the stuff was, if you had a big enough piece of paper, but it was hard to get a handle on what it would do. But that was their job. And the way you normally get such a handle, at a bank, is with a number, or numbers, and so everyone grasped at a number.
The problems were (1) the numbers sort of sucked and (2) everyone used a different number. Here I drew you a picture:1
Everyone tried to understand the pool of stuff through one or two or three numbers, and everyone failed dismally through some combination of myopia and the fact that each of those numbers was sort of horrible or tampered or both, each in its own special way. Starting with:
VaR: Value-at-risk is the #1 thing that people talk about when they want to talk about measuring risk. To the point that, if you want to be all “don’t look at one number to measure risk, you jerks,” VaR is the one number you tell the jerks not to look at.
It didn’t help that much here, for reasons both systematic (VaR is based on historic moves in prices and didn’t take into account that (1) the CIO basically was the market in some of what it traded and (2) everyone knew it) and idiosyncratic (JPM built a new VaR model and implemented it and only later realized there was an error in the spreadsheet2). More troubling, when JPMorgan built its new VaR model, which reduced VaR with no change in assets, it kept its VaR limits the same.3 If your risk limits made sense under the old model, and the new model is designed to show less risk for the same assets, then why would the same risk limits still make sense under the new model?
RWAs: With Basel III slowly phasing in to JPM’s reporting, the CIO was given a (not particularly strictly enforced) mandate to reduce Basel III risk-weighted assets. RWAs are among other things a risk measure. The more risky assets you have the more risk you have, n’est-ce pas? But it’s not that simple: if you have two inversely correlated risky assets you should have less risk than having either of them on its own. So when you’re told, as the CIO was, to reduce RWAs, you can (1) get rid of some risky assets, or you can (2) add inversely correlated assets, or you can, as a CIO executive marvelously put it, (3) “win on the methodology.”4 Note that methods (1) and (2) reduce risk; method (3) just changes some spreadsheets. The actual approach was some mix of (2) and (3), with an uncomfortably high dose of (3) and, for that matter, an uncomfortably high dose of being wrong about the correlations.
P&L: P&L is a sort of a risk measure insofar as losing a ton of money today provides some evidence that you might lose a ton of money again tomorrow. Conversely not losing a lot of money today should be some weak indication either (1) that you won’t lose a lot of money tomorrow or (2) that you’ve manipulated your P&L. Guess which? Page 64:5
April 10 was the first trading day in London after the “London Whale” articles were published. When the U.S. markets opened (i.e., towards the middle of the London trading day), one of the traders informed another that he was estimating a loss of approximately $700 million for the day. The latter reported this information to a more senior team member, who became angry and accused the third trader of undermining his credibility at JPMorgan. At 7:02 p.m. GMT on April 10, the trader with responsibility for the P&L Predict circulated a P&L Predict indicating a $5 million loss for the day; according to one of the traders, the trader who circulated this P&L Predict did so at the direction of another trader.
Five million or seven hundred million, ONE OR THE OTHER.6
CSW 10% and CS01: JPMorgan tracked sensitivity of its portfolio to (1) one-basis-point moves in credit spreads (CS01 or CSBPV, as JPM called it) and (2) 10% moves in credit spreads (CSW 10%). The former was criticized for not capturing convexity, among other issues; the latter was more popular but still pretty first-order. After all the whole point of the CIO book was to be CSW 10% neutral, while serving as a tail-risk hedge: as they put it in an internal presentation (page 38), it was meant to be an “option with positive convexity, positive carry and upside on large spread widening and default waves.”
This was a good idea but it didn’t work: the measurement didn’t reflect what happened, because the raw sensitivities to across-the-board credit spread moves didn’t reflect the correlation between moves in investment-grade spreads (which JPMorgan was mostly long) and high-yield spreads (which they were mostly short). So the “neutral” portfolio kept losing money. Pages 40-43:
By mid-March, the Synthetic Credit Portfolio was still experiencing mark-to-market losses. A trader performed a detailed analysis around this time and determined that, even though the Synthetic Credit Portfolio appeared to be balanced under CSW 10%, its actual performance – and in particular, the fact that it lost money when the markets rallied – suggested that it continued to have a short bias. The trader attributed this to the significant amounts of protection that he had purchased since January, and he therefore considered what steps he might take to finally balance the Synthetic Credit Portfolio. … On March 23, a trader explained to CIO Market Risk the trading he had done: “[I] switched the book to long risk[.] [I] am done[.]” He explained his view that “this is it for a neutral profile[, and] right now we have a market neutral ratio between HY and IG.”
So they got longer investment-grade credit, compounding the eventual losses.
Scenario analysis: After JPMorgan got advanced copies of the “London Whale” articles, members of the CIO team were directed to do scome scenario analyses of potential losses. Good idea! But the actual analysis had two main problems. First, it analyzed some scenarios (“bond market crash” or a “Middle East shock”), but neglected to address the one that happened (“we get picked off after articles on our positions are published”). This is sort of natural in the abstract – you can’t predict every scenario that might happen – though a little embarrassing insofar as the analyses were done after the articles came out, though before the next trading day. In estimating their risk, CIO’s traders didn’t even to attempt to analyze trading dynamics in the product that they traded.
Second … second this is just terrible (pages 59-60):
Specifically, [one CIO trader] informed the trader who had generated the estimates that he had too many negative scenarios in his initial work, and that he was going to scare Ms. Drew if he said they could lose more than $200 or $300 million. He therefore directed that trader to run a so-called “Monte Carlo” simulation to determine the potential losses for the second quarter. A Monte Carlo simulation involves running a portfolio through a series of scenarios and averaging the results. The trader who had generated the estimates did not believe the Monte Carlo simulation was a meaningful stress analysis because it included some scenarios in which the Synthetic Credit Portfolio would make money which, when averaged together with the scenarios in which it lost money, would result in an estimate that was relatively close to zero. He performed the requested analysis, however, and sent the results to the other trader in a series of written presentations over the course of the weekend. This work was the basis for a second-quarter loss estimate of -$150 million to +$250 million provided to senior Firm management.
There are contexts where you can say “well, our worst-case loss is $1bn, and our best-case gain is $950mm, so our expected loss is $50mmm,” but telling your CEO what your worst-case loss might be is not one of them. Bad time to average.7
I’m gonna give the prize for worst measurement, though, to Ina Drew. Here is Felix Salmon:
Ina Drew, the person in charge of the portfolio, sent an email to JP Morgan’s CEO and CFO, in which she observed that the move [after the stories came out] was “an eight-standard-deviation event”.
The report doesn’t say how many eight-sigma events the CIO has ever seen: my guess is that this is the only one. But here’s an idea of how crazy eight-sigma events are: under a normal distribution, they’re meant to happen with a probability of roughly one in 800 trillion. The universe, by contrast, is roughly 5 trillion days old: you could run the universe a hundred times, under a normal distribution, and still never see an eight-sigma event. If anything was a black swan, this was a black swan. And it didn’t help JP Morgan’s “tail risk book” one bit. Quite the opposite.
Both Felix and Ina are wrong here in important ways. First, Felix. The JPMorgan CIO book was billed internally (and externally) as a “tail risk book” but no one should interpret that as meaning “this makes money on the occurrence of every tail event.” That is impossible. Things occur. The book won’t make money if an asteroid hits the earth.8 A tail risk book isn’t “this book will make money if the market moves against it” – that’s impossible – it’s “this book will make money if some specifically identified somewhat unlikely bad thing happens.” The unlikely bad thing here was to some approximation “credit spreads blowing out”; the actual event that happened was “the CIO got correlations very wrong and then everyone ganged up on them.” It’s hard to hedge the risk of getting your hedge wrong.9
But second, Ina Drew bought into that same pseudo-methodology. You can’t tell someone that a thing that just happened was an eight sigma event. It just happened. “We ran some numbers, and they showed that this was an eight sigma event, so the event is wrong.” Nope. The numbers were wrong. Or: the numbers were right.10 Treating those numbers as a substitute for the thing was wrong.
As losses mounted, everyone figured this out:
These losses were inconsistent with the earlier loss estimates and prediction from one of the traders that the market would “mean revert,” and they caused Messrs. Dimon, Braunstein and Hogan as well as Ms. Drew to question whether the Synthetic Credit Portfolio team adequately understood the Synthetic Credit Portfolio or had the ability to properly manage it.
I don’t fully buy the popular position that no one could understand the Synthetic Credit Portfolio or properly manage it, but I’ll cheerfully believe that no one could adequately understand it just by focusing on a few high-level numbers. Particularly when all of those numbers were massaged to tell those managers what they wanted to hear. That, in the end, is what undermines your credibility at JPMorgan.
Report of JPMorgan Chase & Co. Management Task Force Regarding 2012 CIO Losses [JPM]
1. This picture is just for pretend; don’t take the lines too seriously. Consider, though, for instance:
- “Ms. Drew did not receive detailed trading or position reports on the Synthetic Credit Portfolio in the ordinary course, did not request any such reports during this time, and regularly monitored only the Synthetic Credit Portfolio’s profits and losses, VaR and stress VaR.” (page 32)
- “In the last week of December, Mr. Braunstein asked CIO to evaluate the impact of a further reduction of $20, $40 or $60 billion of RWA.” (page 27)
- “On March 30, the executive responsible for the Synthetic Credit Portfolio requested assistance from Firm-wide Market Risk in understanding the relationship between their trading and RWA. … He also stated that he was no longer confident in his team’s ‘ability to achieve the targeted RWA and their understanding of the synthetic levers to achieve the RWA objectives.’” (page 45)
2. Page 128:
The Model Review Group employee discovered that West End defaulted to running Uniform Rate rather than Gaussian Copula in this cell, including for purposes of calculating the VaR, contrary to the language in the Model Review Group approval. Although this error did not have a significant effect on the VaR, the incident focused the reviewer’s attention on the VaR model and ultimately led to the discovery of additional problems with it.
3. Pages 77-79:
Beginning in mid-January 2012, CIO breached its VaR limit on multiple days, which also contributed to breaches of the Firm’s VaR limit. CIO explained to Mr. Hogan and Firm-wide Market Risk that the breaches were being addressed in two ways: (1) continued management of CIO’s positions, and (2) implementation of a new, “improved” VaR model for CIO. … In an e-mail to Mr. Hogan on January 25, Mr. Goldman reported that the new model would be implemented by January 31 “at the latest” and that it would result in a “significant reduction” in the VaR. … There was no corresponding change made to the CIO Global 10-Q VaR limit at the time of the new model’s implementation – i.e., it remained at $95 million. Following implementation of the new model, the CIO VaR fell below the limit, as expected.
4. Page 39:
On March 1, the day after the CIO Business Review, an executive with responsibility for the Synthetic Credit Portfolio e-mailed one of the traders to express concern that if the traders needed to “[a]ctually reduce the [Synthetic Credit Portfolio]” in order to decrease RWA, they would not be able to “defend” their positions. This e-mail appears to address the concern that an unwind of positions to reduce RWA would be in tension with “defending” the position. The executive therefore informed the trader (among other things) that CIO would have to “win on the methodology” in order to reduce RWA. This phrase refers to the traders’ goal, described above, to convince the Firm that it should change the methodology of the model used to calculate RWA for the Synthetic Credit Portfolio.
5. And it was pretty systematic. Page 47:
At certain points throughout early 2012, the information the [junior] trader [who marked the CIO's book] was collecting from
the market indicated losses in the Synthetic Credit Portfolio. But on a number of days beginning in at least mid-March, at the direction of his manager, he assigned values to certain of the positions in the Synthetic Credit Portfolio that were more beneficial to CIO than the values being indicated by the market. The result was that CIO underreported the losses, both on a daily basis and on a year-to-date basis.
6. Pages 64-65. Note they settled on $400mm, why not.
7. Also enjoyable is that, during these discussions, this happened (fn. 76 on page 61):
The full text of the senior team member’s e-mail stated that they must “prove the point [that everything's fine] today with as much ambiguity as poss[ible].” It is the Task Force’s understanding that he meant to say “little” rather than “much.”
Freud’s understanding might differ.
8. Nor will it make money if the counterparties from whom JPMorgan bought protection defaulted on their trades. This is a thing that a lot of people spend a lot of time worrying about, and I found it sort of poignant to see JPMorgan worrying about it as the Whale story broke:
On April 11, Mr. Wilmot circulated to Messrs. Dimon, Braunstein and others a presentation on the Synthetic Credit Portfolio that addressed, among other things, notional exposure relative to various counterparties, maturities, certain positions and profit-and-loss scenarios, noting that it had been reviewed with Jes Staley, Mr. Braunstein, Ms. Drew, Mr. Zubrow and Mr. Hogan.
Of course in the end those counterparties made out like bandits and one of them hired Jes Staley.
9. Oversimplified! Nassim Taleb would not approve. It’s easy to hedge the risk of getting your hedge wrong: make your hedge, and the thing it hedges, smaller! That’s sort of the opposite of what CIO did; a number that they didn’t pay much attention to was just notional, which ballooned throughout the life of this story.
10. Actually Ina was being a little conservative. If she believed her trader’s estimate (page 60) that he was “80pct confident the pnl for q2 is going to range between -150m and 250m,” then some simple fake-o math gives you an eight-sigma quarterly loss of $1.25bn, or an eight-sigma daily loss of $155mm. The actual one-day loss was like three times that. Twenty sigmas. Impossimable!