Qualts vs. Quants

One descriptive you often see in finance-related job listings is "quant," which is short for "quantitative analyst." In a nutshell, this refers to individuals, usually with high-level degrees in math or physics, who apparently understand complicated, jargon-filled formulas and esoteric academic theories, and who know how to translate that knowledge into the models and programs that Wall Street has come to rely on in recent years to try and make money and control risk.

But what you rarely see, no matter how hard you look, are ads for "qualts," or qualitative analysts. These are individuals who've been around the block a few times, who grasp the concept that just because a model says something is so doesn't mean it is, who know that "low-risk" is not the same as "no risk," who have a "nose" for danger, and who understand that even though you can't model something mathematically does not mean it is not relevant or important in terms of risk-management or overall decision-making.

Given the gushers of red ink and the far-reaching structural damage that many "quants" -- and their models -- appear to have caused to the markets, the economy, and the global financial system in recent times, I wonder why we are not seeing much less of the former kinds of ads -- in fact, they seem as popular as ever -- and a lot more of the latter types?

Anyway, in "Death of VaR Evoked as Risk-Taking Vim Meets Taleb's Black Swan," Bloomberg's Christine Harper discusses the issue of flawed models and features some insights from Nassim Taleb, a long-time nemesis of today's false idols, the quants.

The risk-taking model that emboldened Wall Street to trade with impunity is broken and everyone from Merrill Lynch & Co. Chief Executive Officer John Thain to Morgan Stanley Chief Financial Officer Colm Kelleher is coming to the realization that no algorithm or triple-A rating can substitute for old-fashioned due diligence.

Value at risk, the measure banks use to calculate the maximum their trades can lose each day, failed to detect the scope of the U.S. subprime mortgage market's collapse as it triggered more than $130 billion of losses since June for the biggest securities firms led by Citigroup Inc., Merrill, Morgan Stanley and UBS AG.

The past six months have exposed the flaws of a financial measure based on historical prices that securities firms use idiosyncratically and that doesn't anticipate every potential disaster, such as the mistaken credit ratings on defaulted subprime debt.

"Finance is an area that's dominated by rare events," said Nassim Taleb, a research professor at London Business School and former options trader. "The tools we have in quantitative finance do not work in what I call the 'Black Swan' domain."

Taleb's book "The Black Swan," published last year by Random House, describes how people underestimate the impact of infrequent occurrences. Just as it was assumed that all swans were white until the first black species was spotted in Australia during the 17th century, historical analysis is an inadequate way to judge risk, he said.

Merrill vs. Goldman

Executives at Merrill, Morgan Stanley and UBS took steps in the past six weeks to overhaul their risk-management groups after internal models failed to foresee the first annual decline in house prices since the Great Depression that eroded five years of trading gains.

Goldman Sachs Group Inc., the firm with the highest nominal VaR, was the sole investment bank to report record earnings in the fourth quarter, while New York-based Merrill, which had the second-lowest nominal VaR of the five biggest U.S. securities firms, posted a $9.8 billion loss for the last three months of 2007, the biggest in its 94-year history.

Thain, who replaced the ousted Stan O'Neal last month at Merrill, said Jan. 17 that the largest U.S. brokerage should stop making trades that have the potential to wipe out profits. He revamped the unit overseeing trading positions and hired former Goldman executive Noel Donohoe as co-chief risk officer.

UBS CEO Marcel Rohner told employees two weeks ago that Europe's biggest bank will scale back risk taking after reporting a $15 billion writedown last year for subprime- infected investments. As part of the plan, Zurich-based UBS shut a U.S. fixed-income trading unit.

'Necessary Changes'

At New York-based Morgan Stanley, which disclosed a $3.56 billion fourth-quarter loss after writing down mortgage-related and other securities by $9.4 billion, the risk department will now report directly to Kelleher instead of to the trading heads. The firm said it plans to hire more risk managers.

"This loss was a result of an error in judgment that occurred on one desk in our fixed-income area and also a failure to manage that risk appropriately," Morgan Stanley Chief Executive Officer John Mack said on Dec. 19. "We're moving aggressively to make necessary changes."

Stronger management might help protect firms against rogue employees. Societe Generale SA, France's second-largest bank by market value, said last week that unauthorized trades caused a $7.2 billion loss, the biggest in banking history. The trading wiped out about two years of pretax profit at the Paris-based company's investment-banking unit.

Rogue Trader

Societe Generale said the trader was Jerome Kerviel, 31, who joined the bank's trading division in 2006 after working six years in the company's back office. He had "intimate and perverse" knowledge of the bank's controls, which enabled him to avoid detection, Co-Chief Executive Officer Philippe Citerne told reporters on Jan. 24. Citerne didn't identify the trader.

"That's a whole other side of risk management," said Benjamin Wallace, who helps oversee $850 million at Grimes & Co. in Westborough, Massachusetts, which owns shares of Morgan Stanley and Merrill. "That's not the modeling, that's making sure that people are overseen properly."

Hiring risk managers and giving them more power won't alter the mistake that led to last year's slump and that was Wall Street's dependence on statistics to quantify risks, Taleb said.

"We have had dismal failures in quantitative finance in measuring these risks, yet people hire quants and hire risk managers simply to back up their desire to take these risks," he said. "There are some probabilities that you cannot compute."

Bigger Bets

Banks and securities firms increased the size of their trades during the past decade on interest rates, stocks, commodities and credit. Trading revenue for the five largest securities firms -- Goldman, Morgan Stanley, Merrill, Lehman Brothers Holdings Inc. and Bear Stearns Cos. -- climbed to a combined $71.1 billion by 2006 from $29.1 billion in 2002. The higher profits added to the firms' capital, enabling even bigger trading bets.

"You can scale your value at risk relative to your book value," said Wallace. "It was a self-reinforcing part of the cycle."

Goldman's average daily VaR more than tripled to $151 million in the fourth quarter from $46 million five years earlier, according to company reports. Goldman's VaR was almost twice as high as Merrill's in the third quarter.

Merrill said third-quarter daily average VaR was $76 million, compared with Goldman's $139 million, Morgan Stanley's $87 million, Lehman's $96 million and Bear Stearns's $32 million.

Different Methodologies

All the New York-based firms base their calculations at a confidence level of 95 percent, meaning they don't expect one- day drops to exceed the reported amount more than 5 percent of the time.

The amounts differ in part because every firm uses their own methodology and data. For instance, Lehman uses four years of historical data to calculate VaR, with a higher weighting given to more recent time periods, while Morgan Stanley provides VaR calculations using both four years and one year of market data.

"If you compare what peoples' values at risk are versus what their losses were in the third quarter or fourth quarter, the numbers are astounding," said David Einhorn, president and co-founder of hedge fund Greenlight Capital LLC in New York. "There are a lot of things that probably the value-at-risk model said would have trivial losses 95 percent of the time or 99 percent of the time but are now having a huge loss."

Mortgage Defaults

Merrill's highest one-day value at risk in the third quarter was $92 million, indicating that the firm's maximum expected cost during the 63-trading day period would be $5.8 billion. In fact, the firm wrote down $8.4 billion from the value of collateralized debt obligations, subprime mortgages and leveraged finance commitments, 45 percent more than the worst- case scenario.

All of the risk-measurement tools failed to prepare Merrill for the unforeseen declines on triple-A rated securities backed by subprime mortgages, according to the company's third-quarter filing with the U.S. Securities and Exchange Commission. The firm's writedowns related to the highest-rated portions of CDOs backed by pools of home loans, which plunged in value as defaults on the underlying mortgages soared.

"VaR, stress tests and other risk measures significantly underestimated the magnitude of actual loss from the unprecedented credit market environment," Merrill's filing said. "In the past, these AAA ABS CDO securities had never experienced a significant loss in value."

Meriwether's Fund

Securities firms developed statistical models during the early 1990s to better quantify risks as the trading of bonds, stocks, currencies and derivatives increased. J.P. Morgan & Co., now part of JPMorgan Chase & Co., helped popularize the use of value at risk as the primary measurement tool in 1994 when it published its so-called RiskMetrics system.

Four years later, two events helped demonstrate the drawbacks in using statistical analysis based on historical market movements to measure risk. Russia's bond default sent fixed-income markets into a tailspin and Long-Term Capital Management LP, the Greenwich, Connecticut-based hedge fund run by former Salomon Brothers trader John W. Meriwether, had to be bailed out after $4 billion of trading declines.

'Market Stress'

Russia's default risk was underestimated because value-at- risk computations used by investment banks depended on market events of the preceding two to three years, when nothing similar had occurred, according to Wilson Ervin, who's now chief risk officer at Zurich-based Credit Suisse Group, Switzerland's second-biggest bank after UBS.

Long-Term Capital Management, which amplified its risk by relying on borrowed money for most of its trading bets, blew up in part because it didn't anticipate that investor panic after the Russian default would cut the value of any risky debt, whether it was issued by a country, sold by a company, or backed by mortgages.

The riskiest Russian and Brazilian bonds owned by the fund plunged far more than the safer Russian and Brazilian bonds that it had bet against as a hedge, according to "When Genius Failed," the book written by Roger Lowenstein.

"In a market stress event, some individual sectors that previously appeared unrelated do move together, and as a result, the organization could take losses on both of them or even on positions that were previously deemed to be a hedge," said Ed Hida, the partner who runs the risk strategy and analytics services group at Deloitte & Touche LLP in New York.

Worst-Case Scenario

The other risk tool commonly used by securities firms, known as stress testing or scenario analysis, also failed to prepare the industry for the plummeting value of AAA-rated securities that had previously been deemed the most creditworthy, he said.

"Stress tests are only as good or as predictive as the scenarios used and in many cases the scenarios that played out were much more severe than people anticipated," Hida said. "One lesson learned is that these stress tests should be broader, should consider more scenarios."

Kelleher, who became Morgan Stanley's CFO in October, explained the flaw in the firm's stress testing in a Dec. 19 interview, the day the company reported its first unprofitable quarter.

"Our assumptions included what at the time was deemed to be a worst-case scenario," he said. "History has proven that the worst-case scenario was not the worst case."

Credit Suisse

At Credit Suisse, one of the firms that have so far skirted the worst subprime declines, Ervin said value at risk played no role in helping him navigate the market turmoil.

"Once you go into a crisis like this, I think risk is much more about sitting down with traders, and talking about very specific issues and scenarios," he said. "VaR we know will kind of lag going into a crisis so we don't really watch that as a crisis indicator."

Still, Ervin said VaR provides a service if used every day because it can pick up fluctuations in the risk that the firm is taking in some distant region or an arcane product that might not otherwise be noticed.

Investment banks will continue to take unsafe risks as long as traders are rewarded for making profits, leaving shareholders, bondholders and sometimes taxpayers to shoulder the consequences, Taleb said.

Wall Street traders "make an annual bonus and get an annual review based on risks that don't show up on an annual basis," Taleb said. "You have all the incentive in the world to take these risks."

Comments

[Independent Accountant]

I've been saying these things for years.

[Deron Kawamoto]

The fundamental problem with VaR is that it is based on a flawed statistical model. Benoit Mandelbrot demonstrated years ago that investment returns cannot be accurately described using a standard gaussian distribution (normal bell curve). Instead, he found that they follow a non-gaussian levy stable distribution. Basically, the tails of the distribution are much "fatter" than the Black-Scholes based approach would suggest. Confidence intervals and standard deviations based on a normal distribution will be worthless as predictors of volatility since the variance of the stable distribution that Mandelbrot fitted to the data is extreme.

It is no surprise then that the VaR models break down under stress. It's almost comical for a trader to say that there were 20 sigma events 3 days in a row. The problem with the VaR model is not that it is a complete failure. The problem is that it actually describes market behavior in a reasonably accurate fashion most of the time. But only fails when it is needed most - at points of maximum volatility. It therefore creates a false sense of confidence in the risk manager until it's too late.

Although Taleb mentions Mandelbrot, I think that the research by this mathematician deserves much more coverage.

[Contrarian Investors' Journal]

We would like to share with all of you a very interesting quote from a long buried Austrian School book, Crises & Cycles by Wilhelm Röpk: "Why is the market so easily tossed and turned by dribs and drabs of data?" at http://cij.inspiriting.com/?p=352. This book was first published in 1936 and was originally written in German.

It looks that the seeds of today's quant (empirical/quantitative) extremism is sown for a very long time already.

[Mario]

It's pathetic that all those “savvy” investors made bets based on one or two years of data! It makes non sense! Imagine if nasa projected their rockets based only on surface conditions. If they hire economics historian the outcome would be other!

[W.C. Varones]

>>"In a market stress event, some individual sectors that previously appeared unrelated do move together, and as a result, the organization could take losses on both of them or even on positions that were previously deemed to be a hedge," said Ed Hida, the partner who runs the risk strategy and analytics services group at Deloitte & Touche LLP in New York.

The only person dumber than Deloitte is someone who pays them for advice:

http://wcvarones.blogspot.com/2008/02/toilette-douche.html

[Marcel]

Absolutely agree that the VAR is not sufficient, here is a nice example I saw on Bloomberg.

On Tuesday night, Bloomberg Television, the London-based financial channel, said that Merrill Lynch found that using the Value at Risk (VAR) model, required by the supervisors, had calculated its maximum exposure equal to $65 million.

Instead, it lost $8 billion.

Yesterday, the president of Merrill Lynch gave a lecture, saying that VAR is not able to calculate market risk exposure, and it is incapable of estimating credit risk.

This risk has been identified in this book from 2004 book Economic Capital Allocation (Chorafas). This book also gave advice on how to develop and use a better model than VAR.