Types
Burke regards failure to use a model (instead over-relying on expert judgment) as a type of model risk.http://www.siiglobal.org/SII/WEB5/sii_files/Membership/PIFs/Risk/Model%20Risk%2024%2011%2009%20Final.pdf Derman describes various types of model risk that arise from using a model:Wrong model
* Inapplicability of model. * Incorrect model specification.Model implementation
* Programming errors. * Technical errors. * Use of inaccurate numerical approximations.Model usage
* Implementation risk. * Data issues. * Calibration errors.Sources
Uncertainty on volatility
Volatility is the most important input in risk management models and pricing models. Uncertainty on volatility leads to model risk. Derman believes that products whose value depends on aTime inconsistency
Buraschi and Corielli formalise the concept of 'time inconsistency' with regards to no-arbitrage models that allow for a perfect fit of the term structure of the interest rates. In these models the currentCorrelation uncertainty
Uncertainty on correlation parameters is another important source of model risk. Cont and Deguest propose a method for computing model risk exposures in multi-asset equity derivatives and show that options which depend on the worst or best performances in a basket (so calledComplexity
Complexity of a model or a financial contract may be a source of model risk, leading to incorrect identification of its risk factors. This factor was cited as a major source of model risk for mortgage backed securities portfolios during the 2007 crisis.Illiquidity and model risk
Model risk does not only exist for complex financial contracts. Frey (2000) presents a study of how market illiquidity is a source of model risk. He writes "Understanding the robustness of models used for hedging and risk-management purposes with respect to the assumption of perfectly liquid markets is therefore an important issue in the analysis of model risk in general." Convertible bonds,Spreadsheet Errors
Many models are built using spreadsheet technology, which can be particularly prone to implementation errors. Mitigation strategies include adding consistency checks, validating inputs, and using specialized tools.Quantitative approaches
Model averaging vs worst-case approach
Rantala (2006) mentions that "In the face of model risk, rather than to base decisions on a single selected 'best' model, the modeller can base his inference on an entire set of models by using model averaging." This approach avoids the "flaw of averages". Another approach to model risk is the worst-case, or minmax approach, advocated in decision theory by Gilboa and Schmeidler. In this approach one considers a range of models and minimizes the loss encountered in the worst-case scenario. This approach to model risk has been developed by Cont (2006). Jokhadze and Schmidt (2018) propose several model risk measures using Bayesian methodology. They introduce superposed risk measures that incorporate model risk and enables consistent market and model risk management. Further, they provide axioms of model risk measures and define several practical examples of superposed model risk measures in the context of financial risk management and contingent claim pricing.Quantifying model risk exposure
To measure the risk induced by a model, it has to be compared to an alternative model, or a set of alternative benchmark models. The problem is how to choose these benchmark models. In the context of derivative pricing Cont (2006) proposes a quantitative approach to measurement of model risk exposures in derivatives portfolios: first, a set of benchmark models is specified and calibrated to market prices of liquid instruments, then the target portfolio is priced under all benchmark models. A measure of exposure to model risk is then given by the difference between the current portfolio valuation and the worst-case valuation under the benchmark models. Such a measure may be used as a way of determining a reserve for model risk for derivatives portfolios.Position limits and valuation reserves
Jokhadze and Schmidt (2018) introduce monetary market risk measures that covers model risk losses. Their methodology enables to harmonize market and model risk management and define limits and required capitals for risk positions. Kato and Yoshiba discuss qualitative and quantitative ways of controlling model risk. They write "From a quantitative perspective, in the case of pricing models, we can set up a reserve to allow for the difference in estimations using alternative models. In the case of risk measurement models, scenario analysis can be undertaken for various fluctuation patterns of risk factors, or position limits can be established based on information obtained from scenario analysis." Cont (2006) advocates the use of model risk exposure for computing such reserves.Mitigation
Theoretical basis
* Considering key assumptions. * Considering simple cases and their solutions (model boundaries). * Parsimony.Implementation
* Pride of ownership. * Disseminating the model outwards in an orderly manner.Testing
* Stress testing andExamples of model risk mitigation
Parsimony
Taleb wrote when describing why most new models that attempted to correct the inadequacies of the Black–Scholes model failed to become accepted: "Traders are not fooled by the Black–Scholes–Merton model. The existence of a 'volatility surface' is one such adaptation. But they find it preferable to fudge one parameter, namely volatility, and make it a function of time to expiry and strike price, rather than have to precisely estimate another." However, Cherubini and Della Lunga describe the disadvantages of parsimony in the context of volatility and correlation modelling. Using an excessive number of parameters may induce overfitting while choosing a severely specified model may easily induce model misspecification and a systematic failure to represent the future distribution.Model risk premium
Fender and Kiff (2004) note that holding complex financial instruments, such as CDOs, "translates into heightened dependence on these assumptions and, thus, higher model risk. As this risk should be expected to be priced by the market, part of the yield pick-up obtained relative to equally rated single obligor instruments is likely to be a direct reflection of model risk."Case studies
* NatWest—Interest rate options and swaptions—incorrect model specification. * Bank of Tokyo-Mitsubishi—Interest rate options and swaptions. * LTCM—lack of stress testing—Crouhy, Galai, and Mark. * Barclays de Zoete Wedd (BZW)—Mispriced currency options. Evaluation of various finance models * National Australia Bank $3 Billion AUD loss on Homeside interest rate model. * 2007–2012 global financial crisis – Over-reliance on David X. Li's Gaussian copula model misprices the risk of collateralized debt obligations.See also
* Risk management * Statistical model specification *Notes
References
* * * * * * * * * * {{cite book , first=Nassim , last=Taleb , year=2006 , title=Fooled by Randomness: The Hidden Role of Chance in Life and in the Markets , publisher=Wiley , isbn=1-4000-6793-6 , url-access=registration , url=https://archive.org/details/fooledbyrandomne00tale * US Federal Reserve Policy http://www.federalreserve.gov/bankinforeg/srletters/sr1107a1.pdf SUPERVISORY GUIDANCE ON MODEL RISK MANAGEMENT Actuarial science Financial risk Financial models