VAR Backtesting Methodology
The Basel backtesting procedure tests the null hypothesis that the bank’s VaR model predicts losses accurately against the alternative hypothesis that the model predicts losses incorrectly.
The test statistic used is based on the number of exceptions generated by the VaR model. For a given trading day, an exception occurs when the loss exceeds the model-based VaR. The test postulates that the probability of an exception, p, is equal to 0.01 and tests it against the alternative hypothesis that the probability of an exception is greater than 0.01. The test is based on the number of exceptions in 250 trading days. Given the chosen test statistic, the test rejects the VaR model if the number of exceptions is greater than or equal to 10 and accepts the model otherwise (see technical details below).
When evaluating statistical tests, it is common practice to examine their type I and II error rates. Under the Basel backtesting procedure, the type I error rate is the probability of rejecting the VaR model when the model is correct, while the type II error rate is the probability of accepting the model when the model is incorrect.
The Basel backtesting procedure is designed for controlling the type I error rate (or α level), the probability of rejecting the VaR model when the model is correct. For this test, type I error rate is the probability that number of exceptions, out of 250 daily observations, is greater than or equal to 10, when the probability of an exception is p = 0.01. The type I error rate of the test is 0.0003 (or α = 0.03%).
Although the test establishes a very conservative threshold for controlling the type I error rate, it is not designed to control the type II error rate. Therefore, it does not control the probability of accepting the VaR model when the model is incorrect.
References: