A hydrological model skill score and revised R-squared

Abstract

Despite the advances in methods of statistical andmathematical modeling, there is considerable lack of focus on improving how to judgemodels’ quality. Coefficient of determination (R2) is arguably the most widely applied ‘goodness-of-fit’ metric inmodelling and prediction of environmental systems. However, known issues of R2 are that it: (i) can be low and high for an accurate and imperfect model, respectively; (ii) yields the same value when we regress observed on modelled series and vice versa; and (iii) does not quantify a model’s bias. A new model skill score E and revised R-squared (RRS) are presented to combine correlation, bias measure and capacity to capture variability. Differences between E and RRS lie in the forms of correlation and variabilitymeasure used for eachmetric. Acceptability of E and RRS was demonstrated through comparison of results from a large number of hydrological simulations. By applying E and RRS, the modeller can diagnostically identify and expose systematic issues behind model optimizations based on othDespite the advances in methods of statistical andmathematical modeling, there is considerable lack of focus on improving how to judgemodels’ quality. Coefficient of determination (R2) is arguably the most widely applied ‘goodness-of-fit’ metric inmodelling and prediction of environmental systems. However, known issues of R2 are that it: (i) can be low and high for an accurate and imperfect model, respectively; (ii) yields the same value when we regress observed on modelled series and vice versa; and (iii) does not quantify a model’s bias. A new model skill score E and revised R-squared (RRS) are presented to combine correlation, bias measure and capacity to capture variability. Differences between E and RRS lie in the forms of correlation and variabilitymeasure used for eachmetric. Acceptability of E and RRS was demonstrated through comparison of results from a large number of hydrological simulations. By applying E and RRS, the modeller can diagnostically identify and expose systematic issues behind model optimizations based on oDespite the advances in methods of statistical andmathematical modeling, there is considerable lack of focus on improving how to judgemodels’ quality. Coefficient of determination (R2) is arguably the most widely applied ‘goodness-of-fit’ metric inmodelling and prediction of environmental systems. However, known issues of R2 are that it: (i) can be low and high for an accurate and imperfect model, respectively; (ii) yields the same value when we regress observed on modelled series and vice versa; and (iii) does not quantify a model’s bias. A new model skill score E and revised R-squared (RRS) are presented to combine correlation, bias measure and capacity to capture variability. Differences between E and RRS lie in the forms of correlation and variabilitymeasure used for eachmetric. Acceptability of E and RRS was demonstrated through comparison of results from a large number of hydrological simulations. By applying E and RRS, the modeller can diagnostically identify and expose systematic issues behind model optimizations based on other ‘goodness-of-fits’ such as Nash–Sutcliffe efficiency (NSE) and mean squared error. Unlike NSE, which varies from ∞ to 1, E and RRS occur over the range 0–1. MATLAB codes for computing E and RRS are provided.ther ‘goodness-of-fits’ such as Nash–Sutcliffe efficiency (NSE) and mean squared error. Unlike NSE, which varies from ∞ to 1, E and RRS occur over the range 0–1. MATLAB codes for computing E and RRS are provided.er ‘goodness-of-fits’ such as Nash–Sutcliffe efficiency (NSE) and mean squared error. Unlike NSE, which varies from ∞ to 1, E and RRS occur over the range 0–1. MATLAB codes for computing E and RRS are provided.