15.1. Metrics for Model evaluation

Methods commonly used to evaluate model performance, include:

  • Mean absolute error (MAE)

(15.1)\[\mathrm{MAE}=\frac{1}{N} \sum_{i=1}^{N}\left|y_{i}-\hat{y}_{i}\right|\]

where \(N\) is number of observations, \(y_i\) the actual expected output and \(\hat{y}_{i}\) the model’s prediction (same notations below if not indicated otherwise).

  • Mean bias error (MBE)

(15.2)\[\mathrm{MBE}=\frac{1}{N} \sum_{i=1}^{N}\left(y_{i}-\hat{y}_{i}\right)\]

  • Mean square error (MSE)

(15.3)\[\mathrm{MSE}=\frac{1}{N} \sum_{i=1}^{N}\left(y_{i}-\hat{y}_{i}\right)^{2}\]

  • Root mean square error (RMSE)

(15.4)\[\mathrm{RMSE}=\sqrt{\frac{1}{N} \sum_{i=1}^{N}\left(y_{i}-\hat{y}_{i}\right)^{2}}\]

  • Coefficient of determination (\(R^2\))

(15.5)\[ \begin{align}\begin{aligned}R^{2}= 1-\frac{\mathrm{MSE}(\text { model })} {\mathrm{MSE}(\text { baseline })}\\\mathrm{MSE}(\text { baseline })= \frac{1}{N} \sum_{i=1}^{N}\left(y_{i}-\overline{y}\right)^{2}\end{aligned}\end{align} \]

where \(\overline{y}\) is mean of observed \(y_i\).

Combined with plots (e.g. scatter, time series) allows identification of periods when a model performs well/poorly relative to observations. It should be remembered that both the model (e.g. parameters, forcing data) and the evaluation observations have errors.

Tip

  1. Stuck? the help and FAQ pages are useful places to start.

  2. Please report workshop manual issues at GitHub Issues. Go from the page with the problem - an automatical link will be inserted. Thanks.