13.1. Metrics for Model evaluation

Methods commonly used to evaluate model performance, include:

  • Mean absolute error (MAE)

(13.1)\[\mathrm{MAE}=\frac{1}{N} \sum_{i=1}^{N}\left|y_{i}-\hat{y}_{i}\right|\]

where \(N\) is number of observations, \(y_i\) the actual expected output and \(\hat{y}_{i}\) the model’s prediction (same notations below if not indicated otherwise).

  • Mean bias error (MBE)

(13.2)\[\mathrm{MBE}=\frac{1}{N} \sum_{i=1}^{N}\left(y_{i}-\hat{y}_{i}\right)\]

  • Mean square error (MSE)

(13.3)\[\mathrm{MSE}=\frac{1}{N} \sum_{i=1}^{N}\left(y_{i}-\hat{y}_{i}\right)^{2}\]

  • Root mean square error (RMSE)

(13.4)\[\mathrm{RMSE}=\sqrt{\frac{1}{N} \sum_{i=1}^{N}\left(y_{i}-\hat{y}_{i}\right)^{2}}\]

  • Coefficient of determination (\(R^2\))

(13.5)\[ \begin{align}\begin{aligned}R^{2}= 1-\frac{\mathrm{MSE}(\text { model })} {\mathrm{MSE}(\text { baseline })}\\\mathrm{MSE}(\text { baseline })= \frac{1}{N} \sum_{i=1}^{N}\left(y_{i}-\overline{y}\right)^{2}\end{aligned}\end{align} \]

where \(\overline{y}\) is mean of observed \(y_i\).

Combined with plots (e.g. scatter, time series) allows identification of periods when a model performs well/poorly relative to observations. It should be remembered that both the model (e.g. parameters, forcing data) and the evaluation observations have errors.

Tip

  1. Stuck? the help page is a useful page to start.

  2. Please report workshop manual issues at GitHub Issues. Please go from the page with problem as an automatical link will be inserted. Thanks.