Home / Summaries / Course literature / model-value-nse

Models and model evaluation - Scaefli & Gupta 2007

Q: What type of model will give you a good and what model type a bad NSE value? Why? Is this good or bad?

In the case of strongly seasonal time series, a model that only explains the seasonality but fails to reproduce any smaller time scale fluctuations will report a good NSE value; for predictions at the daily time step, this (high) value will be misleading. In contrast, if the model is intended to simulate the fluctuations around a relatively constant mean value, it can only achieve high NSE values if it explains the small time-scale fluctuations.

Q: What is the main conclusion of this paper in relation to interpretation of model output and performance/What does Gupta pledge for?

For hydrologic case studies, it may be difficult–if not impossible–to establish a general and widely applicable benchmark model. However, it may be possible to at least decide on benchmark models that ‘speak’ to the modellers or the end-users, i.e. benchmark models that impose performance constraints that are readily interpretable for a given context. For efficient communication, the benchmark should fulfill the basic requirement that every hydrologist can immediately understand its explanatory power for the given case study and, therefore, appreciate how much better the actual hydrologic model is.

4 important questions on Models and model evaluation - Scaefli & Gupta 2007

Explain how the ASPB (adjusted smoothed precip benchmark) is calculated?

A further important characteristic of catchments is
to filter (smooth) the rainfall to remove higher frequency
variability. We can, therefore, further add a
simple dispersion process (a moving average) to adjust
the smoothness of the scaled-down and translated precipitation
to match the smoothness of the observed
discharge. One simple way to choose the degree of
smoothness (the size of the moving-average window)
is so as to maximize the correlation between the
adjusted precipitation and the observed flow.

Give an example of high, as good as mean target variable and low NSE values?

an NSE value = 1·0 indicates perfect model performance
(the model perfectly simulates the target output),

an NSE value = 0 indicates that the model is, on
average, performing only as good as the use of the
mean target value as prediction,

and an NSE value
<0·0 indicates an altogether questionable choice of
model. We, therefore, prefer NSE values to be larger
than 0·0 and approaching 1·0.

What type of model will give you a good and what model type a bad NSE value? Why? Is this good or bad?

In the case of strongly seasonal time series, a model
that only explains the seasonality but fails to reproduce
any smaller time scale fluctuations will report
a good NSE value; for predictions at the daily time
step, this (high) value will be misleading.

In contrast,
if the model is intended to simulate the fluctuations
around a relatively constant mean value, it can
only achieve high NSE values if it explains the small time-scale fluctuations.

What is the main conclusion of this paper in relation to interpretation of model output and performance/What does Gupta pledge for?

For hydrologic case studies, it may
be difficult–if not impossible–to establish a general
and widely applicable benchmark model.

However, it may be possible
to at least decide on benchmark models that
‘speak’ to the modellers or the end-users, i.e. benchmark
models that impose performance constraints
that are readily interpretable for a given context.

For efficient communication,
the benchmark should fulfill the basic requirement
that every hydrologist can immediately understand
its explanatory power for the given case study and,
therefore, appreciate how much better the actual
hydrologic model is.

The question on the page originate from the summary of the following study material:

Course literature

View summary