Model Evaluation
After obtaining base forecasts or coherent reconciled forecasts, you can use evaluate the forecasting accuracy
through pyhts.Hierarchy.accuracy_base()
, pyhts.Hierarchy.accuracy()
, or pyhts.TemporalHierarchy.accuracy()
(for temporal hierarchies).
- Hierarchy.accuracy_base(real, pred, hist=None, levels=None, measure=None)
Calculate the forecast accuracy of base forecasts.
- Parameters
real – real future observations, array-like of shape (forecast_horizon, m)
pred – forecast values, array-like of shape (forecast_horizon, n)
levels (
Union
[str
,Iterable
[str
],None
]) – which levels. None means all levels.hist – historical time series.
measure (
Optional
[List
[str
]]) – list of measures, e.g., [‘mase’], [‘mse’, ‘mase’].
- Return type
DataFrame
- Returns
forecast accuracy of base forecasts.
- Hierarchy.accuracy(real, pred, hist=None, levels=None, measure=None)
Calculate the forecast accuracy.
- Parameters
real – real future observations, array-like of shape (forecast_horizon, m)
pred – forecast values, array-like of shape (forecast_horizon, m)
levels (
Union
[str
,Iterable
[str
],None
]) – which levels, None means all levels.hist – historical time series.
measure (
Optional
[List
[str
]]) – list of measures, e.g., [‘mase’], [‘mse’, ‘mase’].
- Returns
forecast accuracy of reconciled forecasts.
- TemporalHierarchy.accuracy(real, pred, hist=None, measure=None)
function to compute forecast accuracy
- Parameters
real (
ndarray
) – univariate time series at forecast periodspred (
dict
) – dict containing forecasts, either reconciled or basehist (
Optional
[ndarray
]) – univariate historical time seriesmeasure – measures
- Returns
Assuming ht
is a defined hierarchy, model
is a fitted HFModel
, test
is real observations in the forecasting horizon. train
is the history bottom time series.
>>> forecasts = model.predict(h=12)
>>> base_forecasts = model.generate_base_forecasts(h=12)
>>> ht.accuracy_base(test, base_forecasts, hist=train, levels=[0], measures=['mase', 'rmse'])
>>> ht.accuracy(test, forecasts, hist=train, levels=[0], measures=['mase', 'rmse'])
Supported forecasting measurements:
- pyhts.mase(y_true, y_pred, history, m)
- pyhts.rmse(y_true, y_pred, history, m)
- pyhts.mse(y_true, y_pred, history, m)
- pyhts.mape(y_true, y_pred, history, m)
- pyhts.mae(y_true, y_pred, history, m)
- pyhts.smape(y_true, y_pred, history, m)