How to do validation with gluonts.torch.model.deepar.DeepAREstimator

[toby-coleman]

I'm using the Torch implementation of DeepAREstimator, and am having problems with overfitting. I would like to be able to use a validation set when training, but I cannot see how this works.

When I train the model with this code:

estimator = DeepAREstimator(
    freq="30min",
    prediction_length=48,
    context_length=24,
    scaling=False,
    num_feat_dynamic_real=1,
    trainer_kwargs={"max_epochs": 10},
    dropout_rate=0.2,
    num_batches_per_epoch=len(df[df.time_stamp < pd.Timestamp(dtest)]) // 32,
    batch_size=32,
)

predictor = estimator.train(ds_train)

I get the following output:

You defined a `validation_step` but have no `val_dataloader`. Skipping val loop.

I can't see any documentation on how to set val_dataloader.

[HariniS2506]

Hi, did you find out how to do this?

[toby-coleman]

@HariniS2506 no I didn't. I ended up using a different library/modelling approach because it was difficult to control over-fitting with the DeepAREstimator.

[SohnBene]

This works work for me:

trainingData, __ = split(ds, date=tuneUntil)
validationData, __ = split(ds, date=valUntil)

estimator = DeepAREstimator(
    freq="30min",
    prediction_length=48,
    context_length=24,
    scaling=False,
    num_feat_dynamic_real=1,
    trainer_kwargs={"max_epochs": 10},
    dropout_rate=0.2,
    num_batches_per_epoch=len(df[df.time_stamp < pd.Timestamp(dtest)]) // 32,
    batch_size=32,
)

trained_model = estimator.train(training_data = ds_train, validation_data = ds_val)