I get an error when trying to finetune a model: Error(s) in loading state_dict for VitsModel

[sabexzero]

I tried using the checkpoints that you posted in similar tickets, but I still fail
Model: RU, this is a mistake with any speaker

2024-10-29 17:42:46.969285: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-10-29 17:42:47.000764: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-10-29 17:42:47.010368: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-10-29 17:42:47.031678: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-10-29 17:42:48.595751: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
DEBUG:piper_train:Namespace(dataset_dir='/content/drive/MyDrive/colab/piper/VKuksa', checkpoint_epochs=5, quality='high', resume_from_single_speaker_checkpoint=None, logger=True, enable_checkpointing=True, default_root_dir=None, gradient_clip_val=None, gradient_clip_algorithm=None, num_nodes=1, num_processes=None, devices='1', gpus=None, auto_select_gpus=False, tpu_cores=None, ipus=None, enable_progress_bar=True, overfit_batches=0.0, track_grad_norm=-1, check_val_every_n_epoch=1, fast_dev_run=False, accumulate_grad_batches=None, max_epochs=10000, min_epochs=None, max_steps=-1, min_steps=None, max_time=None, limit_train_batches=None, limit_val_batches=None, limit_test_batches=None, limit_predict_batches=None, val_check_interval=None, log_every_n_steps=1000, accelerator='gpu', strategy=None, sync_batchnorm=False, precision=32, enable_model_summary=True, weights_save_path=None, num_sanity_val_steps=2, resume_from_checkpoint='/content/pretrained.ckpt', profiler=None, benchmark=None, deterministic=None, reload_dataloaders_every_n_epochs=0, auto_lr_find=False, replace_sampler_ddp=True, detect_anomaly=False, auto_scale_batch_size=False, plugins=None, amp_backend='native', amp_level=None, move_metrics_to_cpu=False, multiple_trainloader_mode='max_size_cycle', batch_size=12, validation_split=0.0, num_test_examples=0, max_phoneme_ids=None, hidden_channels=192, inter_channels=192, filter_channels=768, n_layers=6, n_heads=2, seed=1234, num_ckpt=0, save_last=True)
/usr/local/lib/python3.10/dist-packages/pytorch_lightning/trainer/connectors/checkpoint_connector.py:52: LightningDeprecationWarning: Setting `Trainer(resume_from_checkpoint=)` is deprecated in v1.5 and will be removed in v1.7. Please pass `Trainer.fit(ckpt_path=)` directly instead.
  rank_zero_deprecation(
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
DEBUG:piper_train:Checkpoints will be saved every 5 epoch(s)
DEBUG:piper_train:0 Checkpoints will be saved
DEBUG:vits.dataset:Loading dataset: /content/drive/MyDrive/colab/piper/VKuksa/dataset.jsonl
/usr/local/lib/python3.10/dist-packages/pytorch_lightning/trainer/trainer.py:731: LightningDeprecationWarning: `trainer.resume_from_checkpoint` is deprecated in v1.5 and will be removed in v2.0. Specify the fit checkpoint path with `trainer.fit(ckpt_path=)` instead.
  ckpt_path = ckpt_path or self.resume_from_checkpoint
Missing logger folder: /content/drive/MyDrive/colab/piper/VKuksa/lightning_logs
Restoring states from the checkpoint path at /content/pretrained.ckpt
DEBUG:fsspec.local:open file: /content/pretrained.ckpt
Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/content/piper/src/python/piper_train/__main__.py", line 173, in <module>
    main()
  File "/content/piper/src/python/piper_train/__main__.py", line 150, in main
    trainer.fit(model)
  File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/trainer/trainer.py", line 696, in fit
    self._call_and_handle_interrupt(
  File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/trainer/trainer.py", line 650, in _call_and_handle_interrupt
    return trainer_fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/trainer/trainer.py", line 735, in _fit_impl
    results = self._run(model, ckpt_path=self.ckpt_path)
  File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/trainer/trainer.py", line 1110, in _run
    self._restore_modules_and_callbacks(ckpt_path)
  File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/trainer/trainer.py", line 1065, in _restore_modules_and_callbacks
    self._checkpoint_connector.restore_model()
  File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/trainer/connectors/checkpoint_connector.py", line 182, in restore_model
    self.trainer.strategy.load_model_state_dict(self._loaded_checkpoint)
  File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/strategies/strategy.py", line 343, in load_model_state_dict
    self.lightning_module.load_state_dict(checkpoint["state_dict"])
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1667, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for VitsModel:
	Missing key(s) in state_dict: "model_g.dec.ups.3.bias", "model_g.dec.ups.3.weight_g", "model_g.dec.ups.3.weight_v", "model_g.dec.resblocks.0.convs1.0.bias", "model_g.dec.resblocks.0.convs1.0.weight_g", "model_g.dec.resblocks.0.convs1.0.weight_v", "model_g.dec.resblocks.0.convs1.1.bias", "model_g.dec.resblocks.0.convs1.1.weight_g", "model_g.dec.resblocks.0.convs1.1.weight_v", "model_g.dec.resblocks.0.convs1.2.bias", "model_g.dec.resblocks.0.convs1.2.weight_g", "model_g.dec.resblocks.0.convs1.2.weight_v", "model_g.dec.resblocks.0.convs2.0.bias", "model_g.dec.resblocks.0.convs2.0.weight_g", "model_g.dec.resblocks.0.convs2.0.weight_v", "model_g.dec.resblocks.0.convs2.1.bias", "model_g.dec.resblocks.0.convs2.1.weight_g", "model_g.dec.resblocks.0.convs2.1.weight_v", "model_g.dec.resblocks.0.convs2.2.bias", "model_g.dec.resblocks.0.convs2.2.weight_g", "model_g.dec.resblocks.0.convs2.2.weight_v", "model_g.dec.resblocks.1.convs1.0.bias", "model_g.dec.resblocks.1.convs1.0.weight_g", "model_g.dec.resblocks.1.convs1.0.weight_v", "model_g.dec.resblocks.1.convs1.1.bias", "model_g.dec.resblocks.1.convs1.1.weight_g", "model_g.dec.resblocks.1.convs1.1.weight_v", "model_g.dec.resblocks.1.convs1.2.bias", "model_g.dec.resblocks.1.convs1.2.weight_g", "model_g.dec.resblocks.1.convs1.2.weight_v", "model_g.dec.resblocks.1.convs2.0.bias", "model_g.dec.resblocks.1.convs2.0.weight_g", "model_g.dec.resblocks.1.convs2.0.weight_v", "model_g.dec.resblocks.1.convs2.1.bias", "model_g.dec.resblocks.1.convs2.1.weight_g", "model_g.dec.resblocks.1.convs2.1.weight_v", "model_g.dec.resblocks.1.convs2.2.bias", "model_g.dec.resblocks.1.convs2.2.weight_g", "model_g.dec.resblocks.1.convs2.2.weight_v", "model_g.dec.resblocks.2.convs1.0.bias", "model_g.dec.resblocks.2.convs1.0.weight_g", "model_g.dec.resblocks.2.convs1.0.weight_v", "model_g.dec.resblocks.2.convs1.1.bias", "model_g.dec.resblocks.2.convs1.1.weight_g", "model_g.dec.resblocks.2.convs1.1.weight_v", "model_g.dec.resblocks.2.convs1.2.bias", "model_g.dec.resblocks.2.convs1.2.weight_g", "model_g.dec.resblocks.2.convs1.2.weight_v", "model_g.dec.resblocks.2.convs2.0.bias", "model_g.dec.resblocks.2.convs2.0.weight_g", "model_g.dec.resblocks.2.convs2.0.weight_v", "model_g.dec.resblocks.2.convs2.1.bias", "model_g.dec.resblocks.2.convs2.1.weight_g", "model_g.dec.resblocks.2.convs2.1.weight_v", "model_g.dec.resblocks.2.convs2.2.bias", "model_g.dec.resblocks.2.convs2.2.weight_g", "model_g.dec.resblocks.2.convs2.2.weight_v", "model_g.dec.resblocks.3.convs1.0.bias", "model_g.dec.resblocks.3.convs1.0.weight_g", "model_g.dec.resblocks.3.convs1.0.weight_v", "model_g.dec.resblocks.3.convs1.1.bias", "model_g.dec.resblocks.3.convs1.1.weight_g", "model_g.dec.resblocks.3.convs1.1.weight_v", "model_g.dec.resblocks.3.convs1.2.bias", "model_g.dec.resblocks.3.convs1.2.weight_g", "model_g.dec.resblocks.3.convs1.2.weight_v", "model_g.dec.resblocks.3.convs2.0.bias", "model_g.dec.resblocks.3.convs2.0.weight_g", "model_g.dec.resblocks.3.convs2.0.weight_v", "model_g.dec.resblocks.3.convs2.1.bias", "model_g.dec.resblocks.3.convs2.1.weight_g", "model_g.dec.resblocks.3.convs2.1.weight_v", "model_g.dec.resblocks.3.convs2.2.bias", "model_g.dec.resblocks.3.convs2.2.weight_g", "model_g.dec.resblocks.3.convs2.2.weight_v", "model_g.dec.resblocks.4.convs1.0.bias", "model_g.dec.resblocks.4.convs1.0.weight_g", "model_g.dec.resblocks.4.convs1.0.weight_v", "model_g.dec.resblocks.4.convs1.1.bias", "model_g.dec.resblocks.4.convs1.1.weight_g", "model_g.dec.resblocks.4.convs1.1.weight_v", "model_g.dec.resblocks.4.convs1.2.bias", "model_g.dec.resblocks.4.convs1.2.weight_g", "model_g.dec.resblocks.4.convs1.2.weight_v", "model_g.dec.resblocks.4.convs2.0.bias", "model_g.dec.resblocks.4.convs2.0.weight_g", "model_g.dec.resblocks.4.convs2.0.weight_v", "model_g.dec.resblocks.4.convs2.1.bias", "model_g.dec.resblocks.4.convs2.1.weight_g", "model_g.dec.resblocks.4.convs2.1.weight_v", "model_g.dec.resblocks.4.convs2.2.bias", "model_g.dec.resblocks.4.convs2.2.weight_g", "model_g.dec.resblocks.4.convs2.2.weight_v", "model_g.dec.resblocks.5.convs1.0.bias", "model_g.dec.resblocks.5.convs1.0.weight_g", "model_g.dec.resblocks.5.convs1.0.weight_v", "model_g.dec.resblocks.5.convs1.1.bias", "model_g.dec.resblocks.5.convs1.1.weight_g", "model_g.dec.resblocks.5.convs1.1.weight_v", "model_g.dec.resblocks.5.convs1.2.bias", "model_g.dec.resblocks.5.convs1.2.weight_g", "model_g.dec.resblocks.5.convs1.2.weight_v", "model_g.dec.resblocks.5.convs2.0.bias", "model_g.dec.resblocks.5.convs2.0.weight_g", "model_g.dec.resblocks.5.convs2.0.weight_v", "model_g.dec.resblocks.5.convs2.1.bias", "model_g.dec.resblocks.5.convs2.1.weight_g", "model_g.dec.resblocks.5.convs2.1.weight_v", "model_g.dec.resblocks.5.convs2.2.bias", "model_g.dec.resblocks.5.convs2.2.weight_g", "model_g.dec.resblocks.5.convs2.2.weight_v", "model_g.dec.resblocks.6.convs1.0.bias", "model_g.dec.resblocks.6.convs1.0.weight_g", "model_g.dec.resblocks.6.convs1.0.weight_v", "model_g.dec.resblocks.6.convs1.1.bias", "model_g.dec.resblocks.6.convs1.1.weight_g", "model_g.dec.resblocks.6.convs1.1.weight_v", "model_g.dec.resblocks.6.convs1.2.bias", "model_g.dec.resblocks.6.convs1.2.weight_g", "model_g.dec.resblocks.6.convs1.2.weight_v", "model_g.dec.resblocks.6.convs2.0.bias", "model_g.dec.resblocks.6.convs2.0.weight_g", "model_g.dec.resblocks.6.convs2.0.weight_v", "model_g.dec.resblocks.6.convs2.1.bias", "model_g.dec.resblocks.6.convs2.1.weight_g", "model_g.dec.resblocks.6.convs2.1.weight_v", "model_g.dec.resblocks.6.convs2.2.bias", "model_g.dec.resblocks.6.convs2.2.weight_g", "model_g.dec.resblocks.6.convs2.2.weight_v", "model_g.dec.resblocks.7.convs1.0.bias", "model_g.dec.resblocks.7.convs1.0.weight_g", "model_g.dec.resblocks.7.convs1.0.weight_v", "model_g.dec.resblocks.7.convs1.1.bias", "model_g.dec.resblocks.7.convs1.1.weight_g", "model_g.dec.resblocks.7.convs1.1.weight_v", "model_g.dec.resblocks.7.convs1.2.bias", "model_g.dec.resblocks.7.convs1.2.weight_g", "model_g.dec.resblocks.7.convs1.2.weight_v", "model_g.dec.resblocks.7.convs2.0.bias", "model_g.dec.resblocks.7.convs2.0.weight_g", "model_g.dec.resblocks.7.convs2.0.weight_v", "model_g.dec.resblocks.7.convs2.1.bias", "model_g.dec.resblocks.7.convs2.1.weight_g", "model_g.dec.resblocks.7.convs2.1.weight_v", "model_g.dec.resblocks.7.convs2.2.bias", "model_g.dec.resblocks.7.convs2.2.weight_g", "model_g.dec.resblocks.7.convs2.2.weight_v", "model_g.dec.resblocks.8.convs1.0.bias", "model_g.dec.resblocks.8.convs1.0.weight_g", "model_g.dec.resblocks.8.convs1.0.weight_v", "model_g.dec.resblocks.8.convs1.1.bias", "model_g.dec.resblocks.8.convs1.1.weight_g", "model_g.dec.resblocks.8.convs1.1.weight_v", "model_g.dec.resblocks.8.convs1.2.bias", "model_g.dec.resblocks.8.convs1.2.weight_g", "model_g.dec.resblocks.8.convs1.2.weight_v", "model_g.dec.resblocks.8.convs2.0.bias", "model_g.dec.resblocks.8.convs2.0.weight_g", "model_g.dec.resblocks.8.convs2.0.weight_v", "model_g.dec.resblocks.8.convs2.1.bias", "model_g.dec.resblocks.8.convs2.1.weight_g", "model_g.dec.resblocks.8.convs2.1.weight_v", "model_g.dec.resblocks.8.convs2.2.bias", "model_g.dec.resblocks.8.convs2.2.weight_g", "model_g.dec.resblocks.8.convs2.2.weight_v", "model_g.dec.resblocks.9.convs1.0.bias", "model_g.dec.resblocks.9.convs1.0.weight_g", "model_g.dec.resblocks.9.convs1.0.weight_v", "model_g.dec.resblocks.9.convs1.1.bias", "model_g.dec.resblocks.9.convs1.1.weight_g", "model_g.dec.resblocks.9.convs1.1.weight_v", "model_g.dec.resblocks.9.convs1.2.bias", "model_g.dec.resblocks.9.convs1.2.weight_g", "model_g.dec.resblocks.9.convs1.2.weight_v", "model_g.dec.resblocks.9.convs2.0.bias", "model_g.dec.resblocks.9.convs2.0.weight_g", "model_g.dec.resblocks.9.convs2.0.weight_v", "model_g.dec.resblocks.9.convs2.1.bias", "model_g.dec.resblocks.9.convs2.1.weight_g", "model_g.dec.resblocks.9.convs2.1.weight_v", "model_g.dec.resblocks.9.convs2.2.bias", "model_g.dec.resblocks.9.convs2.2.weight_g", "model_g.dec.resblocks.9.convs2.2.weight_v", "model_g.dec.resblocks.10.convs1.0.bias", "model_g.dec.resblocks.10.convs1.0.weight_g", "model_g.dec.resblocks.10.convs1.0.weight_v", "model_g.dec.resblocks.10.convs1.1.bias", "model_g.dec.resblocks.10.convs1.1.weight_g", "model_g.dec.resblocks.10.convs1.1.weight_v", "model_g.dec.resblocks.10.convs1.2.bias", "model_g.dec.resblocks.10.convs1.2.weight_g", "model_g.dec.resblocks.10.convs1.2.weight_v", "model_g.dec.resblocks.10.convs2.0.bias", "model_g.dec.resblocks.10.convs2.0.weight_g", "model_g.dec.resblocks.10.convs2.0.weight_v", "model_g.dec.resblocks.10.convs2.1.bias", "model_g.dec.resblocks.10.convs2.1.weight_g", "model_g.dec.resblocks.10.convs2.1.weight_v", "model_g.dec.resblocks.10.convs2.2.bias", "model_g.dec.resblocks.10.convs2.2.weight_g", "model_g.dec.resblocks.10.convs2.2.weight_v", "model_g.dec.resblocks.11.convs1.0.bias", "model_g.dec.resblocks.11.convs1.0.weight_g", "model_g.dec.resblocks.11.convs1.0.weight_v", "model_g.dec.resblocks.11.convs1.1.bias", "model_g.dec.resblocks.11.convs1.1.weight_g", "model_g.dec.resblocks.11.convs1.1.weight_v", "model_g.dec.resblocks.11.convs1.2.bias", "model_g.dec.resblocks.11.convs1.2.weight_g", "model_g.dec.resblocks.11.convs1.2.weight_v", "model_g.dec.resblocks.11.convs2.0.bias", "model_g.dec.resblocks.11.convs2.0.weight_g", "model_g.dec.resblocks.11.convs2.0.weight_v", "model_g.dec.resblocks.11.convs2.1.bias", "model_g.dec.resblocks.11.convs2.1.weight_g", "model_g.dec.resblocks.11.convs2.1.weight_v", "model_g.dec.resblocks.11.convs2.2.bias", "model_g.dec.resblocks.11.convs2.2.weight_g", "model_g.dec.resblocks.11.convs2.2.weight_v". 
	Unexpected key(s) in state_dict: "model_g.dec.resblocks.0.convs.0.bias", "model_g.dec.resblocks.0.convs.0.weight_g", "model_g.dec.resblocks.0.convs.0.weight_v", "model_g.dec.resblocks.0.convs.1.bias", "model_g.dec.resblocks.0.convs.1.weight_g", "model_g.dec.resblocks.0.convs.1.weight_v", "model_g.dec.resblocks.1.convs.0.bias", "model_g.dec.resblocks.1.convs.0.weight_g", "model_g.dec.resblocks.1.convs.0.weight_v", "model_g.dec.resblocks.1.convs.1.bias", "model_g.dec.resblocks.1.convs.1.weight_g", "model_g.dec.resblocks.1.convs.1.weight_v", "model_g.dec.resblocks.2.convs.0.bias", "model_g.dec.resblocks.2.convs.0.weight_g", "model_g.dec.resblocks.2.convs.0.weight_v", "model_g.dec.resblocks.2.convs.1.bias", "model_g.dec.resblocks.2.convs.1.weight_g", "model_g.dec.resblocks.2.convs.1.weight_v", "model_g.dec.resblocks.3.convs.0.bias", "model_g.dec.resblocks.3.convs.0.weight_g", "model_g.dec.resblocks.3.convs.0.weight_v", "model_g.dec.resblocks.3.convs.1.bias", "model_g.dec.resblocks.3.convs.1.weight_g", "model_g.dec.resblocks.3.convs.1.weight_v", "model_g.dec.resblocks.4.convs.0.bias", "model_g.dec.resblocks.4.convs.0.weight_g", "model_g.dec.resblocks.4.convs.0.weight_v", "model_g.dec.resblocks.4.convs.1.bias", "model_g.dec.resblocks.4.convs.1.weight_g", "model_g.dec.resblocks.4.convs.1.weight_v", "model_g.dec.resblocks.5.convs.0.bias", "model_g.dec.resblocks.5.convs.0.weight_g", "model_g.dec.resblocks.5.convs.0.weight_v", "model_g.dec.resblocks.5.convs.1.bias", "model_g.dec.resblocks.5.convs.1.weight_g", "model_g.dec.resblocks.5.convs.1.weight_v", "model_g.dec.resblocks.6.convs.0.bias", "model_g.dec.resblocks.6.convs.0.weight_g", "model_g.dec.resblocks.6.convs.0.weight_v", "model_g.dec.resblocks.6.convs.1.bias", "model_g.dec.resblocks.6.convs.1.weight_g", "model_g.dec.resblocks.6.convs.1.weight_v", "model_g.dec.resblocks.7.convs.0.bias", "model_g.dec.resblocks.7.convs.0.weight_g", "model_g.dec.resblocks.7.convs.0.weight_v", "model_g.dec.resblocks.7.convs.1.bias", "model_g.dec.resblocks.7.convs.1.weight_g", "model_g.dec.resblocks.7.convs.1.weight_v", "model_g.dec.resblocks.8.convs.0.bias", "model_g.dec.resblocks.8.convs.0.weight_g", "model_g.dec.resblocks.8.convs.0.weight_v", "model_g.dec.resblocks.8.convs.1.bias", "model_g.dec.resblocks.8.convs.1.weight_g", "model_g.dec.resblocks.8.convs.1.weight_v". 
	size mismatch for model_g.dec.conv_pre.weight: copying a param with shape torch.Size([256, 192, 7]) from checkpoint, the shape in current model is torch.Size([512, 192, 7]).
	size mismatch for model_g.dec.conv_pre.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([512]).
	size mismatch for model_g.dec.ups.0.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
	size mismatch for model_g.dec.ups.0.weight_g: copying a param with shape torch.Size([256, 1, 1]) from checkpoint, the shape in current model is torch.Size([512, 1, 1]).
	size mismatch for model_g.dec.ups.0.weight_v: copying a param with shape torch.Size([256, 128, 16]) from checkpoint, the shape in current model is torch.Size([512, 256, 16]).
	size mismatch for model_g.dec.ups.1.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
	size mismatch for model_g.dec.ups.1.weight_g: copying a param with shape torch.Size([128, 1, 1]) from checkpoint, the shape in current model is torch.Size([256, 1, 1]).
	size mismatch for model_g.dec.ups.1.weight_v: copying a param with shape torch.Size([128, 64, 16]) from checkpoint, the shape in current model is torch.Size([256, 128, 16]).
	size mismatch for model_g.dec.ups.2.bias: copying a param with shape torch.Size([32]) from checkpoint, the shape in current model is torch.Size([64]).
	size mismatch for model_g.dec.ups.2.weight_g: copying a param with shape torch.Size([64, 1, 1]) from checkpoint, the shape in current model is torch.Size([128, 1, 1]).
	size mismatch for model_g.dec.ups.2.weight_v: copying a param with shape torch.Size([64, 32, 8]) from checkpoint, the shape in current model is torch.Size([128, 64, 4]).

[dokempf]

Are you trying to finetune a model of quality high without passing --quality high to the training? This solved this one for me (only to be stuck on the next thing, currently)