a problem of ERROR: RuntimeError: unexpected EOF, expected 39359 more bytes. The file might be corrupted

[puallee]

there was a problem when I tried to start processing of the Nii image or dicom image

Runlist: ['C:/Users/V/Desktop/imagesTr/004_0000.nii.gz'].
1 volumes detected.

Will save outputs in C:/Users/V/Desktop/pred.

Click start processing to start.

ERROR: RuntimeError: unexpected EOF, expected 39359 more bytes. The file might be corrupted.

Aborting processing! This was most likely an error regarding the data input or missing weights before doing pip install.

Feel free to create an issue in our GitHub.

Closing worker thread...

Done.

Runlist: ['C:/Users/V/Desktop/imagesTr/059_0000.nii.gz'].
1 volumes detected.

Will save outputs in C:/Users/V/Desktop/pred.

Click start processing to start.

ERROR: RuntimeError: unexpected EOF, expected 39359 more bytes. The file might be corrupted.

Aborting processing! This was most likely an error regarding the data input or missing weights before doing pip install.

Feel free to create an issue in our GitHub.

Closing worker thread...

Done.

Please load a folder or image before starting processing.

Please load a folder or image before starting processing.

Please load a folder or image before starting processing.

Runlist: ['G:/DICOM/dicom/dicom/1.2.276.0.7230010.3.1.4.3507096321.30414.1535700143.988584.dcm'].
1 volumes detected.

Will save outputs in C:/Users/V/Desktop/pred.

Click start processing to start.

ERROR: RuntimeError: unexpected EOF, expected 39359 more bytes. The file might be corrupted.

Aborting processing! This was most likely an error regarding the data input or missing weights before doing pip install.

Feel free to create an issue in our GitHub.

Closing worker thread...

Done.

[dscarmo]

Hi! Can you please answer these questions and I can try and help you.

Can you share what is the full output in the terminal where you ran the "medpseg" command?

Have you downloaded the network weights? They might be corrupted, I suggest downloading them again.

The DICOM/Nift files open in software such as ITK-Snap or 3D Slicer without errors?

[puallee]

Yes， the full output in the terminal is： Click start processing to start.

ERROR: RuntimeError: unexpected EOF, expected 39359 more bytes. The file might be corrupted.

Aborting processing! This was most likely an error regarding the data input or missing weights before doing pip install.

Feel free to create an issue in our GitHub.

ERROR: RuntimeError: unexpected EOF, expected 39359 more bytes. The file might be corrupted.

Aborting processing! This was most likely an error regarding the data input or missing weights before doing pip install.

Feel free to create an issue in our GitHub.

Closing worker thread...

Done.

Closing worker thread...

[puallee]

I downloaded the network weights again， but I still have this problem.

[puallee]

yes, all CTs can open ITK-Snap or 3D Slicer without errors

[puallee]

PS D:\medpseg> medpseg
Could not find platform dependent libraries <exec_prefix>
Running MEDPSeg with GPU support. CUDA available? False.
C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\lightning_fabric\utilities\cloud_io.py:57: You are using torch.load with weights_only=False (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for weights_only will be flipped to True. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via torch.serialization.add_safe_globals. We recommend you start setting weights_only=True for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
WARNING: default expand_bifpn changed to upsample_sum after many ablations and literature support.
Using upsample for bifpn result expansion and upsample 2 expansion of convnext features
Traceback (most recent call last):
File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\medpseg\pipeline.py", line 118, in pipeline
poly_model = PolySegmentationPipeline(weight=poly_weight,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\medpseg\poly_pipeline.py", line 212, in init
self.model = PolySeg2DModule.load_from_checkpoint(weight, map_location="cpu").eval()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\pytorch_lightning\utilities\model_helpers.py", line 125, in wrapper
return self.method(cls, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\pytorch_lightning\core\module.py", line 1586, in load_from_checkpoint
loaded = _load_from_checkpoint(
^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\pytorch_lightning\core\saving.py", line 91, in _load_from_checkpoint
model = _load_state(cls, checkpoint, strict=strict, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\pytorch_lightning\core\saving.py", line 165, in _load_state
obj = instantiator(cls, _cls_kwargs) if instantiator else cls(cls_kwargs)
^^^^^^^^^^^^^^^^^^
File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\medpseg\poly_seg_2d_module.py", line 126, in init
self.model = MEDSeg(self.hparams.nin, self.hparams.seg_nout, apply_sigmoid=False, backbone=self.backbone, expand_bifpn=self.expand_bifpn, pretrained=not self.scratch,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\medpseg\architecture.py", line 64, in init
self.model = EfficientDetForSemanticSegmentation(num_classes=nout,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\medpseg\edet\modeling_efficientdet.py", line 224, in init
self.backbone_net = EfficientNet(self.backbone_compound_coef[self.compound_coef], load_weights)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\medpseg\edet\efficientdet\model.py", line 563, in init
model = EffNet.from_pretrained(f'efficientnet-b{compound_coef}')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\efficientnet_pytorch\model.py", line 378, in from_pretrained
load_pretrained_weights(model, model_name, weights_path=weights_path,
File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\efficientnet_pytorch\utils.py", line 602, in load_pretrained_weights
state_dict = model_zoo.load_url(url_map[model_name])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\hub.py", line 769, in load_state_dict_from_url
return torch.load(cached_file, map_location=map_location, weights_only=weights_only)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\serialization.py", line 1114, in load
return _legacy_load(
^^^^^^^^^^^^^
File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\serialization.py", line 1357, in _legacy_load
typed_storage._untyped_storage._set_from_file(
RuntimeError: unexpected EOF, expected 39359 more bytes. The file might be corrupted.

[dscarmo]

Thanks for the outputs, It looks like you are not using a separate environment for medpseg, I recommend you create a new environment and install the requirements listed in requirements.txt. If the model weights are not corrupted in download, I suspect the model weights are not loading correctly due to library versions.

What version is your PyTorch and PyTorch Lightning?

Could you install miniconda and create an specific environment as detailed on the repository readme, installing torch and subsequently installing the requirements with pip install -r requirements.txt?

[dscarmo]

I have provided an alternative link to download the weights, in case they are being corrupted when downloaded to your specific location. I have tested these today and they should work!

https://drive.google.com/file/d/1biSkeLhDNpxUA6xqOVEOhpaElJ1Y8mWt/view?usp=sharing

[puallee]

torch 2.4.0
torchinfo 1.8.0
torchmetrics 1.4.0.post0
torchvision 0.19.0

[puallee]

My python version is 3.11

[puallee]

pytools 2024.1.11
pytorch-lightning 2.3.3

[puallee]

SimpleITK==2.2.1

[dscarmo]

@puallee Hello! were you able to solve the problem using the alternative link to the weights I provided?

[puallee]

sorry，the same error， I think the weight maybe not the reason