Kserve TorchserveModel can't handle torch's auth

[AntPeixe]

🐛 Describe the bug

Kserve's TorchserveModel load doesn't take into consideration the fact that torchserve has auth enabled by default.

Error logs

INFO:kserve:Copying contents of /mnt/models/model-store to local
INFO:root:Loading model .. 1 of 10 tries..
INFO:root:The model layoutlm-pytorch-model is not ready
INFO:root:Sleep 30 seconds for load layoutlm-pytorch-model..
...
...
...
INFO:root:Loading model .. 2 of 10 tries..
2024-08-29T10:16:01,232 [INFO ] epollEventLoopGroup-3-1 ACCESS_LOG - /127.0.0.1:50126 "GET /models/model?customized=false HTTP/1.1" 400 4

Installation instructions

Using docker image pytorch/torchserve-kfs:0.11.1 to serve a model

Model Packaging

Model was packaged using torch-model-archiver and the same mar file would work on the older pytorch/torchserve-kfs:0.8.0

config.properties

No response

Versions

Copied from within docker image pytorch/torchserve-kfs:0.11.1

------------------------------------------------------------------------------------------
Environment headers
------------------------------------------------------------------------------------------
Torchserve branch:

torchserve==0.11.1
torch-model-archiver==0.11.1

Python version: 3.9 (64-bit runtime)
Python executable: /home/venv/bin/python

Versions of relevant python libraries:
captum==0.6.0
numpy==1.24.3
pillow==10.3.0
psutil==5.9.8
requests==2.32.0
requests-oauthlib==2.0.0
torch==2.3.0+cpu
torch-model-archiver==0.11.1
torch-workflow-archiver==0.2.14
torchaudio==2.3.0+cpu
torchserve==0.11.1
torchtext==0.18.0+cpu
torchvision==0.18.0+cpu
transformers==4.42.4
wheel==0.42.0
torch==2.3.0+cpu
torchtext==0.18.0+cpu
torchvision==0.18.0+cpu
torchaudio==2.3.0+cpu

Java Version:


OS: Ubuntu 20.04.6 LTS
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
Clang version: N/A
CMake version: N/A

Environment:
library_path (LD_/DYLD_):

Repro instructions

• create model mar file
• create config.properties file and DO NOT disable auth
• run the image pytorch/torchserve-kfs:0.11.1 - should fail to load as the management request doesn't include auth token

Possible Solution

Read key_file.json and use the token in the request