Executor doesn't support recent CUDA capabilities

[freddyheppell]

When I try and use this executor on a compute server with an RTX 3090, I get the following warning:

executor0_1  | NVIDIA GeForce RTX 3090 with CUDA capability sm_86 is not compatible with the current PyTorch installation.
executor0_1  | The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70.

I'm using the latest-gpu tag of the executor, inside a Docker container, from running jina export docker-compose on this flow:

jtype: Flow
executors:
  - uses: 'jinahub+docker://TransformerTorchEncoder/latest-gpu'
    uses_with:
      pretrained_model_name_or_path: 'sentence-transformers/distiluse-base-multilingual-cased-v2'
      device: cuda
    gpus: 1

[samsja]

This is a known issue with pytorch and the rtx 3090 : https://discuss.pytorch.org/t/geforce-rtx-3090-with-cuda-capability-sm-86-is-not-compatible-with-the-current-pytorch-installation/123499?u=samsja

Unfortunately we build our docker image with the default pytorch version that is available on pypi and which is incompatible with the rtx 3090. This version being the one which is compatible for most of the people with have to stick with it.

In your case, unfortunately, the only current solution is to build the docker image for the TransformerTorchEncoder yourself.

You can follow these steps

• clone the Executor repo : https://github.com/jina-ai/executor-text-transformers-torch-encoder
• Modify the gpu_requirement.txt file and change the torch version with the good one for you rtx 3090 ( see here for more details about the version)
• build the docker image yourself
• modifiy the docker-compose.yml and change the docker image name to the one that you just build