[HuggingFace][Neuronx] Inference - Optimum Neuron 0.0.25 - Neuron sdk 2.20.0 - Transformers to 4.43.2

[dacorvo]

GitHub Issue #4307

Description

This PR creates Hugginface's PyTorch DLC for inference on neuron-v2 devices (namely Inferentia 2 and Trainium).

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[Captainia]

Thanks for making the changes, could you follow the PR template to trigger tests for PR hooks? You can edit the developer toml file based on instruction

deep-learning-containers/dlc_developer_config.toml

Line 2 in fe838a7

# Set to "huggingface", for example, if you are a huggingface developer. Default is ""

.

[dacorvo]

Thanks for making the changes, could you follow the PR template to trigger tests for PR hooks? You can edit the developer toml file based on instruction

I added a revertme commit with the toml changes.

[Captainia]

Thanks for making the changes, could you follow the PR template to trigger tests for PR hooks? You can edit the developer toml file based on instruction

I added a revertme commit with the toml changes.

Checking with team why only the format check is triggered

[Captainia]

/rerun

[sirutBuasai]

/rerun

[Captainia]

/rerun

[Captainia]

Code build failures when building the docker file-

E: Unable to locate package libml2
--
407 |  
408 | The command '/bin/sh -c apt-get update  && apt-get upgrade -y  && apt-get install -y --no-install-recommends software-properties-common  && add-apt-repository ppa:openjdk-r/ppa  && apt-get update  && apt-get install -y --no-install-recommends     build-essential     apt-transport-https     ca-certificates     cmake     curl     emacs     git     jq     libgl1-mesa-glx     libsm6     libxext6     libxrender-dev     libgssapi-krb5-2     openjdk-11-jdk     vim     wget     unzip     zlib1g-dev     libcap-dev     gpg-agent     libexpat1     libml2  && rm -rf /var/lib/apt/lists/*  && rm -rf /tmp/tmp*  && apt-get clean' returned a non-zero code: 100
409 | [ERROR] ERROR during Docker BUILD
410 | [ERROR] Error message received for docker/2.1/py3/sdk2.20.0/Dockerfile.neuronx while docker build: {'errorDetail': {'code': 100, 'message': "The command '/bin/sh -c apt-get update  && apt-get upgrade -y  && apt-get install -y --no-install-recommends software-properties-common  && add-apt-repository ppa:openjdk-r/ppa  && apt-get update  && apt-get install -y --no-install-recommends     build-essential     apt-transport-https     ca-certificates     cmake     curl     emacs     git     jq     libgl1-mesa-glx     libsm6     libxext6     libxrender-dev     libgssapi-krb5-2     openjdk-11-jdk     vim     wget     unzip     zlib1g-dev     libcap-dev     gpg-agent     libexpat1     libml2  && rm -rf /var/lib/apt/lists/*  && rm -rf /tmp/tmp*  && apt-get clean' returned a non-zero code: 100"}, 'error': "The command '/bin/sh -c apt-get update  && apt-get upgrade -y  && apt-get install -y --no-install-recommends software-properties-common  && add-apt-repository ppa:openjdk-r/ppa  && apt-get update  && apt-get install -y --no-install-recommends     build-essential     apt-transport-https     ca-certificates     cmake     curl     emacs     git     jq     libgl1-mesa-glx     libsm6     libxext6     libxrender-dev     libgssapi-krb5-2     openjdk-11-jdk     vim     wget     unzip     zlib1g-dev     libcap-dev     gpg-agent     libexpat1     libml2  && rm -rf /var/lib/apt/lists/*  && rm -rf /tmp/tmp*  && apt-get clean' returned a non-zero code: 100"}
411 | [INFO] Exiting with image build status 1 without image check.
412 | Not in terminal, reprint now using normal build-in print function.

@dacorvo could you take a look?

[Captainia]

Some errors seen in sagemaker tests:

{
--
1006 | E                       "timestamp": 1728989020547,
1007 | E                       "message": "ImportError: cannot import name 'Conversation' from 'transformers.pipelines' (/opt/conda/lib/python3.10/site-packages/transformers/pipelines/__init__.py)",
1008 | E                       "ingestionTime": 1728989021084
1009 | E                   }

In sanity-check tests:

------------------------------ Captured log call -------------------------------
--
759 | ERROR    test.dlc_tests.sanity.test_safety_report_file:test_safety_report_file.py:97 SAFETY_REPORT (FAILED) [pkg: requests] [installed: 2.31.0] [vulnerabilities: [SafetyVulnerabilityAdvisory(vulnerability_id='71064', advisory='Affected versions of Requests, when making requests through a Requests `Session`, if the first request is made with `verify=False` to disable cert verification, all subsequent requests to the same host will continue to ignore cert verification regardless of changes to the value of `verify`. This behavior will continue for the lifecycle of the connection in the connection pool. Requests 2.32.0 fixes the issue, but versions 2.32.0 and 2.32.1 were yanked due to conflicts with CVE-2024-35195 mitigation.', reason_to_ignore='N/A', spec='<2.32.2', ignored=False)]]

For the sanity test, you can add to ignore list

[dacorvo]

@Captainia the sagemaker-test error comes from sagemaker-huggingface-inference-toolkit and has already been fixed.

However, we would need to do an official release including that fix.

cc @philschmid @JingyaHuang

[Captainia]

Thank you, feel free to update developer config and we will merge it after.