Add nvtx. · coreweave/ml-containers@4de958f

Triggered via push August 28, 2024 18:35

wbrown

pushed 4de958f

wbrown.add-transformerengine

Status Cancelled

Total duration 53m 32s

Artifacts –

torch-nccl.yml

on: push

Get torch:nccl Config / Read Configuration File

7s

Matrix: Build torch:nccl

Annotations

12 errors and 3 warnings

Build torch:nccl (12.4.1, cudnn, ubuntu20.04, 2.21.5-1, 85f9143) / Build torch / Build Images

buildx failed with: ERROR: failed to solve: process "/bin/sh -c export CUDA_MAJOR_VERSION=$(echo $CUDA_VERSION | cut -d. -f1) CUDA_MINOR_VERSION=$(echo $CUDA_VERSION | cut -d. -f2) && export CUDA_PACKAGE_VERSION=\"${CUDA_MAJOR_VERSION}-${CUDA_MINOR_VERSION}\" && apt-get -qq update && apt-get -qq install --no-upgrade -y cudnn9-cuda-${CUDA_MAJOR_VERSION} libcurand-${CUDA_PACKAGE_VERSION} libcufft-${CUDA_PACKAGE_VERSION} libcublas-${CUDA_PACKAGE_VERSION} cuda-nvrtc-${CUDA_PACKAGE_VERSION} libcusparse-${CUDA_PACKAGE_VERSION} libcusolver-${CUDA_PACKAGE_VERSION} cuda-cupti-${CUDA_PACKAGE_VERSION} libnvjpeg-${CUDA_PACKAGE_VERSION} libnvtoolsext1 && { if [ $CUDA_MAJOR_VERSION -ge 12 ]; then apt-get -qq install --no-upgrade -y libnvjitlink-${CUDA_PACKAGE_VERSION}; fi; } && { if [ ! -d /opt/nccl-tests ]; then export NCCL_PACKAGE_VERSION=\"2.*+cuda${CUDA_MAJOR_VERSION}.${CUDA_MINOR_VERSION}\" && apt-get -qq install --no-upgrade -y \"libnccl2=$NCCL_PACKAGE_VERSION\"; fi; } && apt-get clean && ldconfig" did not complete successfully: exit code: 100

Build torch:nccl (12.3.2, cudnn9, ubuntu20.04, 2.20.3-1, 85f9143) / Build torch / Build Images

buildx failed with: ERROR: failed to solve: process "/bin/sh -c export CUDA_MAJOR_VERSION=$(echo $CUDA_VERSION | cut -d. -f1) CUDA_MINOR_VERSION=$(echo $CUDA_VERSION | cut -d. -f2) && export CUDA_PACKAGE_VERSION=\"${CUDA_MAJOR_VERSION}-${CUDA_MINOR_VERSION}\" && apt-get -qq update && apt-get -qq install --no-upgrade -y cudnn9-cuda-${CUDA_MAJOR_VERSION} libcurand-${CUDA_PACKAGE_VERSION} libcufft-${CUDA_PACKAGE_VERSION} libcublas-${CUDA_PACKAGE_VERSION} cuda-nvrtc-${CUDA_PACKAGE_VERSION} libcusparse-${CUDA_PACKAGE_VERSION} libcusolver-${CUDA_PACKAGE_VERSION} cuda-cupti-${CUDA_PACKAGE_VERSION} libnvjpeg-${CUDA_PACKAGE_VERSION} libnvtoolsext1 && { if [ $CUDA_MAJOR_VERSION -ge 12 ]; then apt-get -qq install --no-upgrade -y libnvjitlink-${CUDA_PACKAGE_VERSION}; fi; } && { if [ ! -d /opt/nccl-tests ]; then export NCCL_PACKAGE_VERSION=\"2.*+cuda${CUDA_MAJOR_VERSION}.${CUDA_MINOR_VERSION}\" && apt-get -qq install --no-upgrade -y \"libnccl2=$NCCL_PACKAGE_VERSION\"; fi; } && apt-get clean && ldconfig" did not complete successfully: exit code: 100

Build torch:nccl (12.3.2, cudnn9, ubuntu22.04, 2.20.3-1, 85f9143) / Build torch / Build Images

FailFast: cancelling since parallel instance has failed

Build torch:nccl (12.3.2, cudnn9, ubuntu22.04, 2.20.3-1, 85f9143) / Build torch / Build Images

The operation was canceled.

Build torch:nccl (12.4.1, cudnn, ubuntu22.04, 2.21.5-1, 85f9143) / Build torch / Build Images

FailFast: cancelling since parallel instance has failed

Build torch:nccl (12.4.1, cudnn, ubuntu22.04, 2.21.5-1, 85f9143) / Build torch / Build Images

The operation was canceled.

Build torch:nccl (12.2.2, cudnn8, ubuntu22.04, 2.19.3-1, 85f9143) / Build torch / Build Images

FailFast: cancelling since parallel instance has failed

Build torch:nccl (12.2.2, cudnn8, ubuntu22.04, 2.19.3-1, 85f9143) / Build torch / Build Images

The operation was canceled.