Updating ubi-tag e vllm-tgis-adapter #147
Conversation
Fixed upstream in vllm-project#5091
fixes RHOAIENG-8043 Co-authored-by: Chih-Chieh-Yang <[email protected]> Signed-off-by: Thomas Parnell <[email protected]>
- get rid cuda-devel stage, use cuda 12.4 - add build flags - remove useless installs
add libsodium for tensorizer encryption --------- Signed-off-by: Prashant Gupta <[email protected]> Co-authored-by: Daniele <[email protected]>
this is the default when `--worker-use-ray` is not provided and world-size > 1
Signed-off-by: Nathan Weinberg <[email protected]>
Signed-off-by: Travis Johnson <[email protected]>
Signed-off-by: Travis Johnson <[email protected]>
Sync vllm with upstream/v0.5.5 to odh/main for 2.13
This turns off tracking by default. If someone wants to, they can simply override this in yaml.
Disable usage tracking
Signed-off-by: Jefferson Fialho <[email protected]>
fialhocoelho
requested review from
joerunde,
prashantgupta24,
njhill and
dtrifiro
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: fialhocoelho The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
@fialhocoelho: The following tests failed, say
Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
| @@ -1,5 +1,5 @@ | |||
| ## Global Args ################################################################# | |||
| ARG BASE_UBI_IMAGE_TAG=9.4 | |||
| ARG BASE_UBI_IMAGE_TAG=9.4-1227 | |||
There was a problem hiding this comment.
This is not required, using 9.4 will automatically fetch the latest available version.
There was a problem hiding this comment.
@dtrifiro IMO it's best to use the specific version tags. Otherwise you have no idea what was actually pulled .. the image contents just depends on when the build happened to run and it isn't reproducible.
There was a problem hiding this comment.
We want to be able to clearly say yes this build is based on this specific UBI image, without having to dig through CI logs or inspect the internal of the built image.
There was a problem hiding this comment.
I understand your reproducibility concern, but unless we configure something like renovate (https://docs.renovatebot.com) to automatically open PRs whenever a new UBI image is available, I think that it'd be better to avoid the overhead of having to bump the version manually as it's something that's easy to be forgotten
There was a problem hiding this comment.
I created an issue and will start working on configuring renovate for this repo.
|
PR needs rebase. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
* Log more HPU memory metrics during vLLM startup * Overhaul memory management in HPUGraph capture * fix percentage in decode buckets
vllm-tgis-adaptertov0.4.1to support the latestvllm v0.5.0.9.4-1227.