Skip to content

images/gpu: update test images for Blackwell (sm_100/sm_120) #14477

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
copybara-service wants to merge 1 commit into
base: master
Choose a base branch
from
+11 −4
Open

images/gpu: update test images for Blackwell (sm_100/sm_120) #14477

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion images/gpu/nccl-tests/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
FROM nvidia/cuda:12.5.0-devel-ubuntu22.04
# CUDA 12.8 is the first toolkit whose nvcc knows the Blackwell arches
# (sm_100/sm_120); nccl-tests derives its gencode list from the toolkit.
FROM nvidia/cuda:12.8.1-devel-ubuntu22.04

RUN apt-get update && apt-get install git -y

Expand Down
11 changes: 8 additions & 3 deletions images/gpu/vllm/Dockerfile.x86_64
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,15 @@ RUN apk add git-lfs && \
rm /opt-125m/tf_model.h5
# post checkout hook. checks that command git lfs is available.
RUN GIT_CLONE_PROTECTION_ACTIVE=false git clone https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered /dataset
RUN git clone https://github.com/vllm-project/vllm.git /vllm && cd /vllm && git checkout v0.4.2
RUN git clone https://github.com/vllm-project/vllm.git /vllm && cd /vllm && git checkout v0.27.1

# v0.4.2
FROM vllm/vllm-openai@sha256:c63bd9e99c6d5914032b396be38a80fd9640693da62b64b907434d38bf38a8e9
# v0.27.1. Do not downgrade below this without checking torch's compiled
# arch list: releases before this shipped torch 2.3.0+cu121, whose kernels
# stop at sm_90, so every CUDA op fails on Blackwell (sm_100/sm_120) with
# "no kernel image is available for execution on the device" under any
# runtime. v0.27.1 ships torch 2.13.0+cu130 built for
# ['sm_75','sm_80','sm_86','sm_90','sm_100','sm_120'].
FROM vllm/vllm-openai@sha256:0a51ea5b4ae2dc5d81890e5173f54203d2a3ae0cfffe51b8fd2afd4391bfd967
COPY --from=downloader /vllm/examples/template_chatml.jinja /vllm/examples/template_chatml.jinja
COPY --from=downloader /vllm/benchmarks /vllm/benchmarks
COPY --from=downloader /opt-125m /model
Expand Down
Loading