cjw/amd-r9700-ai-toolboxes
1
0
Fork 0
Code Issues Pull Requests Actions 6 Packages Projects Releases Wiki Activity

chore: Remove deprecated ROCm and rocWMMA Dockerfiles and associated build scripts, updating CI/CD workflows and documentation.

Browse Source
This commit is contained in:
Donato Capitella
2026-02-27 14:49:24 +00:00
parent 2d90fc1d90
commit 7e8603f91e
15 changed files with 25 additions and 1026 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/build_and_publish.yml
+1 -1
View File
@@ -28,7 +28,7 @@ jobs:
IN='${{ inputs.backends }}'
if [[ "$IN" == "all" || -z "$IN" ]]; then
JSON='["rocm-6.4.4","rocm-6.4.4-rocwmma","rocm-7.1.1","rocm-7.1.1-mmf","rocm-7.1.1-rocwmma","rocm-7-nightly","rocm-7-nightly-rocwmma","rocm-7.9","rocm-7.9-rocwmma","vulkan-amdvlk","vulkan-radv"]'
JSON='["rocm-6.4.4","rocm-7.2","rocm7-nightlies","vulkan-amdvlk","vulkan-radv"]'
else
# Remove spaces and build JSON array from comma list
IN_CLEAN=$(echo "$IN" | tr -d '[:space:]')
.github/workflows/prune-old-toolboxes.yml
+1 -1
View File
@@ -44,7 +44,7 @@ jobs:
run: |
IN='${{ github.event.inputs.backends }}'
if [[ "$IN" == "all" || -z "$IN" ]]; then
JSON='["rocm-6.4.2","rocm-6.4.2-rocwmma","rocm-6.4.3","rocm-6.4.3-rocwmma","rocm-6.4.4","rocm-6.4.4-rocwmma","rocm-7.1.1","rocm-7.1.1-rocwmma","rocm-7-nightly","rocm-7-nightly-rocwmma","rocm-7.9","rocm-7.9-rocwmma","vulkan-amdvlk","vulkan-radv"]'
JSON='["rocm-6.4.4","rocm-7.2","rocm7-nightlies","vulkan-amdvlk","vulkan-radv"]'
else
IN_CLEAN=$(echo "$IN" | tr -d '[:space:]')
JSON='["'${IN_CLEAN//,/\",\"}'"]'
README.md
+13 -15
View File
@@ -21,10 +21,10 @@ toolbox create llama-vulkan-radv \
-- --device /dev/dri --group-add video --security-opt seccomp=unconfined
```
**Option B: ROCm (7.1)**
**Option B: ROCm (7.2)**
```bash
toolbox create llama-rocm-7.1-rocwmma \
--image docker.io/kyuz0/amd-r9700-toolboxes:rocm-7.1-rocwmma \
toolbox create llama-rocm-7.2 \
--image docker.io/kyuz0/amd-r9700-toolboxes:rocm-7.2 \
-- --device /dev/dri --device /dev/kfd \
--group-add video --group-add render --group-add sudo --security-opt seccomp=unconfined
```
@@ -34,7 +34,7 @@ toolbox create llama-rocm-7.1-rocwmma \
### 2. Enter the Toolbox
```bash
toolbox enter llama-vulkan-radv
# or: toolbox enter llama-rocm-7.1-rocwmma
# or: toolbox enter llama-rocm-7.2
```
### 3. Download a Model
@@ -95,10 +95,10 @@ llama-server -m models/qwen3-coder-30B-A3B/BF16/Qwen3-Coder-30B-A3B-Instruct-BF1
#### Ubuntu Users (Distrobox)
If you are on Ubuntu, use Distrobox to ensure proper GPU access:
```bash
distrobox create -n llama-rocm-7.1 \
--image docker.io/kyuz0/amd-r9700-toolboxes:rocm-7.1-rocwmma \
distrobox create -n llama-rocm-7.2 \
--image docker.io/kyuz0/amd-r9700-toolboxes:rocm-7.2 \
--additional-flags "--device /dev/kfd --device /dev/dri --group-add video --group-add render --security-opt seccomp=unconfined"
distrobox enter llama-rocm-7.1
distrobox enter llama-rocm-7.2
```
#### Updating Toolboxes
@@ -108,7 +108,7 @@ To pull the latest images and recreate your toolboxes (useful when Llama.cpp upd
./refresh-toolboxes.sh all
# Or refresh specific ones
./refresh-toolboxes.sh llama-vulkan-radv llama-rocm-7.1-rocwmma
./refresh-toolboxes.sh llama-vulkan-radv llama-rocm-7.2
```
## 📦 Architecture & Containers
@@ -118,19 +118,17 @@ To pull the latest images and recreate your toolboxes (useful when Llama.cpp upd
* **RADV (Mesa)**: Best compatibility.
* **AMDVLK**: Official AMD driver. Faster in some cases but has a strict 2GB single buffer limit (some large models won't load).
* **ROCm**: AMD's compute stack (CUDA-like).
* **ROCWMMA**: Variants with optimizations for RDNA3 matrix cores (better Flash Attention).
### Supported Container Images
Images are hosted on [Docker Hub](https://hub.docker.com/r/kyuz0/amd-r9700-toolboxes/tags) and automatically rebuilt on Llama.cpp updates.
| Tag | Backend | Notes |
| :--- | :--- | :--- |
| `vulkan-radv` | Vulkan (Mesa) | **Recommended**. Most stable. |
| `vulkan-amdvlk` | Vulkan (AMDVLK) | Fast, but strict memory limits. |
| `rocm-7.1-rocwmma` | ROCm 7.1 + ROCWMMA | Good balance of features and stability. |
| `rocm-7.9-rocwmma` | ROCm 7.9.0 + ROCWMMA | Newer ROCm release. |
| `rocm-7-nightly-rocwmma` | ROCm 7 Nightly | Bleeding edge. |
| `rocm-6.4.4` | ROCm 6.4.4 | Stable release. |
| `vulkan-radv` | Vulkan (Mesa RADV) | Most stable and compatible. Recommended for most users and all models. |
| `vulkan-amdvlk` | Vulkan (AMDVLK) | Fastest backend—AMD open-source driver. ≤2 GiB single buffer allocation limit, some large models won't load. |
| `rocm-6.4.4` | ROCm 6.4.4 (Fedora 43) | Latest stable 6.x build. Uses Fedora 43 packages with backported patch for kernel 6.18.4+ support. |
| `rocm-7.2` | ROCm 7.2 | Latest stable 7.x build. Includes patch for kernel 6.18.4+ support. |
| `rocm7-nightlies` | ROCm 7 Nightlies | Nightly build for ROCm 7. |
## ⚡ Performance & Planning
refresh-toolboxes.sh
+2 -8
View File
@@ -8,14 +8,8 @@ declare -A TOOLBOXES
TOOLBOXES["llama-vulkan-amdvlk"]="docker.io/kyuz0/amd-r9700-toolboxes:vulkan-amdvlk --device /dev/dri --group-add video --security-opt seccomp=unconfined"
TOOLBOXES["llama-vulkan-radv"]="docker.io/kyuz0/amd-r9700-toolboxes:vulkan-radv --device /dev/dri --group-add video --security-opt seccomp=unconfined"
TOOLBOXES["llama-rocm-6.4.4"]="docker.io/kyuz0/amd-r9700-toolboxes:rocm-6.4.4 --device /dev/dri --device /dev/kfd --group-add video --group-add render --group-add sudo --security-opt seccomp=unconfined"
TOOLBOXES["llama-rocm-6.4.4-rocwmma"]="docker.io/kyuz0/amd-r9700-toolboxes:rocm-6.4.4-rocwmma --device /dev/dri --device /dev/kfd --group-add video --group-add render --group-add sudo --security-opt seccomp=unconfined"
TOOLBOXES["llama-rocm-7.1.1"]="docker.io/kyuz0/amd-r9700-toolboxes:rocm-7.1.1 --device /dev/dri --device /dev/kfd --group-add video --group-add render --group-add sudo --security-opt seccomp=unconfined"
TOOLBOXES["llama-rocm-7.1.1-mmf"]="docker.io/kyuz0/amd-r9700-toolboxes:rocm-7.1.1-mmf --device /dev/dri --device /dev/kfd --group-add video --group-add render --group-add sudo --security-opt seccomp=unconfined"
TOOLBOXES["llama-rocm-7.1.1-rocwmma"]="docker.io/kyuz0/amd-r9700-toolboxes:rocm-7.1.1-rocwmma --device /dev/dri --device /dev/kfd --group-add video --group-add render --group-add sudo --security-opt seccomp=unconfined"
TOOLBOXES["llama-rocm-7.9"]="docker.io/kyuz0/amd-r9700-toolboxes:rocm-7.9 --device /dev/dri --device /dev/kfd --group-add video --group-add render --group-add sudo --security-opt seccomp=unconfined"
TOOLBOXES["llama-rocm-7.9-rocwmma"]="docker.io/kyuz0/amd-r9700-toolboxes:rocm-7.9-rocwmma --device /dev/dri --device /dev/kfd --group-add video --group-add render --group-add sudo --security-opt seccomp=unconfined"
TOOLBOXES["llama-rocm-7-nightly"]="docker.io/kyuz0/amd-r9700-toolboxes:rocm-7-nightly --device /dev/dri --device /dev/kfd --group-add video --group-add render --group-add sudo --security-opt seccomp=unconfined"
TOOLBOXES["llama-rocm-7-nightly-rocwmma"]="docker.io/kyuz0/amd-r9700-toolboxes:rocm-7-nightly-rocwmma --device /dev/dri --device /dev/kfd --group-add video --group-add render --group-add sudo --security-opt seccomp=unconfined"
TOOLBOXES["llama-rocm-7.2"]="docker.io/kyuz0/amd-r9700-toolboxes:rocm-7.2 --device /dev/dri --device /dev/kfd --group-add video --group-add render --group-add sudo --security-opt seccomp=unconfined"
TOOLBOXES["llama-rocm7-nightlies"]="docker.io/kyuz0/amd-r9700-toolboxes:rocm7-nightlies --device /dev/dri --device /dev/kfd --group-add video --group-add render --group-add sudo --security-opt seccomp=unconfined"
function usage() {
echo "Usage: $0 [all|toolbox-name1 toolbox-name2 ...]"
toolboxes/Dockerfile.rocm-6.4.4-rocwmma
-127
View File
@@ -1,127 +0,0 @@
# build stage
FROM registry.fedoraproject.org/fedora:43 AS builder
# rocm 6.4.4 repo
RUN <<'EOF'
tee /etc/yum.repos.d/rocm.repo <<REPO
[ROCm-6.4.4]
name=ROCm6.4.4
baseurl=https://repo.radeon.com/rocm/el9/6.4.4/main
enabled=1
priority=50
gpgcheck=1
gpgkey=https://repo.radeon.com/rocm/rocm.gpg.key
REPO
EOF
# deps
RUN dnf -y --nodocs --setopt=install_weak_deps=False \
--exclude='*sdk*' --exclude='*samples*' --exclude='*-doc*' --exclude='*-docs*' \
install \
make gcc cmake lld clang clang-devel compiler-rt libcurl-devel ninja-build \
rocm-llvm rocm-device-libs hip-runtime-amd hip-devel \
rocblas rocblas-devel hipblas hipblas-devel rocm-cmake libomp-devel libomp \
rocminfo radeontop \
git-core vim sudo rsync \
&& dnf clean all && rm -rf /var/cache/dnf/*
# rocm env
ENV ROCM_PATH=/opt/rocm \
HIP_PATH=/opt/rocm \
HIP_CLANG_PATH=/opt/rocm/llvm/bin \
HIP_DEVICE_LIB_PATH=/opt/rocm/amdgcn/bitcode \
PATH=/opt/rocm/bin:/opt/rocm/llvm/bin:$PATH
# rocWMMA
WORKDIR /opt
COPY ./build-rocwmma.sh .
RUN chmod +x build-rocwmma.sh && ./build-rocwmma.sh
# llama.cpp
WORKDIR /opt/llama.cpp
RUN git clone --recursive https://github.com/ggerganov/llama.cpp.git .
RUN git clean -xdf \
&& git submodule update --recursive
# overwrite upstream header with our local fixed version
COPY ggml/src/ggml-cuda/hip_shfl_fix.h /opt/llama.cpp/ggml/src/ggml-cuda/hip_shfl_fix.h
# Apply # rocWMMA patch
COPY ./apply-rocwmma-fix.sh /opt/apply-rocwmma-fix.sh
RUN chmod +x /opt/apply-rocwmma-fix.sh && /opt/apply-rocwmma-fix.sh /opt/llama.cpp
# Build
RUN set -euo pipefail \
&& cmake -S . -B build \
-DGGML_HIP=ON \
-DAMDGPU_TARGETS=gfx1201 \
-DCMAKE_BUILD_TYPE=Release \
-DGGML_RPC=ON \
-DLLAMA_HIP_UMA=ON \
-DGGML_HIP_ROCWMMA_FATTN=ON \
-DROCM_PATH=/opt/rocm \
-DHIP_PATH=/opt/rocm \
-DHIP_PLATFORM=amd \
-DCMAKE_HIP_FLAGS="--rocm-path=/opt/rocm -include /opt/llama.cpp/ggml/src/ggml-cuda/hip_shfl_fix.h -Wno-macro-redefined" \
&& cmake --build build --config Release -- -j$(nproc) \
&& cmake --install build --config Release
# libs
RUN find /opt/llama.cpp/build -type f -name 'lib*.so*' -exec cp {} /usr/lib64/ \; \
&& ldconfig
# helper
COPY gguf-vram-estimator.py /usr/local/bin/gguf-vram-estimator.py
RUN chmod +x /usr/local/bin/gguf-vram-estimator.py
# runtime stage
FROM registry.fedoraproject.org/fedora-minimal:43
# rocm 6.4.3 repo
RUN <<'EOF'
tee /etc/yum.repos.d/rocm.repo <<REPO
[ROCm-6.4.4]
name=ROCm6.4.4
baseurl=https://repo.radeon.com/rocm/el9/6.4.4/main
enabled=1
priority=50
gpgcheck=1
gpgkey=https://repo.radeon.com/rocm/rocm.gpg.key
REPO
EOF
# runtime deps
RUN microdnf -y --nodocs --setopt=install_weak_deps=0 \
--exclude='*sdk*' --exclude='*samples*' --exclude='*-doc*' --exclude='*-docs*' \
install \
bash ca-certificates libatomic libstdc++ libgcc libgomp sudo \
hip-runtime-amd rocblas hipblas \
rocminfo radeontop \
&& microdnf clean all && rm -rf /var/cache/dnf/*
# copy
COPY --from=builder /usr/local/ /usr/local/
COPY --from=builder /opt/llama.cpp/build/bin/rpc-* /usr/local/bin/
# ld
RUN echo "/usr/local/lib" > /etc/ld.so.conf.d/local.conf \
&& echo "/usr/local/lib64" >> /etc/ld.so.conf.d/local.conf \
&& ldconfig \
&& cp -n /usr/local/lib/libllama*.so* /usr/lib64/ 2>/dev/null || true \
&& ldconfig
# helper
COPY gguf-vram-estimator.py /usr/local/bin/gguf-vram-estimator.py
RUN chmod +x /usr/local/bin/gguf-vram-estimator.py
# profile
RUN printf '%s\n' \
'export ROCBLAS_USE_HIPBLASLT=1' \
> /etc/profile.d/rocm.sh && chmod +x /etc/profile.d/rocm.sh \
&& echo 'source /etc/profile.d/rocm.sh' >> /etc/bashrc
# shell
CMD ["/bin/bash"]
toolboxes/Dockerfile.rocm-7-nightly-rocwmma
-126
View File
@@ -1,126 +0,0 @@
# build
FROM registry.fedoraproject.org/fedora:43 AS builder
RUN dnf -y --nodocs --setopt=install_weak_deps=False install \
make gcc cmake lld clang clang-devel compiler-rt libcurl-devel \
radeontop git vim patch curl ninja-build tar xz aria2c \
&& dnf clean all && rm -rf /var/cache/dnf/*
# find & fetch the latest Linux 7.x.x tarball (gfx1151)
WORKDIR /tmp
ARG ROCM_MAJOR_VER=7
ARG GFX=gfx120X-all
RUN set -euo pipefail; \
BASE="https://therock-nightly-tarball.s3.amazonaws.com"; \
PREFIX="therock-dist-linux-${GFX}-${ROCM_MAJOR_VER}"; \
KEY="$(curl -s "${BASE}?list-type=2&prefix=${PREFIX}" \
| tr '<' '\n' \
| grep -o "therock-dist-linux-${GFX}-${ROCM_MAJOR_VER}\..*\.tar\.gz" \
| sort -V | tail -n1)"; \
echo "Latest tarball: ${KEY}"; \
aria2c -x 16 -s 16 -j 16 --file-allocation=none "${BASE}/${KEY}" -o therock.tar.gz
RUN mkdir -p /opt/rocm-7.0 \
&& tar xzf therock.tar.gz -C /opt/rocm-7.0 --strip-components=1
ENV ROCM_PATH=/opt/rocm-7.0 \
HIP_PLATFORM=amd \
HIP_PATH=/opt/rocm-7.0 \
HIP_CLANG_PATH=/opt/rocm-7.0/llvm/bin \
HIP_INCLUDE_PATH=/opt/rocm-7.0/include \
HIP_LIB_PATH=/opt/rocm-7.0/lib \
HIP_DEVICE_LIB_PATH=/opt/rocm-7.0/lib/llvm/amdgcn/bitcode \
PATH=/opt/rocm-7.0/bin:/opt/rocm-7.0/llvm/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
LD_LIBRARY_PATH=/opt/rocm-7.0/lib:/opt/rocm-7.0/lib64:/opt/rocm-7.0/llvm/lib \
LIBRARY_PATH=/opt/rocm-7.0/lib:/opt/rocm-7.0/lib64 \
CPATH=/opt/rocm-7.0/include \
PKG_CONFIG_PATH=/opt/rocm-7.0/lib/pkgconfig
RUN printf '%s\n' \
'export ROCM_PATH=/opt/rocm-7.0' \
'export HIP_PLATFORM=amd' \
'export HIP_PATH=/opt/rocm-7.0' \
'export HIP_CLANG_PATH=/opt/rocm-7.0/llvm/bin' \
'export HIP_INCLUDE_PATH=/opt/rocm-7.0/include' \
'export HIP_LIB_PATH=/opt/rocm-7.0/lib' \
'export HIP_DEVICE_LIB_PATH=/opt/rocm-7.0/lib/llvm/amdgcn/bitcode' \
'export PATH="$ROCM_PATH/bin:$HIP_CLANG_PATH:$PATH"' \
'export LD_LIBRARY_PATH="$HIP_LIB_PATH:$ROCM_PATH/lib:$ROCM_PATH/lib64:$ROCM_PATH/llvm/lib"' \
'export LIBRARY_PATH="$HIP_LIB_PATH:$ROCM_PATH/lib:$ROCM_PATH/lib64"' \
'export CPATH="$HIP_INCLUDE_PATH"' \
'export PKG_CONFIG_PATH="$ROCM_PATH/lib/pkgconfig"' \
'export ROCBLAS_USE_HIPBLASLT=1' \
> /etc/profile.d/rocm.sh \
&& chmod +x /etc/profile.d/rocm.sh \
&& echo 'source /etc/profile.d/rocm.sh' >> /etc/bashrc
WORKDIR /opt
COPY ./build-rocwmma.sh .
RUN chmod +x build-rocwmma.sh && ./build-rocwmma.sh
WORKDIR /opt/llama.cpp
RUN git clone --recursive https://github.com/ggerganov/llama.cpp.git . \
&& git clean -xdf \
&& git submodule update --recursive
COPY ./apply-rocwmma-fix.sh /opt/apply-rocwmma-fix.sh
RUN chmod +x /opt/apply-rocwmma-fix.sh && /opt/apply-rocwmma-fix.sh /opt/llama.cpp
RUN cmake -S . -B build \
-DGGML_HIP=ON \
-DAMDGPU_TARGETS=gfx1201 \
-DCMAKE_BUILD_TYPE=Release \
-DGGML_RPC=ON \
-DGGML_HIP_ROCWMMA_FATTN=ON \
&& cmake --build build --config Release -- -j$(nproc) \
&& cmake --install build --config Release
# runtime
FROM registry.fedoraproject.org/fedora-minimal:43
RUN microdnf -y --nodocs --setopt=install_weak_deps=0 install \
bash ca-certificates libatomic libstdc++ libgcc radeontop vim \
&& microdnf clean all && rm -rf /var/cache/dnf/*
COPY --from=builder /opt/rocm-7.0 /opt/rocm-7.0
COPY --from=builder /usr/local/ /usr/local/
COPY --from=builder /opt/llama.cpp/build/bin/rpc-* /usr/local/bin/
COPY gguf-vram-estimator.py /usr/local/bin/
RUN chmod +x /usr/local/bin/gguf-vram-estimator.py
ENV ROCM_PATH=/opt/rocm-7.0 \
HIP_PLATFORM=amd \
HIP_PATH=/opt/rocm-7.0 \
HIP_CLANG_PATH=/opt/rocm-7.0/llvm/bin \
HIP_INCLUDE_PATH=/opt/rocm-7.0/include \
HIP_LIB_PATH=/opt/rocm-7.0/lib \
HIP_DEVICE_LIB_PATH=/opt/rocm-7.0/lib/llvm/amdgcn/bitcode \
PATH=/opt/rocm-7.0/bin:/opt/rocm-7.0/llvm/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
LD_LIBRARY_PATH=/opt/rocm-7.0/lib:/opt/rocm-7.0/lib64:/opt/rocm-7.0/llvm/lib \
LIBRARY_PATH=/opt/rocm-7.0/lib:/opt/rocm-7.0/lib64 \
CPATH=/opt/rocm-7.0/include \
PKG_CONFIG_PATH=/opt/rocm-7.0/lib/pkgconfig
RUN printf '%s\n' \
'export ROCM_PATH=/opt/rocm-7.0' \
'export HIP_PLATFORM=amd' \
'export HIP_PATH=/opt/rocm-7.0' \
'export HIP_CLANG_PATH=/opt/rocm-7.0/llvm/bin' \
'export HIP_INCLUDE_PATH=/opt/rocm-7.0/include' \
'export HIP_LIB_PATH=/opt/rocm-7.0/lib' \
'export HIP_DEVICE_LIB_PATH=/opt/rocm-7.0/lib/llvm/amdgcn/bitcode' \
'export PATH="$ROCM_PATH/bin:$HIP_CLANG_PATH:$PATH"' \
'export LD_LIBRARY_PATH="$HIP_LIB_PATH:$ROCM_PATH/lib:$ROCM_PATH/lib64:$ROCM_PATH/llvm/lib"' \
'export LIBRARY_PATH="$HIP_LIB_PATH:$ROCM_PATH/lib:$ROCM_PATH/lib64"' \
'export CPATH="$HIP_INCLUDE_PATH"' \
'export PKG_CONFIG_PATH="$ROCM_PATH/lib/pkgconfig"' \
'export ROCBLAS_USE_HIPBLASLT=1' \
> /etc/profile.d/rocm.sh \
&& chmod +x /etc/profile.d/rocm.sh \
&& echo 'source /etc/profile.d/rocm.sh' >> /etc/bashrc
# make /usr/local libs visible without touching env
RUN echo "/usr/local/lib" > /etc/ld.so.conf.d/local.conf \
&& echo "/usr/local/lib64" >> /etc/ld.so.conf.d/local.conf \
&& ldconfig
CMD ["/bin/bash"]
toolboxes/Dockerfile.rocm-7.1.1-mmf
-117
View File
@@ -1,117 +0,0 @@
# build stage
FROM registry.fedoraproject.org/fedora:43 AS builder
# rocm 7.1 repo
RUN <<'EOF'
tee /etc/yum.repos.d/rocm.repo <<REPO
[ROCm-7.1.1]
name=ROCm7.1.1
baseurl=https://repo.radeon.com/rocm/el9/7.1.1/main
enabled=1
priority=50
gpgcheck=1
gpgkey=https://repo.radeon.com/rocm/rocm.gpg.key
REPO
EOF
# deps
RUN dnf -y --nodocs --setopt=install_weak_deps=False \
--exclude='*sdk*' --exclude='*samples*' --exclude='*-doc*' --exclude='*-docs*' \
install \
make gcc cmake lld clang clang-devel compiler-rt libcurl-devel ninja-build \
rocm-llvm rocm-device-libs hip-runtime-amd hip-devel \
rocblas rocblas-devel hipblas hipblas-devel rocm-cmake libomp-devel libomp \
rocminfo radeontop \
git-core vim sudo rsync \
&& dnf clean all && rm -rf /var/cache/dnf/*
# rocm env
ENV ROCM_PATH=/opt/rocm \
HIP_PATH=/opt/rocm \
HIP_CLANG_PATH=/opt/rocm/llvm/bin \
HIP_DEVICE_LIB_PATH=/opt/rocm/amdgcn/bitcode \
PATH=/opt/rocm/bin:/opt/rocm/llvm/bin:$PATH
# llama.cpp
WORKDIR /opt/llama.cpp
RUN git clone --recursive https://github.com/ggerganov/llama.cpp.git .
# patch
COPY patches/mmf.cuh /tmp/mmf.cuh
# build
RUN git clean -xdf \
&& git submodule update --recursive \
&& count=$(find . -name mmf.cuh -exec cp /tmp/mmf.cuh {} \; -print | wc -l) \
&& if [ "$count" -eq 0 ]; then echo "ERROR: mmf.cuh not found! Patch failed."; exit 1; fi \
&& echo "SUCCESS: Patched $count instance(s) of mmf.cuh" \
&& cmake -S . -B build \
-DGGML_HIP=ON \
-DAMDGPU_TARGETS=gfx1201 \
-DCMAKE_BUILD_TYPE=Release \
-DGGML_RPC=ON \
-DLLAMA_HIP_UMA=ON \
-DGGML_CUDA_ENABLE_UNIFIED_MEMORY=ON \
-DROCM_PATH=/opt/rocm \
-DHIP_PATH=/opt/rocm \
-DHIP_PLATFORM=amd \
-DCMAKE_HIP_FLAGS="--rocm-path=/opt/rocm" \
&& cmake --build build --config Release -- -j$(nproc) \
&& cmake --install build --config Release
# libs
RUN find /opt/llama.cpp/build -type f -name 'lib*.so*' -exec cp {} /usr/lib64/ \; \
&& ldconfig
# helper
COPY gguf-vram-estimator.py /usr/local/bin/gguf-vram-estimator.py
RUN chmod +x /usr/local/bin/gguf-vram-estimator.py
# runtime stage
FROM registry.fedoraproject.org/fedora-minimal:43
# rocm 7.1 repo
RUN <<'EOF'
tee /etc/yum.repos.d/rocm.repo <<REPO
[ROCm-7.1.1]
name=ROCm7.1.1
baseurl=https://repo.radeon.com/rocm/el9/7.1.1/main
enabled=1
priority=50
gpgcheck=1
gpgkey=https://repo.radeon.com/rocm/rocm.gpg.key
REPO
EOF
# runtime deps
RUN microdnf -y --nodocs --setopt=install_weak_deps=0 \
--exclude='*sdk*' --exclude='*samples*' --exclude='*-doc*' --exclude='*-docs*' \
install \
bash ca-certificates libatomic libstdc++ libgcc libgomp sudo \
hip-runtime-amd rocblas hipblas \
rocminfo radeontop \
&& microdnf clean all && rm -rf /var/cache/dnf/*
# copy
COPY --from=builder /usr/local/ /usr/local/
COPY --from=builder /opt/llama.cpp/build/bin/rpc-* /usr/local/bin/
# ld
RUN echo "/usr/local/lib" > /etc/ld.so.conf.d/local.conf \
&& echo "/usr/local/lib64" >> /etc/ld.so.conf.d/local.conf \
&& ldconfig \
&& cp -n /usr/local/lib/libllama*.so* /usr/lib64/ 2>/dev/null || true \
&& ldconfig
# helper
COPY gguf-vram-estimator.py /usr/local/bin/gguf-vram-estimator.py
RUN chmod +x /usr/local/bin/gguf-vram-estimator.py
# profile
RUN printf '%s\n' \
'export ROCBLAS_USE_HIPBLASLT=1' \
> /etc/profile.d/rocm.sh && chmod +x /etc/profile.d/rocm.sh \
&& echo 'source /etc/profile.d/rocm.sh' >> /etc/bashrc
# shell
CMD ["/bin/bash"]
toolboxes/Dockerfile.rocm-7.1.1-rocwmma
-122
View File
@@ -1,122 +0,0 @@
# build stage
FROM registry.fedoraproject.org/fedora:43 AS builder
# rocm 7.1 repo
RUN <<'EOF'
tee /etc/yum.repos.d/rocm.repo <<REPO
[ROCm-7.1.1]
name=ROCm7.1.1
baseurl=https://repo.radeon.com/rocm/el9/7.1.1/main
enabled=1
priority=50
gpgcheck=1
gpgkey=https://repo.radeon.com/rocm/rocm.gpg.key
REPO
EOF
# deps
RUN dnf -y --nodocs --setopt=install_weak_deps=False \
--exclude='*sdk*' --exclude='*samples*' --exclude='*-doc*' --exclude='*-docs*' \
install \
make gcc cmake lld clang clang-devel compiler-rt libcurl-devel ninja-build \
rocm-llvm rocm-device-libs hip-runtime-amd hip-devel \
rocblas rocblas-devel hipblas hipblas-devel rocm-cmake libomp-devel libomp \
rocminfo radeontop \
git-core vim sudo rsync \
&& dnf clean all && rm -rf /var/cache/dnf/*
# rocm env
ENV ROCM_PATH=/opt/rocm \
HIP_PATH=/opt/rocm \
HIP_CLANG_PATH=/opt/rocm/llvm/bin \
HIP_DEVICE_LIB_PATH=/opt/rocm/amdgcn/bitcode \
PATH=/opt/rocm/bin:/opt/rocm/llvm/bin:$PATH
# rocWMMA
WORKDIR /opt
COPY ./build-rocwmma.sh .
RUN chmod +x build-rocwmma.sh && ./build-rocwmma.sh
# llama.cpp
WORKDIR /opt/llama.cpp
RUN git clone --recursive https://github.com/ggerganov/llama.cpp.git .
# Apply # rocWMMA patch
COPY ./apply-rocwmma-fix.sh /opt/apply-rocwmma-fix.sh
RUN chmod +x /opt/apply-rocwmma-fix.sh && /opt/apply-rocwmma-fix.sh /opt/llama.cpp
# build
RUN git clean -xdf \
&& git submodule update --recursive \
&& cmake -S . -B build \
-DGGML_HIP=ON \
-DAMDGPU_TARGETS=gfx1201 \
-DCMAKE_BUILD_TYPE=Release \
-DGGML_RPC=ON \
-DLLAMA_HIP_UMA=ON \
-DGGML_CUDA_ENABLE_UNIFIED_MEMORY=ON \
-DGGML_HIP_ROCWMMA_FATTN=ON \
-DROCM_PATH=/opt/rocm \
-DHIP_PATH=/opt/rocm \
-DHIP_PLATFORM=amd \
-DCMAKE_HIP_FLAGS="--rocm-path=/opt/rocm" \
&& cmake --build build --config Release -- -j$(nproc) \
&& cmake --install build --config Release
# libs
RUN find /opt/llama.cpp/build -type f -name 'lib*.so*' -exec cp {} /usr/lib64/ \; \
&& ldconfig
# helper
COPY gguf-vram-estimator.py /usr/local/bin/gguf-vram-estimator.py
RUN chmod +x /usr/local/bin/gguf-vram-estimator.py
# runtime stage
FROM registry.fedoraproject.org/fedora-minimal:43
# rocm 7.1 repo
RUN <<'EOF'
tee /etc/yum.repos.d/rocm.repo <<REPO
[ROCm-7.1.1]
name=ROCm7.1.1
baseurl=https://repo.radeon.com/rocm/el9/7.1.1/main
enabled=1
priority=50
gpgcheck=1
gpgkey=https://repo.radeon.com/rocm/rocm.gpg.key
REPO
EOF
# runtime deps
RUN microdnf -y --nodocs --setopt=install_weak_deps=0 \
--exclude='*sdk*' --exclude='*samples*' --exclude='*-doc*' --exclude='*-docs*' \
install \
bash ca-certificates libatomic libstdc++ libgcc libgomp sudo \
hip-runtime-amd rocblas hipblas \
rocminfo radeontop \
&& microdnf clean all && rm -rf /var/cache/dnf/*
# copy
COPY --from=builder /usr/local/ /usr/local/
COPY --from=builder /opt/llama.cpp/build/bin/rpc-* /usr/local/bin/
# ld
RUN echo "/usr/local/lib" > /etc/ld.so.conf.d/local.conf \
&& echo "/usr/local/lib64" >> /etc/ld.so.conf.d/local.conf \
&& ldconfig \
&& cp -n /usr/local/lib/libllama*.so* /usr/lib64/ 2>/dev/null || true \
&& ldconfig
# helper
COPY gguf-vram-estimator.py /usr/local/bin/gguf-vram-estimator.py
RUN chmod +x /usr/local/bin/gguf-vram-estimator.py
# profile
RUN printf '%s\n' \
'export ROCBLAS_USE_HIPBLASLT=1' \
> /etc/profile.d/rocm.sh && chmod +x /etc/profile.d/rocm.sh \
&& echo 'source /etc/profile.d/rocm.sh' >> /etc/bashrc
# shell
CMD ["/bin/bash"]
toolboxes/Dockerfile.rocm-7.1.1 → toolboxes/Dockerfile.rocm-7.2
+8 -8
View File
@@ -1,12 +1,12 @@
# build stage
FROM registry.fedoraproject.org/fedora:43 AS builder
# rocm 7.1 repo
# rocm 7.2 repo
RUN <<'EOF'
tee /etc/yum.repos.d/rocm.repo <<REPO
[ROCm-7.1.1]
name=ROCm7.1.1
baseurl=https://repo.radeon.com/rocm/el9/7.1.1/main
[ROCm-7.2]
name=ROCm7.2
baseurl=https://repo.radeon.com/rocm/el9/7.2/main
enabled=1
priority=50
gpgcheck=1
@@ -64,12 +64,12 @@ RUN chmod +x /usr/local/bin/gguf-vram-estimator.py
# runtime stage
FROM registry.fedoraproject.org/fedora-minimal:43
# rocm 7.1 repo
# rocm 7.2 repo
RUN <<'EOF'
tee /etc/yum.repos.d/rocm.repo <<REPO
[ROCm-7.1.1]
name=ROCm7.1.1
baseurl=https://repo.radeon.com/rocm/el9/7.1.1/main
[ROCm-7.2]
name=ROCm7.2
baseurl=https://repo.radeon.com/rocm/el9/7.2/main
enabled=1
priority=50
gpgcheck=1
toolboxes/Dockerfile.rocm-7.9
-125
View File
@@ -1,125 +0,0 @@
# build
FROM registry.fedoraproject.org/fedora:43 AS builder
RUN dnf -y --nodocs --setopt=install_weak_deps=False install \
make gcc cmake lld clang clang-devel compiler-rt libcurl-devel \
radeontop git vim patch curl ninja-build tar xz aria2c \
&& dnf clean all && rm -rf /var/cache/dnf/*
# find & fetch the latest Linux 7.x.x rc tarball (gfx1151)
WORKDIR /tmp
ARG ROCM_MAJOR_VER=7
ARG GFX=gfx120X-all
RUN set -euo pipefail; \
BASE="https://therock-nightly-tarball.s3.amazonaws.com"; \
PREFIX="therock-dist-linux-${GFX}-${ROCM_MAJOR_VER}"; \
KEY="$(curl -s "${BASE}?list-type=2&prefix=${PREFIX}" \
| grep -o "therock-dist-linux-${GFX}-${ROCM_MAJOR_VER}\.[0-9]\+\.[0-9]\+rc[0-9]\{8\}\.tar\.gz" \
| sort | tail -n1)"; \
echo "Latest tarball: ${KEY}"; \
aria2c -x 16 -s 16 -j 16 --file-allocation=none "${BASE}/${KEY}" -o therock.tar.gz \
&& mkdir -p /opt/rocm-7.0 \
&& tar xzf therock.tar.gz -C /opt/rocm-7.0 --strip-components=1 \
&& rm -f therock.tar.gz
ENV ROCM_PATH=/opt/rocm-7.0 \
HIP_PLATFORM=amd \
HIP_PATH=/opt/rocm-7.0 \
HIP_CLANG_PATH=/opt/rocm-7.0/llvm/bin \
HIP_INCLUDE_PATH=/opt/rocm-7.0/include \
HIP_LIB_PATH=/opt/rocm-7.0/lib \
HIP_DEVICE_LIB_PATH=/opt/rocm-7.0/lib/llvm/amdgcn/bitcode \
PATH=/opt/rocm-7.0/bin:/opt/rocm-7.0/llvm/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
LD_LIBRARY_PATH=/opt/rocm-7.0/lib:/opt/rocm-7.0/lib64:/opt/rocm-7.0/llvm/lib \
LIBRARY_PATH=/opt/rocm-7.0/lib:/opt/rocm-7.0/lib64 \
CPATH=/opt/rocm-7.0/include \
PKG_CONFIG_PATH=/opt/rocm-7.0/lib/pkgconfig
RUN printf '%s\n' \
'export ROCM_PATH=/opt/rocm-7.0' \
'export HIP_PLATFORM=amd' \
'export HIP_PATH=/opt/rocm-7.0' \
'export HIP_CLANG_PATH=/opt/rocm-7.0/llvm/bin' \
'export HIP_INCLUDE_PATH=/opt/rocm-7.0/include' \
'export HIP_LIB_PATH=/opt/rocm-7.0/lib' \
'export HIP_DEVICE_LIB_PATH=/opt/rocm-7.0/lib/llvm/amdgcn/bitcode' \
'export PATH="$ROCM_PATH/bin:$HIP_CLANG_PATH:$PATH"' \
'export LD_LIBRARY_PATH="$HIP_LIB_PATH:$ROCM_PATH/lib:$ROCM_PATH/lib64:$ROCM_PATH/llvm/lib"' \
'export LIBRARY_PATH="$HIP_LIB_PATH:$ROCM_PATH/lib:$ROCM_PATH/lib64"' \
'export CPATH="$HIP_INCLUDE_PATH"' \
'export PKG_CONFIG_PATH="$ROCM_PATH/lib/pkgconfig"' \
'export ROCBLAS_USE_HIPBLASLT=1' \
> /etc/profile.d/rocm.sh \
&& chmod +x /etc/profile.d/rocm.sh \
&& echo 'source /etc/profile.d/rocm.sh' >> /etc/bashrc
WORKDIR /opt/llama.cpp
RUN git clone --recursive https://github.com/ggerganov/llama.cpp.git . \
&& git clean -xdf \
&& git submodule update --recursive
RUN cmake -S . -B build \
-DGGML_HIP=ON \
-DAMDGPU_TARGETS=gfx1201 \
-DCMAKE_BUILD_TYPE=Release \
-DGGML_RPC=ON \
-DLLAMA_HIP_UMA=ON \
&& cmake --build build --config Release -- -j$(nproc) \
&& cmake --install build --config Release
# keep bin; drop headers/docs/static libs (retain llama.cpp for rpc binaries)
RUN find /opt/rocm-7.0 -type f -name '*.a' -delete \
&& rm -rf /opt/rocm-7.0/include /opt/rocm-7.0/share \
/opt/rocm-7.0/llvm/include /opt/rocm-7.0/llvm/share
# runtime
FROM registry.fedoraproject.org/fedora-minimal:43
RUN microdnf -y --nodocs --setopt=install_weak_deps=0 install \
bash ca-certificates libatomic libstdc++ libgcc radeontop vim \
&& microdnf clean all && rm -rf /var/cache/dnf/*
COPY --from=builder /opt/rocm-7.0 /opt/rocm-7.0
COPY --from=builder /usr/local/ /usr/local/
COPY --from=builder /opt/llama.cpp/build/bin/rpc-* /usr/local/bin/
COPY gguf-vram-estimator.py /usr/local/bin/
RUN chmod +x /usr/local/bin/gguf-vram-estimator.py
ENV ROCM_PATH=/opt/rocm-7.0 \
HIP_PLATFORM=amd \
HIP_PATH=/opt/rocm-7.0 \
HIP_CLANG_PATH=/opt/rocm-7.0/llvm/bin \
HIP_INCLUDE_PATH=/opt/rocm-7.0/include \
HIP_LIB_PATH=/opt/rocm-7.0/lib \
HIP_DEVICE_LIB_PATH=/opt/rocm-7.0/lib/llvm/amdgcn/bitcode \
PATH=/opt/rocm-7.0/bin:/opt/rocm-7.0/llvm/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
LD_LIBRARY_PATH=/opt/rocm-7.0/lib:/opt/rocm-7.0/lib64:/opt/rocm-7.0/llvm/lib \
LIBRARY_PATH=/opt/rocm-7.0/lib:/opt/rocm-7.0/lib64 \
CPATH=/opt/rocm-7.0/include \
PKG_CONFIG_PATH=/opt/rocm-7.0/lib/pkgconfig
RUN printf '%s\n' \
'export ROCM_PATH=/opt/rocm-7.0' \
'export HIP_PLATFORM=amd' \
'export HIP_PATH=/opt/rocm-7.0' \
'export HIP_CLANG_PATH=/opt/rocm-7.0/llvm/bin' \
'export HIP_INCLUDE_PATH=/opt/rocm-7.0/include' \
'export HIP_LIB_PATH=/opt/rocm-7.0/lib' \
'export HIP_DEVICE_LIB_PATH=/opt/rocm-7.0/lib/llvm/amdgcn/bitcode' \
'export PATH="$ROCM_PATH/bin:$HIP_CLANG_PATH:$PATH"' \
'export LD_LIBRARY_PATH="$HIP_LIB_PATH:$ROCM_PATH/lib:$ROCM_PATH/lib64:$ROCM_PATH/llvm/lib"' \
'export LIBRARY_PATH="$HIP_LIB_PATH:$ROCM_PATH/lib:$ROCM_PATH/lib64"' \
'export CPATH="$HIP_INCLUDE_PATH"' \
'export PKG_CONFIG_PATH="$ROCM_PATH/lib/pkgconfig"' \
'export ROCBLAS_USE_HIPBLASLT=1' \
> /etc/profile.d/rocm.sh \
&& chmod +x /etc/profile.d/rocm.sh \
&& echo 'source /etc/profile.d/rocm.sh' >> /etc/bashrc
# make /usr/local libs visible without touching env
RUN echo "/usr/local/lib" > /etc/ld.so.conf.d/local.conf \
&& echo "/usr/local/lib64" >> /etc/ld.so.conf.d/local.conf \
&& ldconfig
CMD ["/bin/bash"]
toolboxes/Dockerfile.rocm-7.9-rocwmma
-132
View File
@@ -1,132 +0,0 @@
# build
FROM registry.fedoraproject.org/fedora:43 AS builder
RUN dnf -y --nodocs --setopt=install_weak_deps=False install \
make gcc cmake lld clang clang-devel compiler-rt libcurl-devel \
radeontop git vim patch curl ninja-build tar xz aria2c \
&& dnf clean all && rm -rf /var/cache/dnf/*
# find & fetch the latest Linux 7.x.x rc tarball (gfx1151)
WORKDIR /tmp
ARG ROCM_MAJOR_VER=7
ARG GFX=gfx120X-all
RUN set -euo pipefail; \
BASE="https://therock-nightly-tarball.s3.amazonaws.com"; \
PREFIX="therock-dist-linux-${GFX}-${ROCM_MAJOR_VER}"; \
KEY="$(curl -s "${BASE}?list-type=2&prefix=${PREFIX}" \
| grep -o "therock-dist-linux-${GFX}-${ROCM_MAJOR_VER}\.[0-9]\+\.[0-9]\+rc[0-9]\{8\}\.tar\.gz" \
| sort | tail -n1)"; \
echo "Latest tarball: ${KEY}"; \
aria2c -x 16 -s 16 -j 16 --file-allocation=none "${BASE}/${KEY}" -o therock.tar.gz \
&& mkdir -p /opt/rocm-7.0 \
&& tar xzf therock.tar.gz -C /opt/rocm-7.0 --strip-components=1 \
&& rm -f therock.tar.gz
ENV ROCM_PATH=/opt/rocm-7.0 \
HIP_PLATFORM=amd \
HIP_PATH=/opt/rocm-7.0 \
HIP_CLANG_PATH=/opt/rocm-7.0/llvm/bin \
HIP_INCLUDE_PATH=/opt/rocm-7.0/include \
HIP_LIB_PATH=/opt/rocm-7.0/lib \
HIP_DEVICE_LIB_PATH=/opt/rocm-7.0/lib/llvm/amdgcn/bitcode \
PATH=/opt/rocm-7.0/bin:/opt/rocm-7.0/llvm/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
LD_LIBRARY_PATH=/opt/rocm-7.0/lib:/opt/rocm-7.0/lib64:/opt/rocm-7.0/llvm/lib \
LIBRARY_PATH=/opt/rocm-7.0/lib:/opt/rocm-7.0/lib64 \
CPATH=/opt/rocm-7.0/include \
PKG_CONFIG_PATH=/opt/rocm-7.0/lib/pkgconfig
RUN printf '%s\n' \
'export ROCM_PATH=/opt/rocm-7.0' \
'export HIP_PLATFORM=amd' \
'export HIP_PATH=/opt/rocm-7.0' \
'export HIP_CLANG_PATH=/opt/rocm-7.0/llvm/bin' \
'export HIP_INCLUDE_PATH=/opt/rocm-7.0/include' \
'export HIP_LIB_PATH=/opt/rocm-7.0/lib' \
'export HIP_DEVICE_LIB_PATH=/opt/rocm-7.0/lib/llvm/amdgcn/bitcode' \
'export PATH="$ROCM_PATH/bin:$HIP_CLANG_PATH:$PATH"' \
'export LD_LIBRARY_PATH="$HIP_LIB_PATH:$ROCM_PATH/lib:$ROCM_PATH/lib64:$ROCM_PATH/llvm/lib"' \
'export LIBRARY_PATH="$HIP_LIB_PATH:$ROCM_PATH/lib:$ROCM_PATH/lib64"' \
'export CPATH="$HIP_INCLUDE_PATH"' \
'export PKG_CONFIG_PATH="$ROCM_PATH/lib/pkgconfig"' \
'export ROCBLAS_USE_HIPBLASLT=1' \
> /etc/profile.d/rocm.sh \
&& chmod +x /etc/profile.d/rocm.sh \
&& echo 'source /etc/profile.d/rocm.sh' >> /etc/bashrc
WORKDIR /opt
COPY ./build-rocwmma.sh .
RUN chmod +x build-rocwmma.sh && ./build-rocwmma.sh && rm -rf rocWMMA
WORKDIR /opt/llama.cpp
RUN git clone --recursive https://github.com/ggerganov/llama.cpp.git . \
&& git clean -xdf \
&& git submodule update --recursive
COPY ./apply-rocwmma-fix.sh /opt/apply-rocwmma-fix.sh
RUN chmod +x /opt/apply-rocwmma-fix.sh && /opt/apply-rocwmma-fix.sh /opt/llama.cpp
RUN cmake -S . -B build \
-DGGML_HIP=ON \
-DAMDGPU_TARGETS=gfx1201 \
-DCMAKE_BUILD_TYPE=Release \
-DGGML_RPC=ON \
-DGGML_HIP_ROCWMMA_FATTN=ON \
&& cmake --build build --config Release -- -j$(nproc) \
&& cmake --install build --config Release
# keep bin; drop headers/docs/static libs (retain llama.cpp for rpc binaries)
RUN find /opt/rocm-7.0 -type f -name '*.a' -delete \
&& rm -rf /opt/rocm-7.0/include /opt/rocm-7.0/share \
/opt/rocm-7.0/llvm/include /opt/rocm-7.0/llvm/share
# runtime
FROM registry.fedoraproject.org/fedora-minimal:43
RUN microdnf -y --nodocs --setopt=install_weak_deps=0 install \
bash ca-certificates libatomic libstdc++ libgcc radeontop vim \
&& microdnf clean all && rm -rf /var/cache/dnf/*
COPY --from=builder /opt/rocm-7.0 /opt/rocm-7.0
COPY --from=builder /usr/local/ /usr/local/
COPY --from=builder /opt/llama.cpp/build/bin/rpc-* /usr/local/bin/
COPY gguf-vram-estimator.py /usr/local/bin/
RUN chmod +x /usr/local/bin/gguf-vram-estimator.py
ENV ROCM_PATH=/opt/rocm-7.0 \
HIP_PLATFORM=amd \
HIP_PATH=/opt/rocm-7.0 \
HIP_CLANG_PATH=/opt/rocm-7.0/llvm/bin \
HIP_INCLUDE_PATH=/opt/rocm-7.0/include \
HIP_LIB_PATH=/opt/rocm-7.0/lib \
HIP_DEVICE_LIB_PATH=/opt/rocm-7.0/lib/llvm/amdgcn/bitcode \
PATH=/opt/rocm-7.0/bin:/opt/rocm-7.0/llvm/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
LD_LIBRARY_PATH=/opt/rocm-7.0/lib:/opt/rocm-7.0/lib64:/opt/rocm-7.0/llvm/lib \
LIBRARY_PATH=/opt/rocm-7.0/lib:/opt/rocm-7.0/lib64 \
CPATH=/opt/rocm-7.0/include \
PKG_CONFIG_PATH=/opt/rocm-7.0/lib/pkgconfig
RUN printf '%s\n' \
'export ROCM_PATH=/opt/rocm-7.0' \
'export HIP_PLATFORM=amd' \
'export HIP_PATH=/opt/rocm-7.0' \
'export HIP_CLANG_PATH=/opt/rocm-7.0/llvm/bin' \
'export HIP_INCLUDE_PATH=/opt/rocm-7.0/include' \
'export HIP_LIB_PATH=/opt/rocm-7.0/lib' \
'export HIP_DEVICE_LIB_PATH=/opt/rocm-7.0/lib/llvm/amdgcn/bitcode' \
'export PATH="$ROCM_PATH/bin:$HIP_CLANG_PATH:$PATH"' \
'export LD_LIBRARY_PATH="$HIP_LIB_PATH:$ROCM_PATH/lib:$ROCM_PATH/lib64:$ROCM_PATH/llvm/lib"' \
'export LIBRARY_PATH="$HIP_LIB_PATH:$ROCM_PATH/lib:$ROCM_PATH/lib64"' \
'export CPATH="$HIP_INCLUDE_PATH"' \
'export PKG_CONFIG_PATH="$ROCM_PATH/lib/pkgconfig"' \
'export ROCBLAS_USE_HIPBLASLT=1' \
> /etc/profile.d/rocm.sh \
&& chmod +x /etc/profile.d/rocm.sh \
&& echo 'source /etc/profile.d/rocm.sh' >> /etc/bashrc
# make /usr/local libs visible without touching env
RUN echo "/usr/local/lib" > /etc/ld.so.conf.d/local.conf \
&& echo "/usr/local/lib64" >> /etc/ld.so.conf.d/local.conf \
&& ldconfig
CMD ["/bin/bash"]
toolboxes/Dockerfile.rocm-7-nightly → toolboxes/Dockerfile.rocm7-nightlies
View File
toolboxes/apply-rocwmma-fix.sh
-155
View File
@@ -1,155 +0,0 @@
#!/bin/bash
# apply-rocwmma-fix.sh - Apply rocWMMA compatibility fixes to llama.cpp
# Usage: ./apply-rocwmma-fix.sh <path-to-llama.cpp-directory>
# Source: https://github.com/lhl/strix-halo-testing/blob/main/llm-bench/apply-rocwmma-fix.sh
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
LLAMA_DIR="${1:-}"
if [[ -z "$LLAMA_DIR" ]]; then
echo "Usage: $0 <path-to-llama.cpp-directory>"
echo ""
echo "This script applies rocWMMA compatibility fixes to a llama.cpp checkout."
echo "The fixes resolve warp synchronization mask type conflicts between"
echo "ROCm headers and CUDA-style code when building with GGML_HIP_ROCWMMA_FATTN=ON."
echo ""
echo "Example:"
echo " $0 ./llama.cpp"
echo " $0 /path/to/your/llama.cpp"
exit 1
fi
if [[ ! -d "$LLAMA_DIR" ]]; then
echo "Error: Directory '$LLAMA_DIR' does not exist"
exit 1
fi
if [[ ! -f "$LLAMA_DIR/CMakeLists.txt" ]] || ! grep -q "llama" "$LLAMA_DIR/CMakeLists.txt" 2>/dev/null; then
echo "Error: '$LLAMA_DIR' does not appear to be a llama.cpp directory"
echo "Expected to find CMakeLists.txt with 'llama' references"
exit 1
fi
VENDOR_HIP_FILE="$LLAMA_DIR/ggml/src/ggml-cuda/vendors/hip.h"
if [[ ! -f "$VENDOR_HIP_FILE" ]]; then
echo "Error: HIP vendor header not found at: $VENDOR_HIP_FILE"
echo "This script requires a llama.cpp version with HIP support"
exit 1
fi
echo "Applying rocWMMA compatibility fixes to: $LLAMA_DIR"
echo ""
# Check if fixes are already applied
if grep -q "GGML_HIP_WARP_MASK" "$VENDOR_HIP_FILE" 2>/dev/null; then
echo "rocWMMA fixes appear to already be applied (found GGML_HIP_WARP_MASK)"
echo "To reapply, please first revert changes and run this script again"
exit 0
fi
echo "Step 1: Modifying HIP vendor header..."
# Backup the original file
cp "$VENDOR_HIP_FILE" "$VENDOR_HIP_FILE.backup"
# Find the line with __shfl_sync and __shfl_xor_sync definitions
SHFL_LINE=$(grep -n "^#define __shfl_sync" "$VENDOR_HIP_FILE" | head -1 | cut -d: -f1)
if [[ -z "$SHFL_LINE" ]]; then
echo "Error: Could not find __shfl_sync macro definition in $VENDOR_HIP_FILE"
echo "This script may need updates for this version of llama.cpp"
exit 1
fi
# Create a temporary file with the fix
{
# Print lines before the __shfl_sync definition
head -n $((SHFL_LINE - 1)) "$VENDOR_HIP_FILE"
# Add our conditional compilation block
cat << 'EOF'
#ifdef GGML_HIP_ROCWMMA_FATTN
// ROCm requires 64-bit masks for __shfl_*_sync functions
#define GGML_HIP_WARP_MASK 0xFFFFFFFFFFFFFFFFULL
#else
#define __shfl_sync(mask, var, laneMask, width) __shfl(var, laneMask, width)
#define __shfl_xor_sync(mask, var, laneMask, width) __shfl_xor(var, laneMask, width)
#define GGML_HIP_WARP_MASK 0xFFFFFFFF
#endif
EOF
# Skip the original __shfl_sync and __shfl_xor_sync lines and print the rest
tail -n +$((SHFL_LINE + 2)) "$VENDOR_HIP_FILE"
} > "$VENDOR_HIP_FILE.tmp"
mv "$VENDOR_HIP_FILE.tmp" "$VENDOR_HIP_FILE"
echo " ✓ Added conditional GGML_HIP_WARP_MASK macro to vendor header"
echo ""
echo "Step 2: Replacing hardcoded warp masks in CUDA files..."
# Find all .cu and .cuh files in the ggml/src/ggml-cuda directory
CUDA_FILES=($(find "$LLAMA_DIR/ggml/src/ggml-cuda" -name "*.cu" -o -name "*.cuh" 2>/dev/null | sort))
if [[ ${#CUDA_FILES[@]} -eq 0 ]]; then
echo "Warning: No CUDA files found in $LLAMA_DIR/ggml/src/ggml-cuda"
echo "This may be expected for some llama.cpp versions"
else
MODIFIED_COUNT=0
for file in "${CUDA_FILES[@]}"; do
# Check if file contains the hardcoded masks
if grep -q "0xFFFFFFFF\|0xffffffff" "$file" 2>/dev/null; then
# Create backup
cp "$file" "$file.backup"
# Replace both uppercase and lowercase versions
sed -i 's/0xFFFFFFFF/GGML_HIP_WARP_MASK/g; s/0xffffffff/GGML_HIP_WARP_MASK/g' "$file"
MODIFIED_COUNT=$((MODIFIED_COUNT + 1))
echo " ✓ Modified: $(basename "$file")"
fi
done
echo " ✓ Modified $MODIFIED_COUNT CUDA files"
fi
echo ""
echo "Step 3: Verification..."
# Verify the vendor header was modified correctly
if grep -q "GGML_HIP_ROCWMMA_FATTN" "$VENDOR_HIP_FILE" && grep -q "GGML_HIP_WARP_MASK" "$VENDOR_HIP_FILE"; then
echo " ✓ Vendor header modification verified"
else
echo " ✗ Vendor header modification failed"
# Restore backup
mv "$VENDOR_HIP_FILE.backup" "$VENDOR_HIP_FILE"
echo " ✓ Restored original vendor header"
exit 1
fi
echo ""
echo "🎉 rocWMMA compatibility fixes applied successfully!"
echo ""
echo "What was changed:"
echo " • Added conditional GGML_HIP_WARP_MASK macro to ggml/src/ggml-cuda/vendors/hip.h"
echo " • Replaced hardcoded 0xFFFFFFFF/0xffffffff with GGML_HIP_WARP_MASK in CUDA files"
echo ""
echo "Behavior:"
echo " • For regular HIP builds: GGML_HIP_WARP_MASK = 0xFFFFFFFF (no change)"
echo " • For rocWMMA builds: GGML_HIP_WARP_MASK = 0xFFFFFFFFFFFFFFFFULL (64-bit masks)"
echo ""
echo "To build with rocWMMA support, use:"
echo " cmake -B build -S '$LLAMA_DIR' -DGGML_HIP=ON -DAMDGPU_TARGETS=\"gfx1201\" -DGGML_HIP_ROCWMMA_FATTN=ON"
echo ""
echo "Backup files were created with .backup extension in case you need to revert."
echo ""
echo "Done! Your llama.cpp checkout now supports rocWMMA builds."
toolboxes/build-rocwmma.sh
-61
View File
@@ -1,61 +0,0 @@
# Source: https://github.com/lhl/strix-halo-testing/blob/main/llm-bench/build-rocwmma.sh
git clone https://github.com/ROCm/rocWMMA
cd rocWMMA
# --- BEGIN: make OpenMP explicit for ROCm toolchains (drop-in) ---
# find libomp (check ROCM_PATH first, then system)
CANDIDATES=(
"${ROCM_PATH}/llvm/lib/libomp.so"
"${ROCM_PATH}/llvm/lib/libomp.a"
"/usr/lib64/libomp.so"
"/usr/lib64/libomp.a"
"/usr/local/lib/libomp.so"
)
FOUND_LIBOMP=""
for p in "${CANDIDATES[@]}"; do
if [ -f "$p" ]; then
FOUND_LIBOMP="$p"
break
fi
done
CMAKE_OPTS=""
if [ -n "$FOUND_LIBOMP" ]; then
# directory & basename
OMP_LIB_DIR="$(dirname "$FOUND_LIBOMP")"
OMP_LIB_BASENAME="$(basename "$FOUND_LIBOMP")"
# set cache vars so FindOpenMP will succeed
CMAKE_OPTS="${CMAKE_OPTS} -DOpenMP_CXX_FLAGS=-fopenmp=libomp"
CMAKE_OPTS="${CMAKE_OPTS} -DOpenMP_C_FLAGS=-fopenmp=libomp"
CMAKE_OPTS="${CMAKE_OPTS} -DOpenMP_CXX_LIB_NAMES=omp"
CMAKE_OPTS="${CMAKE_OPTS} -DOpenMP_C_LIB_NAMES=omp"
CMAKE_OPTS="${CMAKE_OPTS} -DOpenMP_LIBRARY=${FOUND_LIBOMP}"
CMAKE_OPTS="${CMAKE_OPTS} -DOpenMP_INCLUDE_DIR=${ROCM_PATH}/llvm/include"
export LD_LIBRARY_PATH="${OMP_LIB_DIR}${LD_LIBRARY_PATH:+:}$LD_LIBRARY_PATH"
export CXXFLAGS="-fopenmp=libomp ${CXXFLAGS:-}"
export LDFLAGS="-L${OMP_LIB_DIR} -lomp ${LDFLAGS:-}"
else
# fallback: force flags so FindOpenMP might at least get flags
CMAKE_OPTS="${CMAKE_OPTS} -DOpenMP_CXX_FLAGS=-fopenmp=libomp -DOpenMP_C_FLAGS=-fopenmp=libomp"
export CXXFLAGS="-fopenmp=libomp ${CXXFLAGS:-}"
export LDFLAGS="${LDFLAGS:-} -lomp"
fi
# --- END: make OpenMP explicit ---
CC=$ROCM_PATH/llvm/bin/amdclang \
CXX=$ROCM_PATH/llvm/bin/amdclang++ \
cmake -B build -S . -G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=$ROCM_PATH \
-DROCWMMA_BUILD_TESTS=OFF \
-DROCWMMA_BUILD_SAMPLES=OFF \
-DGPU_TARGETS="gfx1201" \
-DOpenMP_CXX_FLAGS="-fopenmp=libomp" \
-DOpenMP_C_FLAGS="-fopenmp=libomp" \
-DOpenMP_omp_LIBRARY="/usr/lib64/libomp.so" \
-DOpenMP_CXX_LIB_NAMES="omp" \
-DOpenMP_C_LIB_NAMES="omp" \
-DOpenMP_INCLUDE_DIRS="/usr/lib64/clang/19/include"
cmake --install build
sudo cmake --install build
toolboxes/hip-rocm7rc.patch
-28
View File
@@ -1,28 +0,0 @@
diff --git a/ggml/src/ggml-cuda/vendors/hip.h b/ggml/src/ggml-cuda/vendors/hip.h
index 8b172e60..b813f523 100644
--- a/ggml/src/ggml-cuda/vendors/hip.h
+++ b/ggml/src/ggml-cuda/vendors/hip.h
@@ -137,19 +137,11 @@
#define CUBLAS_STATUS_INTERNAL_ERROR HIPBLAS_STATUS_INTERNAL_ERROR
#define CUBLAS_STATUS_NOT_SUPPORTED HIPBLAS_STATUS_NOT_SUPPORTED
-#if HIP_VERSION >= 70000000
-#define CUBLAS_COMPUTE_16F HIPBLAS_COMPUTE_16F
-#define CUBLAS_COMPUTE_32F HIPBLAS_COMPUTE_32F
+#define CUBLAS_COMPUTE_16F HIPBLAS_COMPUTE_16F
+#define CUBLAS_COMPUTE_32F HIPBLAS_COMPUTE_32F
#define CUBLAS_COMPUTE_32F_FAST_16F HIPBLAS_COMPUTE_32F_FAST_16F
-#define cublasComputeType_t hipblasComputeType_t
-#define cudaDataType_t hipDataType
-#else
-#define CUBLAS_COMPUTE_16F HIPBLAS_R_16F
-#define CUBLAS_COMPUTE_32F HIPBLAS_R_32F
-#define CUBLAS_COMPUTE_32F_FAST_16F HIPBLAS_R_32F
-#define cublasComputeType_t hipblasDatatype_t
-#define cudaDataType_t hipblasDatatype_t
-#endif // HIP_VERSION >= 7000000
+#define cublasComputeType_t hipblasComputeType_t
+#define cudaDataType_t hipDataType
#if !defined(__HIP_PLATFORM_AMD__)
#error "The HIP backend supports only AMD targets"