cjw/amd-r9700-ai-toolboxes
1
0
Fork 0
Code Issues Pull Requests Actions 6 Packages Projects Releases Wiki Activity

updated benchmarks

Browse Source
This commit is contained in:
Donato Capitella
2025-12-07 10:27:33 +00:00
parent 204493bd4c
commit ffbc6d98ff
74 changed files with 4692 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm7.1.1-mmf__fa1__single.log
+20
View File
@@ -0,0 +1,20 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
/opt/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:92: ROCm error
/usr/local/lib64/libggml-base.so.0(+0x3565) [0x7fd2c4f8e565]
/usr/local/lib64/libggml-base.so.0(ggml_print_backtrace+0x1eb) [0x7fd2c4f8e92b]
/usr/local/lib64/libggml-base.so.0(ggml_abort+0x11f) [0x7fd2c4f8eaaf]
/usr/local/lib64/libggml-hip.so.0(+0x300aef2) [0x7fd2c8055ef2]
/usr/local/lib64/libggml-hip.so.0(+0x30100fe) [0x7fd2c805b0fe]
/usr/local/lib64/libggml-base.so.0(ggml_backend_sched_synchronize+0x2e) [0x7fd2c4fa5a9e]
/usr/local/lib64/libllama.so.0(_ZN13llama_context11synchronizeEv+0x10) [0x7fd2c8722cf0]
/usr/local/bin/llama-bench() [0x40acfc]
/usr/local/bin/llama-bench() [0x408a7d]
/lib64/libc.so.6(+0x35b5) [0x7fd2c49245b5]
/lib64/libc.so.6(__libc_start_main+0x88) [0x7fd2c4924668]
/usr/local/bin/llama-bench() [0x409b65]
✖ ! [rocm7.1.1-mmf] Ministral-3-14B-Instruct-2512-BF16__fa1 failed (exit 0)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm7.1.1-mmf__hblt0__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 374.16 ± 0.31 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 19.87 ± 0.00 |
build: 22577583a (7312)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm7.1.1-rocwmma__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 2544.26 ± 1.34 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 19.82 ± 0.00 |
build: 22577583a (7312)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm7.1.1-rocwmma__hblt0__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 367.12 ± 0.18 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 19.83 ± 0.00 |
build: 22577583a (7312)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm7.1.1__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 2560.35 ± 5.29 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 19.82 ± 0.08 |
build: 22577583a (7312)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm7.1.1__hblt0__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 370.91 ± 0.02 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 19.86 ± 0.00 |
build: 22577583a (7312)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm7.1.1-mmf__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 4352.84 ± 10.67 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.44 ± 0.01 |
build: 22577583a (7312)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm7.1.1-mmf__hblt0__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 602.48 ± 0.50 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.44 ± 0.01 |
build: 22577583a (7312)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm7.1.1-rocwmma__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 4238.60 ± 12.37 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.39 ± 0.01 |
build: 22577583a (7312)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm7.1.1-rocwmma__hblt0__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 598.20 ± 0.19 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.39 ± 0.02 |
build: 22577583a (7312)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm7.1.1__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 4327.37 ± 7.29 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.44 ± 0.01 |
build: 22577583a (7312)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm7.1.1__hblt0__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 599.70 ± 0.58 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.40 ± 0.02 |
build: 22577583a (7312)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-BF16-00001-of-00002__rocm7.1.1-mmf__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 1562.53 ± 124.66 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 57.64 ± 0.13 |
build: 22577583a (7312)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-BF16-00001-of-00002__rocm7.1.1-mmf__hblt0__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 648.98 ± 0.99 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 51.67 ± 0.18 |
build: 22577583a (7312)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-BF16-00001-of-00002__rocm7.1.1-rocwmma__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 1556.37 ± 85.43 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 57.91 ± 0.16 |
build: 22577583a (7312)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-BF16-00001-of-00002__rocm7.1.1-rocwmma__hblt0__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 663.24 ± 4.86 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 52.39 ± 0.01 |
build: 22577583a (7312)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-BF16-00001-of-00002__rocm7.1.1__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 1532.05 ± 91.83 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 57.22 ± 0.15 |
build: 22577583a (7312)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-BF16-00001-of-00002__rocm7.1.1__hblt0__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 667.09 ± 10.31 |
| qwen3moe 30B.A3B BF16 | 56.89 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 52.29 ± 0.01 |
build: 22577583a (7312)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-Q4_K_M__rocm7.1.1-mmf__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2723.67 ± 221.55 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 100.30 ± 0.67 |
build: 22577583a (7312)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-Q4_K_M__rocm7.1.1-mmf__hblt0__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2857.26 ± 9.36 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 100.70 ± 0.42 |
build: 22577583a (7312)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-Q4_K_M__rocm7.1.1-rocwmma__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2816.71 ± 32.12 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 102.96 ± 0.75 |
build: 22577583a (7312)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-Q4_K_M__rocm7.1.1-rocwmma__hblt0__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2788.27 ± 21.84 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 103.20 ± 0.51 |
build: 22577583a (7312)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-Q4_K_M__rocm7.1.1__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2867.19 ± 13.92 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 102.01 ± 0.63 |
build: 22577583a (7312)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-Q4_K_M__rocm7.1.1__hblt0__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2856.57 ± 29.59 |
| qwen3moe 30B.A3B Q4_K - Medium | 17.35 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 100.81 ± 0.47 |
build: 22577583a (7312)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-UD-Q8_K_XL__rocm7.1.1-mmf__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2928.78 ± 251.26 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 76.10 ± 0.29 |
build: 22577583a (7312)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-UD-Q8_K_XL__rocm7.1.1-mmf__hblt0__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2713.01 ± 14.12 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 75.96 ± 0.29 |
build: 22577583a (7312)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-UD-Q8_K_XL__rocm7.1.1-rocwmma__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2979.50 ± 32.77 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 77.26 ± 0.28 |
build: 22577583a (7312)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-UD-Q8_K_XL__rocm7.1.1-rocwmma__hblt0__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2648.65 ± 19.90 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 76.82 ± 0.26 |
build: 22577583a (7312)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-UD-Q8_K_XL__rocm7.1.1__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 3051.37 ± 17.30 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 75.94 ± 0.31 |
build: 22577583a (7312)
benchmark/results/Qwen3-Coder-30B-A3B-Instruct-UD-Q8_K_XL__rocm7.1.1__hblt0__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | pp512 | 2706.95 ± 19.80 |
| qwen3moe 30B.A3B Q8_0 | 33.51 GiB | 30.53 B | ROCm | 99 | 1 | tg128 | 75.97 ± 0.30 |
build: 22577583a (7312)
benchmark/results/Qwen3-Next-80B-A3B-Instruct-UD-Q4_K_XL__rocm7.1.1-mmf__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 843.80 ± 54.72 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 37.77 ± 0.11 |
build: 22577583a (7312)
benchmark/results/Qwen3-Next-80B-A3B-Instruct-UD-Q4_K_XL__rocm7.1.1-mmf__hblt0__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 879.13 ± 5.75 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 34.01 ± 0.03 |
build: 22577583a (7312)
benchmark/results/Qwen3-Next-80B-A3B-Instruct-UD-Q4_K_XL__rocm7.1.1-rocwmma__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 900.01 ± 8.67 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 37.77 ± 0.07 |
build: 22577583a (7312)
benchmark/results/Qwen3-Next-80B-A3B-Instruct-UD-Q4_K_XL__rocm7.1.1-rocwmma__hblt0__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 887.15 ± 1.62 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 33.84 ± 0.04 |
build: 22577583a (7312)
benchmark/results/Qwen3-Next-80B-A3B-Instruct-UD-Q4_K_XL__rocm7.1.1__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 854.90 ± 49.88 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 37.81 ± 0.08 |
build: 22577583a (7312)
benchmark/results/Qwen3-Next-80B-A3B-Instruct-UD-Q4_K_XL__rocm7.1.1__hblt0__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | pp512 | 880.89 ± 2.65 |
| qwen3next ?B Q4_K - Medium | 42.01 GiB | 79.67 B | ROCm | 99 | 1 | tg128 | 33.81 ± 0.02 |
build: 22577583a (7312)
benchmark/results/gemma-3-12b-it-UD-Q8_K_XL__rocm7.1.1-mmf__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 2726.93 ± 218.34 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 31.27 ± 0.03 |
build: 22577583a (7312)
benchmark/results/gemma-3-12b-it-UD-Q8_K_XL__rocm7.1.1-mmf__hblt0__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 1936.02 ± 0.61 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 31.27 ± 0.02 |
build: 22577583a (7312)
benchmark/results/gemma-3-12b-it-UD-Q8_K_XL__rocm7.1.1-rocwmma__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 2672.34 ± 14.98 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 30.98 ± 0.03 |
build: 22577583a (7312)
benchmark/results/gemma-3-12b-it-UD-Q8_K_XL__rocm7.1.1-rocwmma__hblt0__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 1851.96 ± 1.40 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 30.98 ± 0.02 |
build: 22577583a (7312)
benchmark/results/gemma-3-12b-it-UD-Q8_K_XL__rocm7.1.1__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 2854.01 ± 18.63 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 31.27 ± 0.02 |
build: 22577583a (7312)
benchmark/results/gemma-3-12b-it-UD-Q8_K_XL__rocm7.1.1__hblt0__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | pp512 | 1937.83 ± 0.71 |
| gemma3 12B Q8_0 | 13.40 GiB | 11.77 B | ROCm | 99 | 1 | tg128 | 31.24 ± 0.02 |
build: 22577583a (7312)
benchmark/results/gemma-3-27b-it-BF16-00001-of-00002__rocm7.1.1-mmf__fa1__dual.log
+21
View File
@@ -0,0 +1,21 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
/opt/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:92: ROCm error
/usr/local/lib64/libggml-base.so.0(+0x3565) [0x7f50e09b7565]
/usr/local/lib64/libggml-base.so.0(ggml_print_backtrace+0x1eb) [0x7f50e09b792b]
/usr/local/lib64/libggml-base.so.0(ggml_abort+0x11f) [0x7f50e09b7aaf]
/usr/local/lib64/libggml-hip.so.0(+0x300aef2) [0x7f50e3a7eef2]
/usr/local/lib64/libggml-hip.so.0(+0x30100fe) [0x7f50e3a840fe]
/usr/local/lib64/libggml-base.so.0(ggml_backend_sched_synchronize+0x2e) [0x7f50e09cea9e]
/usr/local/lib64/libllama.so.0(_ZN13llama_context11synchronizeEv+0x10) [0x7f50e414bcf0]
/usr/local/bin/llama-bench() [0x40acfc]
/usr/local/bin/llama-bench() [0x408a7d]
/lib64/libc.so.6(+0x35b5) [0x7f50e034d5b5]
/lib64/libc.so.6(__libc_start_main+0x88) [0x7f50e034d668]
/usr/local/bin/llama-bench() [0x409b65]
✖ ! [rocm7.1.1-mmf] gemma-3-27b-it-BF16-00001-of-00002__fa1 failed (exit 0)
benchmark/results/gemma-3-27b-it-BF16-00001-of-00002__rocm7.1.1-mmf__hblt0__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | pp512 | 179.62 ± 0.14 |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | tg128 | 9.37 ± 0.00 |
build: 22577583a (7312)
benchmark/results/gemma-3-27b-it-BF16-00001-of-00002__rocm7.1.1-rocwmma__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | pp512 | 1599.56 ± 7.86 |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | tg128 | 9.48 ± 0.00 |
build: 22577583a (7312)
benchmark/results/gemma-3-27b-it-BF16-00001-of-00002__rocm7.1.1-rocwmma__hblt0__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | pp512 | 177.39 ± 0.07 |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | tg128 | 9.37 ± 0.00 |
build: 22577583a (7312)
benchmark/results/gemma-3-27b-it-BF16-00001-of-00002__rocm7.1.1__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | pp512 | 1666.49 ± 12.25 |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | tg128 | 9.49 ± 0.00 |
build: 22577583a (7312)
benchmark/results/gemma-3-27b-it-BF16-00001-of-00002__rocm7.1.1__hblt0__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | pp512 | 178.86 ± 0.11 |
| gemma3 27B BF16 | 50.31 GiB | 27.01 B | ROCm | 99 | 1 | tg128 | 9.37 ± 0.00 |
build: 22577583a (7312)
benchmark/results/gemma-3-4b-it-Q3_K_S__rocm7.1.1-mmf__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 4B Q3_K - Small | 1.80 GiB | 3.88 B | ROCm | 99 | 1 | pp512 | 6603.82 ± 189.78 |
| gemma3 4B Q3_K - Small | 1.80 GiB | 3.88 B | ROCm | 99 | 1 | tg128 | 125.68 ± 0.24 |
build: 22577583a (7312)
benchmark/results/gemma-3-4b-it-Q3_K_S__rocm7.1.1-mmf__hblt0__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 4B Q3_K - Small | 1.80 GiB | 3.88 B | ROCm | 99 | 1 | pp512 | 6566.74 ± 165.41 |
| gemma3 4B Q3_K - Small | 1.80 GiB | 3.88 B | ROCm | 99 | 1 | tg128 | 124.66 ± 1.92 |
build: 22577583a (7312)
benchmark/results/gemma-3-4b-it-Q3_K_S__rocm7.1.1-rocwmma__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 4B Q3_K - Small | 1.80 GiB | 3.88 B | ROCm | 99 | 1 | pp512 | 6279.64 ± 163.15 |
| gemma3 4B Q3_K - Small | 1.80 GiB | 3.88 B | ROCm | 99 | 1 | tg128 | 122.09 ± 0.54 |
build: 22577583a (7312)
benchmark/results/gemma-3-4b-it-Q3_K_S__rocm7.1.1-rocwmma__hblt0__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 4B Q3_K - Small | 1.80 GiB | 3.88 B | ROCm | 99 | 1 | pp512 | 6273.87 ± 161.21 |
| gemma3 4B Q3_K - Small | 1.80 GiB | 3.88 B | ROCm | 99 | 1 | tg128 | 121.74 ± 0.57 |
build: 22577583a (7312)
benchmark/results/gemma-3-4b-it-Q3_K_S__rocm7.1.1__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 4B Q3_K - Small | 1.80 GiB | 3.88 B | ROCm | 99 | 1 | pp512 | 6548.80 ± 183.77 |
| gemma3 4B Q3_K - Small | 1.80 GiB | 3.88 B | ROCm | 99 | 1 | tg128 | 125.52 ± 0.08 |
build: 22577583a (7312)
benchmark/results/gemma-3-4b-it-Q3_K_S__rocm7.1.1__hblt0__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gemma3 4B Q3_K - Small | 1.80 GiB | 3.88 B | ROCm | 99 | 1 | pp512 | 6521.77 ± 177.30 |
| gemma3 4B Q3_K - Small | 1.80 GiB | 3.88 B | ROCm | 99 | 1 | tg128 | 125.17 ± 0.46 |
build: 22577583a (7312)
benchmark/results/gpt-oss-120b-mxfp4-00001-of-00003__rocm7.1.1-mmf__fa1__dual.log
+9
View File
@@ -0,0 +1,9 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to load model '/home/kyuz0/models/gpt-oss-120b/gpt-oss-120b-mxfp4-00001-of-00003.gguf'
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
✖ ! [rocm7.1.1-mmf] gpt-oss-120b-mxfp4-00001-of-00003__fa1 failed (exit 0)
benchmark/results/gpt-oss-120b-mxfp4-00001-of-00003__rocm7.1.1-mmf__hblt0__fa1__dual.log
+9
View File
@@ -0,0 +1,9 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to load model '/home/kyuz0/models/gpt-oss-120b/gpt-oss-120b-mxfp4-00001-of-00003.gguf'
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
✖ ! [rocm7.1.1-mmf] gpt-oss-120b-mxfp4-00001-of-00003__hblt0__fa1 failed (exit 0)
benchmark/results/gpt-oss-120b-mxfp4-00001-of-00003__rocm7.1.1-rocwmma__fa1__dual.log
+9
View File
@@ -0,0 +1,9 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to load model '/home/kyuz0/models/gpt-oss-120b/gpt-oss-120b-mxfp4-00001-of-00003.gguf'
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
✖ ! [rocm7.1.1-rocwmma] gpt-oss-120b-mxfp4-00001-of-00003__fa1 failed (exit 0)
benchmark/results/gpt-oss-120b-mxfp4-00001-of-00003__rocm7.1.1-rocwmma__hblt0__fa1__dual.log
+9
View File
@@ -0,0 +1,9 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to load model '/home/kyuz0/models/gpt-oss-120b/gpt-oss-120b-mxfp4-00001-of-00003.gguf'
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
✖ ! [rocm7.1.1-rocwmma] gpt-oss-120b-mxfp4-00001-of-00003__hblt0__fa1 failed (exit 0)
benchmark/results/gpt-oss-120b-mxfp4-00001-of-00003__rocm7.1.1__fa1__dual.log
+9
View File
@@ -0,0 +1,9 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to load model '/home/kyuz0/models/gpt-oss-120b/gpt-oss-120b-mxfp4-00001-of-00003.gguf'
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
✖ ! [rocm7.1.1] gpt-oss-120b-mxfp4-00001-of-00003__fa1 failed (exit 0)
benchmark/results/gpt-oss-120b-mxfp4-00001-of-00003__rocm7.1.1__hblt0__fa1__dual.log
+9
View File
@@ -0,0 +1,9 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to load model '/home/kyuz0/models/gpt-oss-120b/gpt-oss-120b-mxfp4-00001-of-00003.gguf'
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
✖ ! [rocm7.1.1] gpt-oss-120b-mxfp4-00001-of-00003__hblt0__fa1 failed (exit 0)
benchmark/results/gpt-oss-20b-mxfp4__rocm7.1.1-mmf__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gpt-oss 20B MXFP4 MoE | 11.27 GiB | 20.91 B | ROCm | 99 | 1 | pp512 | 4588.21 ± 47.38 |
| gpt-oss 20B MXFP4 MoE | 11.27 GiB | 20.91 B | ROCm | 99 | 1 | tg128 | 132.24 ± 0.57 |
build: 22577583a (7312)
benchmark/results/gpt-oss-20b-mxfp4__rocm7.1.1-mmf__hblt0__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gpt-oss 20B MXFP4 MoE | 11.27 GiB | 20.91 B | ROCm | 99 | 1 | pp512 | 4501.95 ± 6.95 |
| gpt-oss 20B MXFP4 MoE | 11.27 GiB | 20.91 B | ROCm | 99 | 1 | tg128 | 132.67 ± 0.25 |
build: 22577583a (7312)
benchmark/results/gpt-oss-20b-mxfp4__rocm7.1.1-rocwmma__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gpt-oss 20B MXFP4 MoE | 11.27 GiB | 20.91 B | ROCm | 99 | 1 | pp512 | 4784.27 ± 37.89 |
| gpt-oss 20B MXFP4 MoE | 11.27 GiB | 20.91 B | ROCm | 99 | 1 | tg128 | 134.51 ± 0.32 |
build: 22577583a (7312)
benchmark/results/gpt-oss-20b-mxfp4__rocm7.1.1-rocwmma__hblt0__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gpt-oss 20B MXFP4 MoE | 11.27 GiB | 20.91 B | ROCm | 99 | 1 | pp512 | 4690.59 ± 84.92 |
| gpt-oss 20B MXFP4 MoE | 11.27 GiB | 20.91 B | ROCm | 99 | 1 | tg128 | 134.54 ± 0.27 |
build: 22577583a (7312)
benchmark/results/gpt-oss-20b-mxfp4__rocm7.1.1__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gpt-oss 20B MXFP4 MoE | 11.27 GiB | 20.91 B | ROCm | 99 | 1 | pp512 | 4558.95 ± 58.68 |
| gpt-oss 20B MXFP4 MoE | 11.27 GiB | 20.91 B | ROCm | 99 | 1 | tg128 | 131.62 ± 0.50 |
build: 22577583a (7312)
benchmark/results/gpt-oss-20b-mxfp4__rocm7.1.1__hblt0__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| gpt-oss 20B MXFP4 MoE | 11.27 GiB | 20.91 B | ROCm | 99 | 1 | pp512 | 4510.84 ± 8.42 |
| gpt-oss 20B MXFP4 MoE | 11.27 GiB | 20.91 B | ROCm | 99 | 1 | tg128 | 132.55 ± 0.37 |
build: 22577583a (7312)
benchmark/results/llama-2-7b.Q4_0__rocm7.1.1-mmf__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| llama 7B Q4_0 | 3.56 GiB | 6.74 B | ROCm | 99 | 1 | pp512 | 4806.21 ± 73.61 |
| llama 7B Q4_0 | 3.56 GiB | 6.74 B | ROCm | 99 | 1 | tg128 | 98.03 ± 0.03 |
build: 22577583a (7312)
benchmark/results/llama-2-7b.Q4_0__rocm7.1.1-mmf__hblt0__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| llama 7B Q4_0 | 3.56 GiB | 6.74 B | ROCm | 99 | 1 | pp512 | 4786.12 ± 86.07 |
| llama 7B Q4_0 | 3.56 GiB | 6.74 B | ROCm | 99 | 1 | tg128 | 98.49 ± 0.03 |
build: 22577583a (7312)
benchmark/results/llama-2-7b.Q4_0__rocm7.1.1-rocwmma__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| llama 7B Q4_0 | 3.56 GiB | 6.74 B | ROCm | 99 | 1 | pp512 | 4653.72 ± 63.28 |
| llama 7B Q4_0 | 3.56 GiB | 6.74 B | ROCm | 99 | 1 | tg128 | 98.72 ± 0.07 |
build: 22577583a (7312)
benchmark/results/llama-2-7b.Q4_0__rocm7.1.1-rocwmma__hblt0__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| llama 7B Q4_0 | 3.56 GiB | 6.74 B | ROCm | 99 | 1 | pp512 | 4652.96 ± 61.07 |
| llama 7B Q4_0 | 3.56 GiB | 6.74 B | ROCm | 99 | 1 | tg128 | 97.98 ± 0.04 |
build: 22577583a (7312)
benchmark/results/llama-2-7b.Q4_0__rocm7.1.1__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| llama 7B Q4_0 | 3.56 GiB | 6.74 B | ROCm | 99 | 1 | pp512 | 4799.82 ± 86.02 |
| llama 7B Q4_0 | 3.56 GiB | 6.74 B | ROCm | 99 | 1 | tg128 | 98.13 ± 0.01 |
build: 22577583a (7312)
benchmark/results/llama-2-7b.Q4_0__rocm7.1.1__hblt0__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| llama 7B Q4_0 | 3.56 GiB | 6.74 B | ROCm | 99 | 1 | pp512 | 4778.61 ± 91.21 |
| llama 7B Q4_0 | 3.56 GiB | 6.74 B | ROCm | 99 | 1 | tg128 | 98.36 ± 0.04 |
build: 22577583a (7312)
docs/results.json
+3931 -1
View File
File diff suppressed because it is too large Load Diff
toolboxes/Dockerfile.rocm-7.1.1-mmf
+3 -1
View File
@@ -42,7 +42,9 @@ COPY patches/mmf.cuh /tmp/mmf.cuh
# build
RUN git clean -xdf \
&& git submodule update --recursive \
&& find . -name mmf.cuh -exec cp /tmp/mmf.cuh {} \; \
&& count=$(find . -name mmf.cuh -exec cp /tmp/mmf.cuh {} \; -print | wc -l) \
&& if [ "$count" -eq 0 ]; then echo "ERROR: mmf.cuh not found! Patch failed."; exit 1; fi \
&& echo "SUCCESS: Patched $count instance(s) of mmf.cuh" \
&& cmake -S . -B build \
-DGGML_HIP=ON \
-DAMDGPU_TARGETS=gfx1201 \