cjw/amd-r9700-ai-toolboxes
1
0
Fork 0
Code Issues Pull Requests Actions 6 Packages Projects Releases Wiki Activity

added Devstral-2 to benchmarks

Browse Source
This commit is contained in:
Donato Capitella
2025-12-27 17:44:35 +00:00
parent 2e8bef127f
commit 2d90fc1d90
55 changed files with 3189 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm-7-nightly-rocwmma__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 1 | pp512 | 251.12 ± 0.62 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 1 | tg128 | 7.53 ± 0.03 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm-7-nightly-rocwmma__fa1__longctx16384__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 124.60 ± 0.00 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 6.52 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm-7-nightly-rocwmma__fa1__longctx32768__dual.log
+9
View File
@@ -0,0 +1,9 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to create context with model '/home/kyuz0/models/Q3_K_S/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002.gguf'
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
✖ ! [rocm-7-nightly-rocwmma] Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__fa1 __longctx32768 failed (exit 0)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm-7-nightly-rocwmma__hblt0__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 1 | pp512 | 246.03 ± 0.16 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 1 | tg128 | 7.41 ± 0.02 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm-7-nightly-rocwmma__hblt0__fa1__longctx16384__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 123.04 ± 0.00 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 6.46 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm-7-nightly-rocwmma__hblt0__fa1__longctx32768__dual.log
+9
View File
@@ -0,0 +1,9 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to create context with model '/home/kyuz0/models/Q3_K_S/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002.gguf'
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
✖ ! [rocm-7-nightly-rocwmma] Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__hblt0__fa1 __longctx32768 failed (exit 0)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm-7-nightly__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 1 | pp512 | 239.85 ± 0.14 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 1 | tg128 | 7.37 ± 0.01 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm-7-nightly__fa1__longctx16384__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 99.59 ± 0.00 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 6.82 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm-7-nightly__fa1__longctx32768__dual.log
+9
View File
@@ -0,0 +1,9 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to create context with model '/home/kyuz0/models/Q3_K_S/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002.gguf'
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
✖ ! [rocm-7-nightly] Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__fa1 __longctx32768 failed (exit 0)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm-7-nightly__hblt0__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 1 | pp512 | 240.74 ± 0.07 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 1 | tg128 | 7.41 ± 0.01 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm-7-nightly__hblt0__fa1__longctx16384__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 100.10 ± 0.00 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 6.83 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm-7-nightly__hblt0__fa1__longctx32768__dual.log
+9
View File
@@ -0,0 +1,9 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to create context with model '/home/kyuz0/models/Q3_K_S/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002.gguf'
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
✖ ! [rocm-7-nightly] Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__hblt0__fa1 __longctx32768 failed (exit 0)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm-7.9-rocwmma__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 1 | pp512 | 235.80 ± 0.18 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 1 | tg128 | 7.01 ± 0.02 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm-7.9-rocwmma__fa1__longctx16384__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 95.03 ± 0.00 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 6.09 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm-7.9-rocwmma__fa1__longctx32768__dual.log
+9
View File
@@ -0,0 +1,9 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to create context with model '/home/kyuz0/models/Q3_K_S/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002.gguf'
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
✖ ! [rocm-7.9-rocwmma] Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__fa1 __longctx32768 failed (exit 0)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm-7.9-rocwmma__hblt0__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 1 | pp512 | 233.71 ± 0.16 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 1 | tg128 | 6.94 ± 0.01 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm-7.9-rocwmma__hblt0__fa1__longctx16384__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 94.33 ± 0.00 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 6.09 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm-7.9-rocwmma__hblt0__fa1__longctx32768__dual.log
+9
View File
@@ -0,0 +1,9 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to create context with model '/home/kyuz0/models/Q3_K_S/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002.gguf'
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
✖ ! [rocm-7.9-rocwmma] Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__hblt0__fa1 __longctx32768 failed (exit 0)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm-7.9__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 1 | pp512 | 235.23 ± 0.14 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 1 | tg128 | 6.94 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm-7.9__fa1__longctx16384__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 98.01 ± 0.00 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 6.45 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm-7.9__fa1__longctx32768__dual.log
+9
View File
@@ -0,0 +1,9 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to create context with model '/home/kyuz0/models/Q3_K_S/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002.gguf'
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
✖ ! [rocm-7.9] Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__fa1 __longctx32768 failed (exit 0)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm-7.9__hblt0__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 1 | pp512 | 237.08 ± 0.15 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 1 | tg128 | 7.00 ± 0.01 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm-7.9__hblt0__fa1__longctx16384__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 99.40 ± 0.00 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 6.47 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm-7.9__hblt0__fa1__longctx32768__dual.log
+9
View File
@@ -0,0 +1,9 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to create context with model '/home/kyuz0/models/Q3_K_S/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002.gguf'
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
✖ ! [rocm-7.9] Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__hblt0__fa1 __longctx32768 failed (exit 0)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm6_4_4-rocwmma__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 1 | pp512 | 235.42 ± 0.14 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 1 | tg128 | 7.02 ± 0.02 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm6_4_4-rocwmma__fa1__longctx16384__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 94.65 ± 0.00 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 6.09 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm6_4_4-rocwmma__fa1__longctx32768__dual.log
+9
View File
@@ -0,0 +1,9 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to create context with model '/home/kyuz0/models/Q3_K_S/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002.gguf'
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
✖ ! [rocm6_4_4-rocwmma] Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__fa1 __longctx32768 failed (exit 0)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm6_4_4-rocwmma__hblt0__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 1 | pp512 | 233.43 ± 0.03 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 1 | tg128 | 6.94 ± 0.01 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm6_4_4-rocwmma__hblt0__fa1__longctx16384__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 94.89 ± 0.00 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 6.09 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm6_4_4-rocwmma__hblt0__fa1__longctx32768__dual.log
+9
View File
@@ -0,0 +1,9 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to create context with model '/home/kyuz0/models/Q3_K_S/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002.gguf'
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
✖ ! [rocm6_4_4-rocwmma] Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__hblt0__fa1 __longctx32768 failed (exit 0)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm6_4_4__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 1 | pp512 | 238.15 ± 0.10 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 1 | tg128 | 6.99 ± 0.01 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm6_4_4__fa1__longctx16384__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 109.79 ± 0.00 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 6.50 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm6_4_4__fa1__longctx32768__dual.log
+9
View File
@@ -0,0 +1,9 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to create context with model '/home/kyuz0/models/Q3_K_S/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002.gguf'
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
✖ ! [rocm6_4_4] Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__fa1 __longctx32768 failed (exit 0)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm6_4_4__hblt0__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 1 | pp512 | 239.15 ± 0.10 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 1 | tg128 | 7.03 ± 0.02 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm6_4_4__hblt0__fa1__longctx16384__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 109.76 ± 0.00 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 6.48 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm6_4_4__hblt0__fa1__longctx32768__dual.log
+9
View File
@@ -0,0 +1,9 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to create context with model '/home/kyuz0/models/Q3_K_S/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002.gguf'
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
✖ ! [rocm6_4_4] Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__hblt0__fa1 __longctx32768 failed (exit 0)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm7.1.1-rocwmma__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 1 | pp512 | 236.00 ± 0.17 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 1 | tg128 | 7.01 ± 0.02 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm7.1.1-rocwmma__fa1__longctx16384__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 94.52 ± 0.00 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 6.09 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm7.1.1-rocwmma__fa1__longctx32768__dual.log
+9
View File
@@ -0,0 +1,9 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to create context with model '/home/kyuz0/models/Q3_K_S/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002.gguf'
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
✖ ! [rocm7.1.1-rocwmma] Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__fa1 __longctx32768 failed (exit 0)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm7.1.1-rocwmma__hblt0__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 1 | pp512 | 233.56 ± 0.08 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 1 | tg128 | 6.93 ± 0.01 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm7.1.1-rocwmma__hblt0__fa1__longctx16384__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 94.48 ± 0.00 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 6.09 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm7.1.1-rocwmma__hblt0__fa1__longctx32768__dual.log
+9
View File
@@ -0,0 +1,9 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to create context with model '/home/kyuz0/models/Q3_K_S/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002.gguf'
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
✖ ! [rocm7.1.1-rocwmma] Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__hblt0__fa1 __longctx32768 failed (exit 0)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm7.1.1__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 1 | pp512 | 235.60 ± 0.08 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 1 | tg128 | 6.93 ± 0.01 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm7.1.1__fa1__longctx16384__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 99.54 ± 0.00 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 6.47 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm7.1.1__fa1__longctx32768__dual.log
+9
View File
@@ -0,0 +1,9 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to create context with model '/home/kyuz0/models/Q3_K_S/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002.gguf'
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
✖ ! [rocm7.1.1] Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__fa1 __longctx32768 failed (exit 0)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm7.1.1__hblt0__fa1__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 1 | pp512 | 237.37 ± 0.19 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 1 | tg128 | 7.01 ± 0.02 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm7.1.1__hblt0__fa1__longctx16384__dual.log
+11
View File
@@ -0,0 +1,11 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 99.62 ± 0.00 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 6.45 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__rocm7.1.1__hblt0__fa1__longctx32768__dual.log
+9
View File
@@ -0,0 +1,9 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
Device 1: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to create context with model '/home/kyuz0/models/Q3_K_S/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002.gguf'
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
✖ ! [rocm7.1.1] Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__hblt0__fa1 __longctx32768 failed (exit 0)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__vulkan_amdvlk__fa1__dual.log
+12
View File
@@ -0,0 +1,12 @@
WARNING: radv is not a conformant Vulkan implementation, testing use only.
WARNING: radv is not a conformant Vulkan implementation, testing use only.
ggml_vulkan: Found 3 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon AI PRO R9700 (AMD open-source driver) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 32768 | int dot: 1 | matrix cores: KHR_coopmat
ggml_vulkan: 1 = AMD Radeon AI PRO R9700 (AMD open-source driver) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 32768 | int dot: 1 | matrix cores: KHR_coopmat
ggml_vulkan: 2 = AMD Ryzen 9 9900X3D 12-Core Processor (AMD open-source driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 32768 | int dot: 1 | matrix cores: none
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | Vulkan | 99 | 1 | pp512 | 133.82 ± 0.17 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | Vulkan | 99 | 1 | tg128 | 7.55 ± 0.01 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__vulkan_amdvlk__fa1__longctx16384__dual.log
+12
View File
@@ -0,0 +1,12 @@
WARNING: radv is not a conformant Vulkan implementation, testing use only.
WARNING: radv is not a conformant Vulkan implementation, testing use only.
ggml_vulkan: Found 3 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon AI PRO R9700 (AMD open-source driver) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 32768 | int dot: 1 | matrix cores: KHR_coopmat
ggml_vulkan: 1 = AMD Radeon AI PRO R9700 (AMD open-source driver) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 32768 | int dot: 1 | matrix cores: KHR_coopmat
ggml_vulkan: 2 = AMD Ryzen 9 9900X3D 12-Core Processor (AMD open-source driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 32768 | int dot: 1 | matrix cores: none
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | Vulkan | 99 | 1 | pp2048 @ d16384 | 44.89 ± 0.00 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | Vulkan | 99 | 1 | tg32 @ d16384 | 4.37 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__vulkan_amdvlk__fa1__longctx32768__dual.log
+12
View File
@@ -0,0 +1,12 @@
WARNING: radv is not a conformant Vulkan implementation, testing use only.
WARNING: radv is not a conformant Vulkan implementation, testing use only.
ggml_vulkan: Found 3 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon AI PRO R9700 (AMD open-source driver) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 32768 | int dot: 1 | matrix cores: KHR_coopmat
ggml_vulkan: 1 = AMD Radeon AI PRO R9700 (AMD open-source driver) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 32768 | int dot: 1 | matrix cores: KHR_coopmat
ggml_vulkan: 2 = AMD Ryzen 9 9900X3D 12-Core Processor (AMD open-source driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 32768 | int dot: 1 | matrix cores: none
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | Vulkan | 99 | 1 | pp2048 @ d32768 | 16.45 ± 0.00 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | Vulkan | 99 | 1 | tg32 @ d32768 | 3.13 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__vulkan_radv__fa1__dual.log
+12
View File
@@ -0,0 +1,12 @@
WARNING: radv is not a conformant Vulkan implementation, testing use only.
WARNING: radv is not a conformant Vulkan implementation, testing use only.
ggml_vulkan: Found 3 Vulkan devices:
ggml_vulkan: 0 = AMD Ryzen 9 9900X3D 12-Core Processor (RADV RAPHAEL_MENDOCINO) (radv) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 65536 | int dot: 1 | matrix cores: none
ggml_vulkan: 1 = AMD Radeon AI PRO R9700 (RADV GFX1201) (radv) | uma: 0 | fp16: 1 | bf16: 1 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
ggml_vulkan: 2 = AMD Radeon AI PRO R9700 (RADV GFX1201) (radv) | uma: 0 | fp16: 1 | bf16: 1 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | Vulkan | 99 | 1 | pp512 | 104.90 ± 0.13 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | Vulkan | 99 | 1 | tg128 | 8.35 ± 0.01 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__vulkan_radv__fa1__longctx16384__dual.log
+12
View File
@@ -0,0 +1,12 @@
WARNING: radv is not a conformant Vulkan implementation, testing use only.
WARNING: radv is not a conformant Vulkan implementation, testing use only.
ggml_vulkan: Found 3 Vulkan devices:
ggml_vulkan: 0 = AMD Ryzen 9 9900X3D 12-Core Processor (RADV RAPHAEL_MENDOCINO) (radv) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 65536 | int dot: 1 | matrix cores: none
ggml_vulkan: 1 = AMD Radeon AI PRO R9700 (RADV GFX1201) (radv) | uma: 0 | fp16: 1 | bf16: 1 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
ggml_vulkan: 2 = AMD Radeon AI PRO R9700 (RADV GFX1201) (radv) | uma: 0 | fp16: 1 | bf16: 1 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | Vulkan | 99 | 1 | pp2048 @ d16384 | 74.48 ± 0.00 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | Vulkan | 99 | 1 | tg32 @ d16384 | 6.44 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Devstral-2-123B-Instruct-2512-Q3_K_S-00001-of-00002__vulkan_radv__fa1__longctx32768__dual.log
+12
View File
@@ -0,0 +1,12 @@
WARNING: radv is not a conformant Vulkan implementation, testing use only.
WARNING: radv is not a conformant Vulkan implementation, testing use only.
ggml_vulkan: Found 3 Vulkan devices:
ggml_vulkan: 0 = AMD Ryzen 9 9900X3D 12-Core Processor (RADV RAPHAEL_MENDOCINO) (radv) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 65536 | int dot: 1 | matrix cores: none
ggml_vulkan: 1 = AMD Radeon AI PRO R9700 (RADV GFX1201) (radv) | uma: 0 | fp16: 1 | bf16: 1 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
ggml_vulkan: 2 = AMD Radeon AI PRO R9700 (RADV GFX1201) (radv) | uma: 0 | fp16: 1 | bf16: 1 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | Vulkan | 99 | 1 | pp2048 @ d32768 | 36.00 ± 0.00 |
| llama ?B Q3_K - Small | 50.63 GiB | 125.03 B | Vulkan | 99 | 1 | tg32 @ d32768 | 2.94 ± 0.00 |
build: 2aa45ef9e (7423)
docs/results.json
+2621 -1
View File
File diff suppressed because it is too large Load Diff