cjw/amd-r9700-ai-toolboxes
1
0
Fork 0
Code Issues Pull Requests Actions 6 Packages Projects Releases Wiki Activity

benchmakr updates

Browse Source
This commit is contained in:
Donato Capitella
2025-12-17 08:01:12 +00:00
parent 29da348bb8
commit 2e8bef127f
818 changed files with 30850 additions and 24154 deletions
Download Patch File Download Diff File Expand all files Collapse all files
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm-7-nightly-rocwmma__fa1__longctx16384__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 959.74 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 17.59 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 964.08 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 19.68 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm-7-nightly-rocwmma__fa1__longctx32768__single.log
+4 -2
View File
@@ -2,7 +2,9 @@ ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to create context with model '/home/kyuz0/models/Ministral-3-14B-Instruct-2512-BF16.gguf'
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
✖ ! [rocm-7-nightly-rocwmma] Ministral-3-14B-Instruct-2512-BF16__fa1 __longctx32768 failed (exit 0)
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 571.14 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 17.45 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm-7-nightly-rocwmma__fa1__single.log
+4 -14
View File
@@ -4,17 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
/opt/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:92: ROCm error
/usr/local/lib64/libggml-base.so.0(+0x3565) [0x7f3606f80565]
/usr/local/lib64/libggml-base.so.0(ggml_print_backtrace+0x1eb) [0x7f3606f8092b]
/usr/local/lib64/libggml-base.so.0(ggml_abort+0x11f) [0x7f3606f80aaf]
/usr/local/lib64/libggml-hip.so.0(+0x2cb5eb2) [0x7f3609cf2eb2]
/usr/local/lib64/libggml-hip.so.0(+0x2cbb07e) [0x7f3609cf807e]
/usr/local/lib64/libggml-base.so.0(ggml_backend_sched_synchronize+0x2e) [0x7f3606f97a1e]
/usr/local/lib64/libllama.so.0(_ZN13llama_context11synchronizeEv+0x10) [0x7f360a3bbd70]
/usr/local/bin/llama-bench() [0x40acfc]
/usr/local/bin/llama-bench() [0x408a7d]
/lib64/libc.so.6(+0x35b5) [0x7f36069165b5]
/lib64/libc.so.6(__libc_start_main+0x88) [0x7f3606916668]
/usr/local/bin/llama-bench() [0x409b65]
✖ ! [rocm-7-nightly-rocwmma] Ministral-3-14B-Instruct-2512-BF16__fa1 failed (exit 0)
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 2843.52 ± 16.68 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 22.18 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm-7-nightly-rocwmma__hblt0__fa1__longctx16384__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 305.03 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 17.59 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 305.90 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 19.69 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm-7-nightly-rocwmma__hblt0__fa1__longctx32768__single.log
+4 -2
View File
@@ -2,7 +2,9 @@ ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to create context with model '/home/kyuz0/models/Ministral-3-14B-Instruct-2512-BF16.gguf'
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
✖ ! [rocm-7-nightly-rocwmma] Ministral-3-14B-Instruct-2512-BF16__hblt0__fa1 __longctx32768 failed (exit 0)
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 252.57 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 17.46 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm-7-nightly-rocwmma__hblt0__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 382.20 ± 0.44 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 19.74 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 380.51 ± 0.29 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 22.17 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm-7-nightly__fa1__longctx16384__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 761.86 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 18.02 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 757.63 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 20.22 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm-7-nightly__fa1__longctx32768__single.log
+4 -2
View File
@@ -2,7 +2,9 @@ ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to create context with model '/home/kyuz0/models/Ministral-3-14B-Instruct-2512-BF16.gguf'
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
✖ ! [rocm-7-nightly] Ministral-3-14B-Instruct-2512-BF16__fa1 __longctx32768 failed (exit 0)
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 449.05 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 18.64 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm-7-nightly__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 2708.86 ± 8.75 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 19.75 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 2767.12 ± 15.29 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 22.21 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm-7-nightly__hblt0__fa1__longctx16384__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 280.13 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 18.01 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 279.80 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 20.23 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm-7-nightly__hblt0__fa1__longctx32768__single.log
+4 -2
View File
@@ -2,7 +2,9 @@ ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to create context with model '/home/kyuz0/models/Ministral-3-14B-Instruct-2512-BF16.gguf'
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
✖ ! [rocm-7-nightly] Ministral-3-14B-Instruct-2512-BF16__hblt0__fa1 __longctx32768 failed (exit 0)
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 222.90 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 18.62 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm-7-nightly__hblt0__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 376.62 ± 0.23 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 19.77 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 380.81 ± 0.21 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 22.21 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm-7.9-rocwmma__fa1__longctx16384__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 665.78 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 17.95 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 731.41 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 20.15 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm-7.9-rocwmma__fa1__longctx32768__single.log
+4 -2
View File
@@ -2,7 +2,9 @@ ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to create context with model '/home/kyuz0/models/Ministral-3-14B-Instruct-2512-BF16.gguf'
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
✖ ! [rocm-7.9-rocwmma] Ministral-3-14B-Instruct-2512-BF16__fa1 __longctx32768 failed (exit 0)
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 419.35 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 18.45 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm-7.9-rocwmma__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 2434.31 ± 160.01 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 19.82 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 2608.48 ± 2.57 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 22.28 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm-7.9-rocwmma__hblt0__fa1__longctx16384__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 262.83 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 17.95 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 271.63 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 20.15 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm-7.9-rocwmma__hblt0__fa1__longctx32768__single.log
+4 -2
View File
@@ -2,7 +2,9 @@ ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to create context with model '/home/kyuz0/models/Ministral-3-14B-Instruct-2512-BF16.gguf'
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
✖ ! [rocm-7.9-rocwmma] Ministral-3-14B-Instruct-2512-BF16__hblt0__fa1 __longctx32768 failed (exit 0)
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 213.61 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 18.44 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm-7.9-rocwmma__hblt0__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 366.55 ± 0.27 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 19.80 ± 0.07 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 368.45 ± 0.14 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 22.29 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm-7.9__fa1__longctx16384__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 767.43 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 18.07 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 761.85 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 20.30 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm-7.9__fa1__longctx32768__single.log
+4 -2
View File
@@ -2,7 +2,9 @@ ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to create context with model '/home/kyuz0/models/Ministral-3-14B-Instruct-2512-BF16.gguf'
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
✖ ! [rocm-7.9] Ministral-3-14B-Instruct-2512-BF16__fa1 __longctx32768 failed (exit 0)
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 447.32 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 18.69 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm-7.9__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 2606.42 ± 11.20 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 19.85 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 2644.29 ± 2.60 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 22.32 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm-7.9__hblt0__fa1__longctx16384__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 275.13 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 18.07 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 274.70 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 20.30 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm-7.9__hblt0__fa1__longctx32768__single.log
+4 -2
View File
@@ -2,7 +2,9 @@ ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to create context with model '/home/kyuz0/models/Ministral-3-14B-Instruct-2512-BF16.gguf'
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
✖ ! [rocm-7.9] Ministral-3-14B-Instruct-2512-BF16__hblt0__fa1 __longctx32768 failed (exit 0)
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 219.43 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 18.70 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm-7.9__hblt0__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 369.67 ± 0.28 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 19.86 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 370.52 ± 0.20 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 22.32 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm6_4_4-rocwmma__fa1__longctx16384__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 688.43 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 17.97 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 719.86 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 20.22 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm6_4_4-rocwmma__fa1__longctx32768__single.log
+4 -2
View File
@@ -2,7 +2,9 @@ ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to create context with model '/home/kyuz0/models/Ministral-3-14B-Instruct-2512-BF16.gguf'
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
✖ ! [rocm6_4_4-rocwmma] Ministral-3-14B-Instruct-2512-BF16__fa1 __longctx32768 failed (exit 0)
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 421.46 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 18.48 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm6_4_4-rocwmma__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 2312.05 ± 24.53 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 19.86 ± 0.03 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 2405.48 ± 3.53 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 22.36 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm6_4_4-rocwmma__hblt0__fa1__longctx16384__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 271.52 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 17.98 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 276.54 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 20.21 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm6_4_4-rocwmma__hblt0__fa1__longctx32768__single.log
+4 -2
View File
@@ -2,7 +2,9 @@ ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to create context with model '/home/kyuz0/models/Ministral-3-14B-Instruct-2512-BF16.gguf'
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
✖ ! [rocm6_4_4-rocwmma] Ministral-3-14B-Instruct-2512-BF16__hblt0__fa1 __longctx32768 failed (exit 0)
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 217.88 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 18.48 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm6_4_4-rocwmma__hblt0__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 377.52 ± 0.21 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 19.86 ± 0.02 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 378.57 ± 0.27 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 22.35 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm6_4_4__fa1__longctx16384__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 829.03 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 18.10 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 828.88 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 20.37 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm6_4_4__fa1__longctx32768__single.log
+4 -2
View File
@@ -2,7 +2,9 @@ ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to create context with model '/home/kyuz0/models/Ministral-3-14B-Instruct-2512-BF16.gguf'
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
✖ ! [rocm6_4_4] Ministral-3-14B-Instruct-2512-BF16__fa1 __longctx32768 failed (exit 0)
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 501.73 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 18.74 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm6_4_4__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 2402.00 ± 8.33 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 19.89 ± 0.02 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 2468.22 ± 5.84 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 22.39 ± 0.01 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm6_4_4__hblt0__fa1__longctx16384__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 290.54 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 18.11 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 290.77 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 20.36 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm6_4_4__hblt0__fa1__longctx32768__single.log
+4 -2
View File
@@ -2,7 +2,9 @@ ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to create context with model '/home/kyuz0/models/Ministral-3-14B-Instruct-2512-BF16.gguf'
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
✖ ! [rocm6_4_4] Ministral-3-14B-Instruct-2512-BF16__hblt0__fa1 __longctx32768 failed (exit 0)
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 237.21 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 18.76 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm6_4_4__hblt0__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 379.17 ± 0.14 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 19.90 ± 0.01 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 380.01 ± 0.30 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 22.38 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm7.1.1-mmf__fa1__single.log
-20
View File
@@ -1,20 +0,0 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
/opt/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:92: ROCm error
/usr/local/lib64/libggml-base.so.0(+0x3565) [0x7fd2c4f8e565]
/usr/local/lib64/libggml-base.so.0(ggml_print_backtrace+0x1eb) [0x7fd2c4f8e92b]
/usr/local/lib64/libggml-base.so.0(ggml_abort+0x11f) [0x7fd2c4f8eaaf]
/usr/local/lib64/libggml-hip.so.0(+0x300aef2) [0x7fd2c8055ef2]
/usr/local/lib64/libggml-hip.so.0(+0x30100fe) [0x7fd2c805b0fe]
/usr/local/lib64/libggml-base.so.0(ggml_backend_sched_synchronize+0x2e) [0x7fd2c4fa5a9e]
/usr/local/lib64/libllama.so.0(_ZN13llama_context11synchronizeEv+0x10) [0x7fd2c8722cf0]
/usr/local/bin/llama-bench() [0x40acfc]
/usr/local/bin/llama-bench() [0x408a7d]
/lib64/libc.so.6(+0x35b5) [0x7fd2c49245b5]
/lib64/libc.so.6(__libc_start_main+0x88) [0x7fd2c4924668]
/usr/local/bin/llama-bench() [0x409b65]
✖ ! [rocm7.1.1-mmf] Ministral-3-14B-Instruct-2512-BF16__fa1 failed (exit 0)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm7.1.1-rocwmma__fa1__longctx16384__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 665.80 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 17.95 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 731.32 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 20.15 ± 0.00 |
build: 22577583a (7312)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm7.1.1-rocwmma__fa1__longctx32768__single.log
+4 -2
View File
@@ -2,7 +2,9 @@ ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to create context with model '/home/kyuz0/models/Ministral-3-14B-Instruct-2512-BF16.gguf'
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
✖ ! [rocm7.1.1-rocwmma] Ministral-3-14B-Instruct-2512-BF16__fa1 __longctx32768 failed (exit 0)
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 419.39 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 18.44 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm7.1.1-rocwmma__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 2544.26 ± 1.34 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 19.82 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 2603.81 ± 5.27 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 22.28 ± 0.00 |
build: 22577583a (7312)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm7.1.1-rocwmma__hblt0__fa1__longctx16384__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 262.76 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 17.95 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 271.42 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 20.17 ± 0.00 |
build: 22577583a (7312)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm7.1.1-rocwmma__hblt0__fa1__longctx32768__single.log
+4 -2
View File
@@ -2,7 +2,9 @@ ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to create context with model '/home/kyuz0/models/Ministral-3-14B-Instruct-2512-BF16.gguf'
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
✖ ! [rocm7.1.1-rocwmma] Ministral-3-14B-Instruct-2512-BF16__hblt0__fa1 __longctx32768 failed (exit 0)
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 213.47 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 18.45 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm7.1.1-rocwmma__hblt0__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 367.12 ± 0.18 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 19.83 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 368.56 ± 0.28 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 22.29 ± 0.00 |
build: 22577583a (7312)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm7.1.1__fa1__longctx16384__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 766.84 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 18.09 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 756.68 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 20.32 ± 0.00 |
build: 22577583a (7312)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm7.1.1__fa1__longctx32768__single.log
+4 -2
View File
@@ -2,7 +2,9 @@ ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to create context with model '/home/kyuz0/models/Ministral-3-14B-Instruct-2512-BF16.gguf'
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
✖ ! [rocm7.1.1] Ministral-3-14B-Instruct-2512-BF16__fa1 __longctx32768 failed (exit 0)
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 447.71 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 18.69 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm7.1.1__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 2560.35 ± 5.29 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 19.82 ± 0.08 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 2649.88 ± 3.07 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 22.33 ± 0.01 |
build: 22577583a (7312)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm7.1.1__hblt0__fa1__longctx16384__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 274.75 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 18.07 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 274.54 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 20.31 ± 0.00 |
build: 22577583a (7312)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm7.1.1__hblt0__fa1__longctx32768__single.log
+4 -2
View File
@@ -2,7 +2,9 @@ ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
main: error: failed to create context with model '/home/kyuz0/models/Ministral-3-14B-Instruct-2512-BF16.gguf'
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
✖ ! [rocm7.1.1] Ministral-3-14B-Instruct-2512-BF16__hblt0__fa1 __longctx32768 failed (exit 0)
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 219.39 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 18.68 ± 0.00 |
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm7.1.1__hblt0__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 370.91 ± 0.02 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 19.86 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 369.68 ± 0.20 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 22.32 ± 0.00 |
build: 22577583a (7312)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__rocm7.1__hblt0__fa1__single.log
-10
View File
@@ -1,10 +0,0 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | pp512 | 376.47 ± 0.35 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | ROCm | 99 | 1 | tg128 | 19.87 ± 0.00 |
build: 6016d0bd4 (7285)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__vulkan_amdvlk__fa1__longctx16384__single.log
+3 -3
View File
@@ -6,7 +6,7 @@ ggml_vulkan: 1 = AMD Radeon AI PRO R9700 (AMD open-source driver) | uma: 0 | fp1
ggml_vulkan: 2 = AMD Ryzen 9 9900X3D 12-Core Processor (AMD open-source driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 32768 | int dot: 1 | matrix cores: none
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | Vulkan | 99 | 1 | pp2048 @ d16384 | 310.96 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | Vulkan | 99 | 1 | tg32 @ d16384 | 13.87 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | Vulkan | 99 | 1 | pp2048 @ d16384 | 287.06 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | Vulkan | 99 | 1 | tg32 @ d16384 | 15.15 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__vulkan_amdvlk__fa1__longctx32768__single.log
+3 -3
View File
@@ -6,7 +6,7 @@ ggml_vulkan: 1 = AMD Radeon AI PRO R9700 (AMD open-source driver) | uma: 0 | fp1
ggml_vulkan: 2 = AMD Ryzen 9 9900X3D 12-Core Processor (AMD open-source driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 32768 | int dot: 1 | matrix cores: none
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | Vulkan | 99 | 1 | pp2048 @ d32768 | 194.73 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | Vulkan | 99 | 1 | tg32 @ d32768 | 10.53 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | Vulkan | 99 | 1 | pp2048 @ d32768 | 126.59 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | Vulkan | 99 | 1 | tg32 @ d32768 | 12.48 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__vulkan_amdvlk__fa1__single.log
+3 -3
View File
@@ -6,7 +6,7 @@ ggml_vulkan: 1 = AMD Radeon AI PRO R9700 (AMD open-source driver) | uma: 0 | fp1
ggml_vulkan: 2 = AMD Ryzen 9 9900X3D 12-Core Processor (AMD open-source driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 32768 | int dot: 1 | matrix cores: none
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | Vulkan | 99 | 1 | pp512 | 540.84 ± 0.82 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | Vulkan | 99 | 1 | tg128 | 18.79 ± 0.02 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | Vulkan | 99 | 1 | pp512 | 528.53 ± 0.15 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | Vulkan | 99 | 1 | tg128 | 20.18 ± 0.02 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__vulkan_radv__fa1__longctx16384__single.log
+3 -3
View File
@@ -6,7 +6,7 @@ ggml_vulkan: 1 = AMD Radeon AI PRO R9700 (RADV GFX1201) (radv) | uma: 0 | fp16:
ggml_vulkan: 2 = AMD Radeon AI PRO R9700 (RADV GFX1201) (radv) | uma: 0 | fp16: 1 | bf16: 1 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | Vulkan | 99 | 1 | pp2048 @ d16384 | 550.05 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | Vulkan | 99 | 1 | tg32 @ d16384 | 15.14 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | Vulkan | 99 | 1 | pp2048 @ d16384 | 531.35 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | Vulkan | 99 | 1 | tg32 @ d16384 | 16.57 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__vulkan_radv__fa1__longctx32768__single.log
+3 -3
View File
@@ -6,7 +6,7 @@ ggml_vulkan: 1 = AMD Radeon AI PRO R9700 (RADV GFX1201) (radv) | uma: 0 | fp16:
ggml_vulkan: 2 = AMD Radeon AI PRO R9700 (RADV GFX1201) (radv) | uma: 0 | fp16: 1 | bf16: 1 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | Vulkan | 99 | 1 | pp2048 @ d32768 | 351.12 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | Vulkan | 99 | 1 | tg32 @ d32768 | 13.32 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | Vulkan | 99 | 1 | pp2048 @ d32768 | 320.22 ± 0.00 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | Vulkan | 99 | 1 | tg32 @ d32768 | 14.81 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-14B-Instruct-2512-BF16__vulkan_radv__fa1__single.log
+3 -3
View File
@@ -6,7 +6,7 @@ ggml_vulkan: 1 = AMD Radeon AI PRO R9700 (RADV GFX1201) (radv) | uma: 0 | fp16:
ggml_vulkan: 2 = AMD Radeon AI PRO R9700 (RADV GFX1201) (radv) | uma: 0 | fp16: 1 | bf16: 1 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | Vulkan | 99 | 1 | pp512 | 930.96 ± 16.21 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | Vulkan | 99 | 1 | tg128 | 17.68 ± 0.05 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | Vulkan | 99 | 1 | pp512 | 939.26 ± 0.80 |
| mistral3 14B BF16 | 25.16 GiB | 13.51 B | Vulkan | 99 | 1 | tg128 | 19.38 ± 0.05 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm-7-nightly-rocwmma__fa1__longctx16384__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 1210.41 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 26.73 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 1239.59 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 29.50 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm-7-nightly-rocwmma__fa1__longctx32768__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 708.36 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 22.94 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 704.24 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 25.34 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm-7-nightly-rocwmma__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 4566.69 ± 9.35 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.17 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 4617.19 ± 21.94 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 34.48 ± 0.02 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm-7-nightly-rocwmma__hblt0__fa1__longctx16384__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 461.89 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 26.74 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 461.59 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 29.51 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm-7-nightly-rocwmma__hblt0__fa1__longctx32768__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 363.06 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 22.91 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 362.20 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 25.36 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm-7-nightly-rocwmma__hblt0__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 624.69 ± 0.18 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.15 ± 0.01 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 621.97 ± 0.44 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 34.46 ± 0.01 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm-7-nightly__fa1__longctx16384__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 958.03 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 27.59 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 946.86 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 30.49 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm-7-nightly__fa1__longctx32768__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 552.93 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 24.80 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 539.19 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 27.49 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm-7-nightly__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 4367.96 ± 32.60 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.22 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 4439.20 ± 15.47 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 34.52 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm-7-nightly__hblt0__fa1__longctx16384__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 413.39 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 27.58 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 411.98 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 30.51 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm-7-nightly__hblt0__fa1__longctx32768__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 313.00 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 24.79 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 311.32 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 27.48 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm-7-nightly__hblt0__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 618.44 ± 0.31 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.20 ± 0.01 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 618.12 ± 0.55 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 34.52 ± 0.01 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm-7.9-rocwmma__fa1__longctx16384__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 832.67 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 27.47 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 918.44 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 30.49 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm-7.9-rocwmma__fa1__longctx32768__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 459.52 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 24.45 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 511.00 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 27.26 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm-7.9-rocwmma__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 4254.15 ± 13.82 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.42 ± 0.01 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 4416.28 ± 7.71 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 34.89 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm-7.9-rocwmma__hblt0__fa1__longctx16384__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 384.05 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 27.46 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 399.12 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 30.49 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm-7.9-rocwmma__hblt0__fa1__longctx32768__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 282.23 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 24.46 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 297.49 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 27.25 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm-7.9-rocwmma__hblt0__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 599.01 ± 0.38 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.40 ± 0.02 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 599.12 ± 0.34 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 34.89 ± 0.01 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm-7.9__fa1__longctx16384__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 963.98 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 27.76 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 954.53 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 30.76 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm-7.9__fa1__longctx32768__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 552.83 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 24.91 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 546.36 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 27.69 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm-7.9__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 4356.51 ± 12.09 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.44 ± 0.01 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 4486.85 ± 9.18 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 34.89 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm-7.9__hblt0__fa1__longctx16384__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 406.44 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 27.71 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 404.63 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 30.79 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm-7.9__hblt0__fa1__longctx32768__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 308.45 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 24.90 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 307.12 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 27.68 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm-7.9__hblt0__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 598.84 ± 0.40 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.41 ± 0.01 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 601.42 ± 0.05 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 34.92 ± 0.01 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm6_4_4-rocwmma__fa1__longctx16384__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 868.42 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 27.51 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 909.65 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 30.59 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm6_4_4-rocwmma__fa1__longctx32768__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 494.21 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 24.48 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 514.42 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 27.32 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm6_4_4-rocwmma__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 3963.40 ± 14.39 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.44 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 4062.50 ± 9.92 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 34.98 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm6_4_4-rocwmma__hblt0__fa1__longctx16384__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 397.69 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 27.47 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 406.07 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 30.58 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm6_4_4-rocwmma__hblt0__fa1__longctx32768__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 295.42 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 24.47 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 303.68 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 27.29 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm6_4_4-rocwmma__hblt0__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 612.14 ± 0.41 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.43 ± 0.01 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 615.80 ± 0.35 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 34.96 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm6_4_4__fa1__longctx16384__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 1058.69 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 27.78 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 1061.21 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 30.86 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm6_4_4__fa1__longctx32768__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 621.76 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 24.94 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 618.25 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 27.77 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm6_4_4__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 4172.76 ± 21.37 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.48 ± 0.01 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 4243.57 ± 22.42 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 35.08 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm6_4_4__hblt0__fa1__longctx16384__single.log
+2 -2
View File
@@ -5,6 +5,6 @@ ggml_cuda_init: found 1 ROCm devices:
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 433.90 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 27.78 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 30.85 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm6_4_4__hblt0__fa1__longctx32768__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 336.61 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 24.93 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 335.46 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 27.79 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm6_4_4__hblt0__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 617.68 ± 0.32 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.49 ± 0.01 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 620.01 ± 0.14 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 35.01 ± 0.00 |
build: 6016d0bd4 (7285)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm7.1-rocwmma__fa1__single.log
-10
View File
@@ -1,10 +0,0 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 4243.27 ± 9.54 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.40 ± 0.01 |
build: 6016d0bd4 (7285)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm7.1-rocwmma__hblt0__fa1__single.log
-10
View File
@@ -1,10 +0,0 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 596.53 ± 0.31 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.39 ± 0.01 |
build: 6016d0bd4 (7285)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm7.1.1-mmf__fa1__single.log
-10
View File
@@ -1,10 +0,0 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 4352.84 ± 10.67 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.44 ± 0.01 |
build: 22577583a (7312)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm7.1.1-mmf__hblt0__fa1__single.log
-10
View File
@@ -1,10 +0,0 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 602.48 ± 0.50 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.44 ± 0.01 |
build: 22577583a (7312)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm7.1.1-rocwmma__fa1__longctx16384__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 832.53 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 27.47 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 918.50 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 30.52 ± 0.00 |
build: 22577583a (7312)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm7.1.1-rocwmma__fa1__longctx32768__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 463.80 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 24.48 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d32768 | 512.14 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d32768 | 27.23 ± 0.00 |
build: 22577583a (7312)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm7.1.1-rocwmma__fa1__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 4238.60 ± 12.37 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.39 ± 0.01 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 4416.42 ± 3.69 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 34.93 ± 0.01 |
build: 22577583a (7312)
build: 2aa45ef9e (7423)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm7.1.1-rocwmma__hblt0__fa1__longctx16384__single.log
+3 -3
View File
@@ -4,7 +4,7 @@ ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | n_ubatch | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 383.90 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 27.46 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | pp2048 @ d16384 | 399.02 ± 0.00 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 2048 | 1 | tg32 @ d16384 | 30.51 ± 0.00 |
build: 22577583a (7312)
build: 2aa45ef9e (7423)

Some files were not shown because too many files have changed in this diff Show More