cjw/amd-r9700-ai-toolboxes
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

added ministral-3-8B

Browse Source
This commit is contained in:
Donato Capitella
2025-12-06 18:13:25 +00:00
parent 9fcc0eac1c
commit 2a783325f0
19 changed files with 1229 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm-7-nightly-rocwmma__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 4566.69 ± 9.35 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.17 ± 0.00 |
build: 6016d0bd4 (7285)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm-7-nightly-rocwmma__hblt0__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 624.69 ± 0.18 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.15 ± 0.01 |
build: 6016d0bd4 (7285)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm-7-nightly__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 4367.96 ± 32.60 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.22 ± 0.00 |
build: 6016d0bd4 (7285)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm-7-nightly__hblt0__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 618.44 ± 0.31 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.20 ± 0.01 |
build: 6016d0bd4 (7285)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm-7.9-rocwmma__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 4254.15 ± 13.82 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.42 ± 0.01 |
build: 6016d0bd4 (7285)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm-7.9-rocwmma__hblt0__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 599.01 ± 0.38 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.40 ± 0.02 |
build: 6016d0bd4 (7285)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm-7.9__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 4356.51 ± 12.09 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.44 ± 0.01 |
build: 6016d0bd4 (7285)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm-7.9__hblt0__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon Graphics, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 598.84 ± 0.40 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.41 ± 0.01 |
build: 6016d0bd4 (7285)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm6_4_4-rocwmma__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 3963.40 ± 14.39 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.44 ± 0.00 |
build: 6016d0bd4 (7285)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm6_4_4-rocwmma__hblt0__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 612.14 ± 0.41 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.43 ± 0.01 |
build: 6016d0bd4 (7285)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm6_4_4__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 4172.76 ± 21.37 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.48 ± 0.01 |
build: 6016d0bd4 (7285)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm6_4_4__hblt0__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 617.68 ± 0.32 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.49 ± 0.01 |
build: 6016d0bd4 (7285)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm7.1-rocwmma__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 4243.27 ± 9.54 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.40 ± 0.01 |
build: 6016d0bd4 (7285)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm7.1-rocwmma__hblt0__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 596.53 ± 0.31 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.39 ± 0.01 |
build: 6016d0bd4 (7285)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm7.1__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 4358.63 ± 10.50 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.46 ± 0.01 |
build: 6016d0bd4 (7285)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__rocm7.1__hblt0__fa1__single.log
+10
View File
@@ -0,0 +1,10 @@
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | pp512 | 598.95 ± 0.48 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | ROCm | 99 | 1 | tg128 | 31.45 ± 0.01 |
build: 6016d0bd4 (7285)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__vulkan_amdvlk__fa1__single.log
+12
View File
@@ -0,0 +1,12 @@
WARNING: radv is not a conformant Vulkan implementation, testing use only.
WARNING: radv is not a conformant Vulkan implementation, testing use only.
ggml_vulkan: Found 3 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon AI PRO R9700 (AMD open-source driver) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 32768 | int dot: 1 | matrix cores: KHR_coopmat
ggml_vulkan: 1 = AMD Radeon AI PRO R9700 (AMD open-source driver) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 32768 | int dot: 1 | matrix cores: KHR_coopmat
ggml_vulkan: 2 = AMD Ryzen 9 9900X3D 12-Core Processor (AMD open-source driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 32768 | int dot: 1 | matrix cores: none
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | Vulkan | 99 | 1 | pp512 | 874.94 ± 7.12 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | Vulkan | 99 | 1 | tg128 | 29.20 ± 0.09 |
build: 6016d0bd4 (7285)
benchmark/results/Ministral-3-8B-Instruct-2512-BF16__vulkan_radv__fa1__single.log
+12
View File
@@ -0,0 +1,12 @@
WARNING: radv is not a conformant Vulkan implementation, testing use only.
WARNING: radv is not a conformant Vulkan implementation, testing use only.
ggml_vulkan: Found 3 Vulkan devices:
ggml_vulkan: 0 = AMD Ryzen 9 9900X3D 12-Core Processor (RADV RAPHAEL_MENDOCINO) (radv) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 65536 | int dot: 1 | matrix cores: none
ggml_vulkan: 1 = AMD Radeon AI PRO R9700 (RADV GFX1201) (radv) | uma: 0 | fp16: 1 | bf16: 1 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
ggml_vulkan: 2 = AMD Radeon AI PRO R9700 (RADV GFX1201) (radv) | uma: 0 | fp16: 1 | bf16: 1 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -: | --------------: | -------------------: |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | Vulkan | 99 | 1 | pp512 | 1578.59 ± 104.98 |
| mistral3 8B BF16 | 15.81 GiB | 8.49 B | Vulkan | 99 | 1 | tg128 | 27.74 ± 0.07 |
build: 6016d0bd4 (7285)
docs/results.json
+1045 -1
View File
File diff suppressed because it is too large Load Diff