This commit is contained in:
2026-03-29 05:57:45 +08:00
parent 23909c205e
commit d295adc300
2 changed files with 12 additions and 0 deletions
+11
View File
@@ -143,3 +143,14 @@ curl -X POST "http://localhost:8001/v1/chat/completions" \
- 首次部署建议设置 `GPU_MEMORY_UTILIZATION=0.90`,稳定后再调高
- 若模型较大且吞吐压力高,可逐步调低 `MAX_MODEL_LEN` 或 `MAX_NUM_SEQS`
- 确保宿主机已正确安装 ROCm 驱动并暴露 `/dev/kfd` 与 `/dev/dri`
## 常见故障排查
- 报错 `model type qwen3_5_moe ... Transformers does not recognize this architecture` 时,先重建镜像以更新依赖:
```bash
docker compose build --no-cache
docker compose up -d --force-recreate
```
- 若仍报相同错误,说明当前 `vLLM/Transformers` 组合不支持该模型架构,建议在 `config.json` 切到其他本地模型,或升级基础镜像到更新的 vLLM 版本。
+1
View File
@@ -1,3 +1,4 @@
fastapi==0.116.1
uvicorn==0.35.0
pydantic==2.11.7
transformers>=4.57.0