x
This commit is contained in:
@@ -143,3 +143,14 @@ curl -X POST "http://localhost:8001/v1/chat/completions" \
|
||||
- 首次部署建议设置 `GPU_MEMORY_UTILIZATION=0.90`,稳定后再调高
|
||||
- 若模型较大且吞吐压力高,可逐步调低 `MAX_MODEL_LEN` 或 `MAX_NUM_SEQS`
|
||||
- 确保宿主机已正确安装 ROCm 驱动并暴露 `/dev/kfd` 与 `/dev/dri`
|
||||
|
||||
## 常见故障排查
|
||||
|
||||
- 报错 `model type qwen3_5_moe ... Transformers does not recognize this architecture` 时,先重建镜像以更新依赖:
|
||||
|
||||
```bash
|
||||
docker compose build --no-cache
|
||||
docker compose up -d --force-recreate
|
||||
```
|
||||
|
||||
- 若仍报相同错误,说明当前 `vLLM/Transformers` 组合不支持该模型架构,建议在 `config.json` 切到其他本地模型,或升级基础镜像到更新的 vLLM 版本。
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
fastapi==0.116.1
|
||||
uvicorn==0.35.0
|
||||
pydantic==2.11.7
|
||||
transformers>=4.57.0
|
||||
|
||||
Reference in New Issue
Block a user