2571 lines
370 KiB
Plaintext
2571 lines
370 KiB
Plaintext
|
Skipping import of cpp extensions due to incompatible torch version 2.10.0a0+rocm7.11.0a20251210 for torchao version 0.14.1 Please see https://github.com/pytorch/ao/issues/2919 for more info
|
||
|
|
WARNING 12-19 17:35:32 [attention.py:82] Using VLLM_V1_USE_PREFILL_DECODE_ATTENTION environment variable is deprecated and will be removed in v0.14.0 or v1.0.0, whichever is soonest. Please use --attention-config.use_prefill_decode_attention command line argument or AttentionConfig(use_prefill_decode_attention=...) config field instead.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:35:32 [api_server.py:1351] vLLM API server version 0.13.0rc2.dev112+g763963aa7.d20251213
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:35:32 [utils.py:253] non-default args: {'model_tag': 'RedHatAI/gemma-3-27b-it-FP8-dynamic', 'host': '127.0.0.1', 'model': 'RedHatAI/gemma-3-27b-it-FP8-dynamic', 'trust_remote_code': True, 'max_model_len': 28900, 'tensor_parallel_size': 2, 'gpu_memory_utilization': 0.94, 'max_num_seqs': 32}
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:35:36 [model.py:514] Resolved architecture: Gemma3ForConditionalGeneration
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:35:36 [model.py:1636] Using max model len 28900
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:35:36 [scheduler.py:228] Chunked prefill is enabled with max_num_batched_tokens=2048.
|
||
|
|
Skipping import of cpp extensions due to incompatible torch version 2.10.0a0+rocm7.11.0a20251210 for torchao version 0.14.1 Please see https://github.com/pytorch/ao/issues/2919 for more info
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m INFO 12-19 17:35:41 [core.py:93] Initializing a V1 LLM engine (v0.13.0rc2.dev112+g763963aa7.d20251213) with config: model='RedHatAI/gemma-3-27b-it-FP8-dynamic', speculative_config=None, tokenizer='RedHatAI/gemma-3-27b-it-FP8-dynamic', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=28900, download_dir=None, load_format=auto, tensor_parallel_size=2, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=True, quantization=compressed-tensors, enforce_eager=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False), seed=0, served_model_name=RedHatAI/gemma-3-27b-it-FP8-dynamic, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'splitting_ops': ['vllm::unified_attention', 'vllm::unified_attention_with_output', 'vllm::unified_mla_attention', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::linear_attention', 'vllm::plamo2_mamba_mixer', 'vllm::gdn_attention_core', 'vllm::kda_attention', 'vllm::sparse_attn_indexer'], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [2048], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': True, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False}, 'max_cudagraph_capture_size': 64, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False}, 'local_cache_dir': None}
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m WARNING 12-19 17:35:41 [multiproc_executor.py:884] Reducing Torch parallelism from 24 threads to 1 to avoid unnecessary CPU contention. Set OMP_NUM_THREADS in the external environment to tune this value as needed.
|
||
|
|
Skipping import of cpp extensions due to incompatible torch version 2.10.0a0+rocm7.11.0a20251210 for torchao version 0.14.1 Please see https://github.com/pytorch/ao/issues/2919 for more info
|
||
|
|
Skipping import of cpp extensions due to incompatible torch version 2.10.0a0+rocm7.11.0a20251210 for torchao version 0.14.1 Please see https://github.com/pytorch/ao/issues/2919 for more info
|
||
|
|
INFO 12-19 17:35:46 [parallel_state.py:1203] world_size=2 rank=0 local_rank=0 distributed_init_method=tcp://127.0.0.1:33879 backend=nccl
|
||
|
|
INFO 12-19 17:35:46 [parallel_state.py:1203] world_size=2 rank=1 local_rank=1 distributed_init_method=tcp://127.0.0.1:33879 backend=nccl
|
||
|
|
INFO 12-19 17:35:46 [pynccl.py:111] vLLM is using nccl==2.27.3
|
||
|
|
INFO 12-19 17:35:46 [parallel_state.py:1411] rank 0 in world size 2 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank 0
|
||
|
|
INFO 12-19 17:35:46 [parallel_state.py:1411] rank 1 in world size 2 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 1, EP rank 1
|
||
|
|
Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
|
||
|
|
Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
|
||
|
|
[0;36m(Worker_TP0 pid=77933)[0;0m INFO 12-19 17:35:52 [gpu_model_runner.py:3562] Starting to load model RedHatAI/gemma-3-27b-it-FP8-dynamic...
|
||
|
|
[0;36m(Worker_TP0 pid=77933)[0;0m WARNING 12-19 17:35:53 [compressed_tensors.py:742] Acceleration for non-quantized schemes is not supported by Compressed Tensors. Falling back to UnquantizedLinearMethod
|
||
|
|
[0;36m(Worker_TP0 pid=77933)[0;0m INFO 12-19 17:35:53 [layer.py:537] Using AttentionBackendEnum.TORCH_SDPA for MultiHeadAttention in multimodal encoder.
|
||
|
|
[0;36m(Worker_TP0 pid=77933)[0;0m WARNING 12-19 17:35:53 [activation.py:544] [ROCm] PyTorch's native GELU with tanh approximation is unstable. Falling back to GELU(approximate='none').
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m WARNING 12-19 17:35:53 [compressed_tensors.py:742] Acceleration for non-quantized schemes is not supported by Compressed Tensors. Falling back to UnquantizedLinearMethod
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m INFO 12-19 17:35:53 [layer.py:537] Using AttentionBackendEnum.TORCH_SDPA for MultiHeadAttention in multimodal encoder.
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m WARNING 12-19 17:35:53 [activation.py:544] [ROCm] PyTorch's native GELU with tanh approximation is unstable. Falling back to GELU(approximate='none').
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m INFO 12-19 17:35:53 [rocm.py:306] Using Rocm Attention backend on V1 engine.
|
||
|
|
[0;36m(Worker_TP0 pid=77933)[0;0m INFO 12-19 17:35:53 [rocm.py:306] Using Rocm Attention backend on V1 engine.
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m WARNING 12-19 17:35:53 [activation.py:220] [ROCm] PyTorch's native GELU with tanh approximation is unstable with torch.compile. For native implementation, fallback to 'none' approximation. The custom kernel implementation is unaffected.
|
||
|
|
[0;36m(Worker_TP0 pid=77933)[0;0m WARNING 12-19 17:35:53 [activation.py:220] [ROCm] PyTorch's native GELU with tanh approximation is unstable with torch.compile. For native implementation, fallback to 'none' approximation. The custom kernel implementation is unaffected.
|
||
|
|
[0;36m(Worker_TP0 pid=77933)[0;0m
|
||
|
|
Loading safetensors checkpoint shards: 0% Completed | 0/6 [00:00<?, ?it/s]
|
||
|
|
[0;36m(Worker_TP0 pid=77933)[0;0m
|
||
|
|
Loading safetensors checkpoint shards: 17% Completed | 1/6 [00:01<00:05, 1.06s/it]
|
||
|
|
[0;36m(Worker_TP0 pid=77933)[0;0m
|
||
|
|
Loading safetensors checkpoint shards: 33% Completed | 2/6 [00:02<00:04, 1.09s/it]
|
||
|
|
[0;36m(Worker_TP0 pid=77933)[0;0m
|
||
|
|
Loading safetensors checkpoint shards: 50% Completed | 3/6 [00:03<00:03, 1.10s/it]
|
||
|
|
[0;36m(Worker_TP0 pid=77933)[0;0m
|
||
|
|
Loading safetensors checkpoint shards: 67% Completed | 4/6 [00:04<00:02, 1.05s/it]
|
||
|
|
[0;36m(Worker_TP0 pid=77933)[0;0m
|
||
|
|
Loading safetensors checkpoint shards: 83% Completed | 5/6 [00:05<00:01, 1.04s/it]
|
||
|
|
[0;36m(Worker_TP0 pid=77933)[0;0m
|
||
|
|
Loading safetensors checkpoint shards: 100% Completed | 6/6 [00:06<00:00, 1.12s/it]
|
||
|
|
[0;36m(Worker_TP0 pid=77933)[0;0m
|
||
|
|
Loading safetensors checkpoint shards: 100% Completed | 6/6 [00:06<00:00, 1.09s/it]
|
||
|
|
[0;36m(Worker_TP0 pid=77933)[0;0m
|
||
|
|
[0;36m(Worker_TP0 pid=77933)[0;0m INFO 12-19 17:36:00 [default_loader.py:308] Loading weights took 6.62 seconds
|
||
|
|
[0;36m(Worker_TP0 pid=77933)[0;0m INFO 12-19 17:36:01 [gpu_model_runner.py:3659] Model loading took 14.2812 GiB memory and 7.486143 seconds
|
||
|
|
[0;36m(Worker_TP0 pid=77933)[0;0m INFO 12-19 17:36:01 [gpu_model_runner.py:4446] Encoder cache will be initialized with a budget of 2048 tokens, and profiled with 7 image items of the maximum feature size.
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m INFO 12-19 17:36:01 [gpu_model_runner.py:4446] Encoder cache will be initialized with a budget of 2048 tokens, and profiled with 7 image items of the maximum feature size.
|
||
|
|
[0;36m(Worker_TP0 pid=77933)[0;0m INFO 12-19 17:36:12 [backends.py:634] Using cache directory: /home/kyuz0/.cache/vllm/torch_compile_cache/2b20f74ec3/rank_0_0/backbone for vLLM's torch.compile
|
||
|
|
[0;36m(Worker_TP0 pid=77933)[0;0m INFO 12-19 17:36:12 [backends.py:694] Dynamo bytecode transform time: 8.90 s
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m INFO 12-19 17:36:17 [backends.py:261] Cache the graph of compile range (1, 2048) for later use
|
||
|
|
[0;36m(Worker_TP0 pid=77933)[0;0m INFO 12-19 17:36:17 [backends.py:261] Cache the graph of compile range (1, 2048) for later use
|
||
|
|
[0;36m(Worker_TP0 pid=77933)[0;0m INFO 12-19 17:36:45 [backends.py:278] Compiling a graph for compile range (1, 2048) takes 29.01 s
|
||
|
|
[0;36m(Worker_TP0 pid=77933)[0;0m INFO 12-19 17:36:45 [monitor.py:34] torch.compile takes 37.90 s in total
|
||
|
|
[0;36m(Worker_TP0 pid=77933)[0;0m INFO 12-19 17:36:49 [gpu_worker.py:375] Available KV cache memory: 14.58 GiB
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m WARNING 12-19 17:36:50 [kv_cache_utils.py:1033] Add 8 padding layers, may waste at most 15.38% KV cache memory
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m INFO 12-19 17:36:50 [kv_cache_utils.py:1291] GPU KV cache size: 54,464 tokens
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m INFO 12-19 17:36:50 [kv_cache_utils.py:1296] Maximum concurrency for 28,900 tokens per request: 8.04x
|
||
|
|
[0;36m(Worker_TP0 pid=77933)[0;0m
|
||
|
|
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/11 [00:00<?, ?it/s]
|
||
|
|
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 9%|▉ | 1/11 [00:00<00:05, 1.83it/s]
|
||
|
|
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 18%|█▊ | 2/11 [00:01<00:04, 1.86it/s]
|
||
|
|
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 27%|██▋ | 3/11 [00:01<00:04, 1.88it/s]
|
||
|
|
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 36%|███▋ | 4/11 [00:02<00:03, 1.89it/s]
|
||
|
|
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 45%|████▌ | 5/11 [00:02<00:03, 1.92it/s]
|
||
|
|
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 55%|█████▍ | 6/11 [00:03<00:02, 1.93it/s]
|
||
|
|
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 64%|██████▎ | 7/11 [00:03<00:02, 1.95it/s]
|
||
|
|
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 73%|███████▎ | 8/11 [00:04<00:01, 1.96it/s]
|
||
|
|
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 82%|████████▏ | 9/11 [00:04<00:01, 1.97it/s]
|
||
|
|
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 91%|█████████ | 10/11 [00:05<00:00, 1.98it/s]
|
||
|
|
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 100%|██████████| 11/11 [00:05<00:00, 1.97it/s]
|
||
|
|
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 100%|██████████| 11/11 [00:05<00:00, 1.94it/s]
|
||
|
|
[0;36m(Worker_TP0 pid=77933)[0;0m
|
||
|
|
Capturing CUDA graphs (decode, FULL): 0%| | 0/7 [00:00<?, ?it/s]
|
||
|
|
Capturing CUDA graphs (decode, FULL): 14%|█▍ | 1/7 [00:00<00:03, 1.77it/s]
|
||
|
|
Capturing CUDA graphs (decode, FULL): 29%|██▊ | 2/7 [00:01<00:02, 1.89it/s]
|
||
|
|
Capturing CUDA graphs (decode, FULL): 43%|████▎ | 3/7 [00:01<00:02, 1.93it/s]
|
||
|
|
Capturing CUDA graphs (decode, FULL): 57%|█████▋ | 4/7 [00:02<00:01, 1.96it/s]
|
||
|
|
Capturing CUDA graphs (decode, FULL): 71%|███████▏ | 5/7 [00:02<00:01, 1.97it/s]
|
||
|
|
Capturing CUDA graphs (decode, FULL): 86%|████████▌ | 6/7 [00:03<00:00, 1.98it/s]
|
||
|
|
Capturing CUDA graphs (decode, FULL): 100%|██████████| 7/7 [00:03<00:00, 1.98it/s]
|
||
|
|
Capturing CUDA graphs (decode, FULL): 100%|██████████| 7/7 [00:03<00:00, 1.95it/s]
|
||
|
|
[0;36m(Worker_TP0 pid=77933)[0;0m INFO 12-19 17:37:00 [gpu_model_runner.py:4610] Graph capturing finished in 10 secs, took 1.58 GiB
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m INFO 12-19 17:37:00 [core.py:259] init engine (profile, create kv cache, warmup model) took 59.41 seconds
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [api_server.py:1099] Supported tasks: ['generate']
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [api_server.py:1425] Starting vLLM API server 0 on http://127.0.0.1:8000
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:38] Available routes are:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /openapi.json, Methods: GET, HEAD
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /docs, Methods: GET, HEAD
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /docs/oauth2-redirect, Methods: GET, HEAD
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /redoc, Methods: GET, HEAD
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /scale_elastic_ep, Methods: POST
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /is_scaling_elastic_ep, Methods: POST
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /tokenize, Methods: POST
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /detokenize, Methods: POST
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /inference/v1/generate, Methods: POST
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /pause, Methods: POST
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /resume, Methods: POST
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /is_paused, Methods: GET
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /metrics, Methods: GET
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /health, Methods: GET
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /load, Methods: GET
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /v1/models, Methods: GET
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /version, Methods: GET
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /v1/responses, Methods: POST
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /v1/responses/{response_id}, Methods: GET
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /v1/responses/{response_id}/cancel, Methods: POST
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /v1/messages, Methods: POST
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /v1/chat/completions, Methods: POST
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /v1/completions, Methods: POST
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /v1/audio/transcriptions, Methods: POST
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /v1/audio/translations, Methods: POST
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /ping, Methods: GET
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /ping, Methods: POST
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /invocations, Methods: POST
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /classify, Methods: POST
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /v1/embeddings, Methods: POST
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /score, Methods: POST
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /v1/score, Methods: POST
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /rerank, Methods: POST
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /v1/rerank, Methods: POST
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /v2/rerank, Methods: POST
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:07 [launcher.py:46] Route: /pooling, Methods: POST
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: Started server process [77686]
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: Waiting for application startup.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: Application startup complete.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:54918 - "GET /v1/models HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:33210 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:33210 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:34404 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:28 [loggers.py:248] Engine 000: Avg prompt throughput: 4.9 tokens/s, Avg generation throughput: 17.1 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.2%, Prefix cache hit rate: 0.0%, MM cache hit rate: 0.0%
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:34418 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:34432 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:34444 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:34448 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:33210 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:33210 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:33210 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:34444 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:38 [loggers.py:248] Engine 000: Avg prompt throughput: 140.6 tokens/s, Avg generation throughput: 98.1 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 2.1%, Prefix cache hit rate: 0.0%, MM cache hit rate: 0.0%
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:33210 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:34432 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:34418 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:34444 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:60172 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:60174 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:60184 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:60174 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:34432 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:48 [loggers.py:248] Engine 000: Avg prompt throughput: 196.2 tokens/s, Avg generation throughput: 125.9 tokens/s, Running: 8 reqs, Waiting: 0 reqs, GPU KV cache usage: 4.1%, Prefix cache hit rate: 0.0%, MM cache hit rate: 0.0%
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:34444 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:34444 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:60184 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:60104 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:60120 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:60128 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:60128 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:41430 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:37:58 [loggers.py:248] Engine 000: Avg prompt throughput: 341.8 tokens/s, Avg generation throughput: 153.9 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 10.0%, Prefix cache hit rate: 0.0%, MM cache hit rate: 0.0%
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:60104 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:60172 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:60128 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:60174 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:60128 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:60174 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:34418 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:33210 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:60120 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:34418 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:60172 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:38:08 [loggers.py:248] Engine 000: Avg prompt throughput: 202.2 tokens/s, Avg generation throughput: 194.0 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 8.7%, Prefix cache hit rate: 0.0%, MM cache hit rate: 0.0%
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:34432 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:60104 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:60174 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:34432 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:34404 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:60128 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:34432 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:34404 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] WorkerProc hit an exception.
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] Traceback (most recent call last):
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/executor/multiproc_executor.py", line 821, in worker_busy_loop
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] output = func(*args, **kwargs)
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] File "/opt/venv/lib64/python3.13/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] return func(*args, **kwargs)
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/worker/gpu_worker.py", line 572, in sample_tokens
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] return self.model_runner.sample_tokens(grammar_output)
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] File "/opt/venv/lib64/python3.13/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] return func(*args, **kwargs)
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/worker/gpu_model_runner.py", line 3281, in sample_tokens
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] ) = self._bookkeeping_sync(
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] ~~~~~~~~~~~~~~~~~~~~~~^
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] scheduler_output,
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] ^^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] ...<4 lines>...
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] spec_decode_metadata,
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] ^^^^^^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] )
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] ^
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/worker/gpu_model_runner.py", line 2625, in _bookkeeping_sync
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] valid_sampled_token_ids = self._to_list(sampled_token_ids)
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/worker/gpu_model_runner.py", line 5492, in _to_list
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] self.transfer_event.synchronize()
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] torch.AcceleratorError: HIP error: an illegal memory access was encountered
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] Search for `hipErrorIllegalAddress' in https://rocm.docs.amd.com/projects/HIP/en/latest/index.html for more information.
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] HIP kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] For debugging consider passing AMD_SERIALIZE_KERNEL=3
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] Compile with `TORCH_USE_HIP_DSA` to enable device-side assertions.
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826]
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] Traceback (most recent call last):
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/executor/multiproc_executor.py", line 821, in worker_busy_loop
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] output = func(*args, **kwargs)
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] File "/opt/venv/lib64/python3.13/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] return func(*args, **kwargs)
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/worker/gpu_worker.py", line 572, in sample_tokens
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] return self.model_runner.sample_tokens(grammar_output)
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] File "/opt/venv/lib64/python3.13/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] return func(*args, **kwargs)
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/worker/gpu_model_runner.py", line 3281, in sample_tokens
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] ) = self._bookkeeping_sync(
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] ~~~~~~~~~~~~~~~~~~~~~~^
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] scheduler_output,
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] ^^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] ...<4 lines>...
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] spec_decode_metadata,
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] ^^^^^^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] )
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] ^
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/worker/gpu_model_runner.py", line 2625, in _bookkeeping_sync
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] valid_sampled_token_ids = self._to_list(sampled_token_ids)
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/worker/gpu_model_runner.py", line 5492, in _to_list
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] self.transfer_event.synchronize()
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] torch.AcceleratorError: HIP error: an illegal memory access was encountered
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] Search for `hipErrorIllegalAddress' in https://rocm.docs.amd.com/projects/HIP/en/latest/index.html for more information.
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] HIP kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] For debugging consider passing AMD_SERIALIZE_KERNEL=3
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826] Compile with `TORCH_USE_HIP_DSA` to enable device-side assertions.
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826]
|
||
|
|
[0;36m(Worker_TP1 pid=77934)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:826]
|
||
|
|
[rank1]:[E1219 17:38:16.877431190 ProcessGroupNCCL.cpp:2093] [PG ID 2 PG GUID 3 Rank 1] Process group watchdog thread terminated with exception: HIP error: an illegal memory access was encountered
|
||
|
|
Search for `hipErrorIllegalAddress' in https://rocm.docs.amd.com/projects/HIP/en/latest/index.html for more information.
|
||
|
|
HIP kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
|
||
|
|
For debugging consider passing AMD_SERIALIZE_KERNEL=3
|
||
|
|
Compile with `TORCH_USE_HIP_DSA` to enable device-side assertions.
|
||
|
|
|
||
|
|
Exception raised from getDevice at /__w/TheRock/TheRock/external-builds/pytorch/pytorch/aten/src/ATen/hip/impl/HIPGuardImplMasqueradingAsCUDA.h:75 (most recent call first):
|
||
|
|
frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) + 0x9d (0x7f01713fafdd in /opt/venv/lib64/python3.13/site-packages/torch/lib/libc10.so)
|
||
|
|
frame #1: <unknown function> + 0x12568 (0x7f0171486568 in /opt/venv/lib64/python3.13/site-packages/torch/lib/libc10_hip.so)
|
||
|
|
frame #2: c10d::ProcessGroupNCCL::Watchdog::runLoop() + 0x922 (0x7f0174a7ccc2 in /opt/venv/lib64/python3.13/site-packages/torch/lib/libtorch_hip.so)
|
||
|
|
frame #3: c10d::ProcessGroupNCCL::Watchdog::run() + 0x105 (0x7f0174a7f1b5 in /opt/venv/lib64/python3.13/site-packages/torch/lib/libtorch_hip.so)
|
||
|
|
frame #4: <unknown function> + 0x4e3e4 (0x7f01dce603e4 in /lib64/libstdc++.so.6)
|
||
|
|
frame #5: <unknown function> + 0x72464 (0x7f01dd0fb464 in /lib64/libc.so.6)
|
||
|
|
frame #6: <unknown function> + 0xf55ac (0x7f01dd17e5ac in /lib64/libc.so.6)
|
||
|
|
|
||
|
|
terminate called after throwing an instance of 'c10::DistBackendError'
|
||
|
|
what(): [PG ID 2 PG GUID 3 Rank 1] Process group watchdog thread terminated with exception: HIP error: an illegal memory access was encountered
|
||
|
|
Search for `hipErrorIllegalAddress' in https://rocm.docs.amd.com/projects/HIP/en/latest/index.html for more information.
|
||
|
|
HIP kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
|
||
|
|
For debugging consider passing AMD_SERIALIZE_KERNEL=3
|
||
|
|
Compile with `TORCH_USE_HIP_DSA` to enable device-side assertions.
|
||
|
|
|
||
|
|
Exception raised from getDevice at /__w/TheRock/TheRock/external-builds/pytorch/pytorch/aten/src/ATen/hip/impl/HIPGuardImplMasqueradingAsCUDA.h:75 (most recent call first):
|
||
|
|
frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) + 0x9d (0x7f01713fafdd in /opt/venv/lib64/python3.13/site-packages/torch/lib/libc10.so)
|
||
|
|
frame #1: <unknown function> + 0x12568 (0x7f0171486568 in /opt/venv/lib64/python3.13/site-packages/torch/lib/libc10_hip.so)
|
||
|
|
frame #2: c10d::ProcessGroupNCCL::Watchdog::runLoop() + 0x922 (0x7f0174a7ccc2 in /opt/venv/lib64/python3.13/site-packages/torch/lib/libtorch_hip.so)
|
||
|
|
frame #3: c10d::ProcessGroupNCCL::Watchdog::run() + 0x105 (0x7f0174a7f1b5 in /opt/venv/lib64/python3.13/site-packages/torch/lib/libtorch_hip.so)
|
||
|
|
frame #4: <unknown function> + 0x4e3e4 (0x7f01dce603e4 in /lib64/libstdc++.so.6)
|
||
|
|
frame #5: <unknown function> + 0x72464 (0x7f01dd0fb464 in /lib64/libc.so.6)
|
||
|
|
frame #6: <unknown function> + 0xf55ac (0x7f01dd17e5ac in /lib64/libc.so.6)
|
||
|
|
|
||
|
|
Exception raised from run at /__w/TheRock/TheRock/external-builds/pytorch/pytorch/torch/csrc/distributed/c10d/ProcessGroupNCCL.cpp:2099 (most recent call first):
|
||
|
|
frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) + 0x9d (0x7f01713fafdd in /opt/venv/lib64/python3.13/site-packages/torch/lib/libc10.so)
|
||
|
|
frame #1: <unknown function> + 0xb51c0c (0x7f0172129c0c in /opt/venv/lib64/python3.13/site-packages/torch/lib/libtorch_hip.so)
|
||
|
|
frame #2: <unknown function> + 0x4e3e4 (0x7f01dce603e4 in /lib64/libstdc++.so.6)
|
||
|
|
frame #3: <unknown function> + 0x72464 (0x7f01dd0fb464 in /lib64/libc.so.6)
|
||
|
|
frame #4: <unknown function> + 0xf55ac (0x7f01dd17e5ac in /lib64/libc.so.6)
|
||
|
|
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:60172 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:16 [multiproc_executor.py:233] Worker proc VllmWorker-1 died unexpectedly, shutting down executor.
|
||
|
|
[0;36m(Worker_TP0 pid=77933)[0;0m INFO 12-19 17:38:16 [multiproc_executor.py:711] Parent process exited, terminating worker
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:59118 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:59124 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:59126 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO 12-19 17:38:18 [loggers.py:248] Engine 000: Avg prompt throughput: 172.0 tokens/s, Avg generation throughput: 150.8 tokens/s, Running: 11 reqs, Waiting: 0 reqs, GPU KV cache usage: 9.1%, Prefix cache hit rate: 0.0%, MM cache hit rate: 0.0%
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:59134 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m INFO: 127.0.0.1:59138 - "POST /v1/completions HTTP/1.1" 200 OK
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [dump_input.py:72] Dumping input data for V1 LLM engine (v0.13.0rc2.dev112+g763963aa7.d20251213) with config: model='RedHatAI/gemma-3-27b-it-FP8-dynamic', speculative_config=None, tokenizer='RedHatAI/gemma-3-27b-it-FP8-dynamic', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=28900, download_dir=None, load_format=auto, tensor_parallel_size=2, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=True, quantization=compressed-tensors, enforce_eager=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False), seed=0, served_model_name=RedHatAI/gemma-3-27b-it-FP8-dynamic, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'splitting_ops': ['vllm::unified_attention', 'vllm::unified_attention_with_output', 'vllm::unified_mla_attention', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::linear_attention', 'vllm::plamo2_mamba_mixer', 'vllm::gdn_attention_core', 'vllm::kda_attention', 'vllm::sparse_attn_indexer'], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [2048], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': True, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False}, 'max_cudagraph_capture_size': 64, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False}, 'local_cache_dir': None},
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [dump_input.py:79] Dumping scheduler output for model execution: SchedulerOutput(scheduled_new_reqs=[NewRequestData(req_id=cmpl-bench-0a333e52-45-0,prompt_token_ids_len=282,mm_features=[],sampling_params=SamplingParams(n=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.0, top_p=1.0, top_k=0, min_p=0.0, seed=None, stop=[], stop_token_ids=[106], bad_words=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=493, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None, structured_outputs=None, extra_args=None),block_ids=([7974, 7975, 7976, 7977, 7978, 7979, 7980, 7981, 7982, 7983, 7984, 7985, 7986, 7987, 7988, 7989, 7990, 7991], [7992, 7993, 7994, 7995, 7996, 7997, 7998, 7999, 8000, 8001, 8002, 8003, 8004, 8005, 8006, 8007, 8008, 8009], [8010, 8011, 8012, 8013, 8014, 8015, 8016, 8017, 8018, 8019, 8020, 8021, 8022, 8023, 8024, 8025, 8026, 8027], [8028, 8029, 8030, 8031, 8032, 8033, 8034, 8035, 8036, 8037, 8038, 8039, 8040, 8041, 8042, 8043, 8044, 8045], [8046, 8047, 8048, 8049, 8050, 8051, 8052, 8053, 8054, 8055, 8056, 8057, 8058, 8059, 8060, 8061, 8062, 8063], [8064, 8065, 8066, 8067, 8068, 8069, 8070, 8071, 8072, 8073, 8074, 8075, 8076, 8077, 8078, 8079, 8080, 8081], [8082, 8083, 8084, 8085, 8086, 8087, 8088, 8089, 8090, 8091, 8092, 8093, 8094, 8095, 8096, 8097, 8098, 8099]),num_computed_tokens=0,lora_request=None,prompt_embeds_shape=None)], scheduled_cached_reqs=CachedRequestData(req_ids=['cmpl-bench-0a333e52-5-0', 'cmpl-bench-0a333e52-20-0', 'cmpl-bench-0a333e52-21-0', 'cmpl-bench-0a333e52-26-0', 'cmpl-bench-0a333e52-34-0', 'cmpl-bench-0a333e52-35-0', 'cmpl-bench-0a333e52-36-0', 'cmpl-bench-0a333e52-39-0', 'cmpl-bench-0a333e52-40-0', 'cmpl-bench-0a333e52-43-0', 'cmpl-bench-0a333e52-44-0'], resumed_req_ids=[], new_token_ids=[], all_token_ids={}, new_block_ids=[null, null, null, null, null, null, null, null, null, null, null], num_computed_tokens=[877, 1009, 430, 726, 538, 213, 838, 87, 79, 47, 27], num_output_tokens=[849, 401, 372, 323, 184, 177, 159, 72, 63, 36, 21]), num_scheduled_tokens={cmpl-bench-0a333e52-40-0: 1, cmpl-bench-0a333e52-43-0: 1, cmpl-bench-0a333e52-44-0: 1, cmpl-bench-0a333e52-45-0: 282, cmpl-bench-0a333e52-39-0: 1, cmpl-bench-0a333e52-20-0: 1, cmpl-bench-0a333e52-5-0: 1, cmpl-bench-0a333e52-26-0: 1, cmpl-bench-0a333e52-21-0: 1, cmpl-bench-0a333e52-34-0: 1, cmpl-bench-0a333e52-36-0: 1, cmpl-bench-0a333e52-35-0: 1}, total_num_scheduled_tokens=293, scheduled_spec_decode_tokens={}, scheduled_encoder_inputs={}, num_common_prefix_blocks=[0, 0, 0, 0, 0, 0, 0], finished_req_ids=[], free_encoder_mm_hashes=[], preempted_req_ids=[], pending_structured_output_tokens=false, kv_connector_metadata=null, ec_connector_metadata=null)
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [dump_input.py:81] Dumping scheduler stats: SchedulerStats(num_running_reqs=12, num_waiting_reqs=0, step_counter=0, current_wave=0, kv_cache_usage=0.09605539236256821, prefix_cache_stats=PrefixCacheStats(reset=False, requests=1, queries=282, hits=0, preempted_requests=0, preempted_queries=0, preempted_hits=0), connector_prefix_cache_stats=None, kv_cache_eviction_events=[], spec_decoding_stats=None, kv_connector_stats=None, waiting_lora_adapters={}, running_lora_adapters={}, cudagraph_stats=None)
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] EngineCore encountered a fatal error.
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] Traceback (most recent call last):
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/core.py", line 859, in run_engine_core
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] engine_core.run_busy_loop()
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] ~~~~~~~~~~~~~~~~~~~~~~~~~^^
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/core.py", line 886, in run_busy_loop
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] self._process_engine_step()
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] ~~~~~~~~~~~~~~~~~~~~~~~~~^^
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/core.py", line 919, in _process_engine_step
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] outputs, model_executed = self.step_fn()
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] ~~~~~~~~~~~~^^
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/core.py", line 353, in step
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] model_output = self.model_executor.sample_tokens(grammar_output)
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/executor/multiproc_executor.py", line 271, in sample_tokens
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] return self.collective_rpc(
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] ~~~~~~~~~~~~~~~~~~~^
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] "sample_tokens",
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] ^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] ...<4 lines>...
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] kv_output_aggregator=self.kv_output_aggregator,
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] )
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] ^
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/executor/multiproc_executor.py", line 361, in collective_rpc
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] return aggregate(get_response())
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] ~~~~~~~~~~~~^^
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/executor/multiproc_executor.py", line 338, in get_response
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] status, result = mq.dequeue(
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] ~~~~~~~~~~^
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] timeout=dequeue_timeout, cancel=shutdown_event
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] )
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] ^
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] File "/opt/venv/lib64/python3.13/site-packages/vllm/distributed/device_communicators/shm_broadcast.py", line 616, in dequeue
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] with self.acquire_read(timeout, cancel, indefinite) as buf:
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] File "/usr/lib64/python3.13/contextlib.py", line 141, in __enter__
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] return next(self.gen)
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] File "/opt/venv/lib64/python3.13/site-packages/vllm/distributed/device_communicators/shm_broadcast.py", line 531, in acquire_read
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] raise RuntimeError("cancelled")
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ERROR 12-19 17:38:20 [core.py:868] RuntimeError: cancelled
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m Process EngineCore_DP0:
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m Traceback (most recent call last):
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m File "/usr/lib64/python3.13/multiprocessing/process.py", line 313, in _bootstrap
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m self.run()
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ~~~~~~~~^^
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m File "/usr/lib64/python3.13/multiprocessing/process.py", line 108, in run
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m self._target(*self._args, **self._kwargs)
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/core.py", line 870, in run_engine_core
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m raise e
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/core.py", line 859, in run_engine_core
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m engine_core.run_busy_loop()
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ~~~~~~~~~~~~~~~~~~~~~~~~~^^
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/core.py", line 886, in run_busy_loop
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m self._process_engine_step()
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ~~~~~~~~~~~~~~~~~~~~~~~~~^^
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/core.py", line 919, in _process_engine_step
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m outputs, model_executed = self.step_fn()
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ~~~~~~~~~~~~^^
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/core.py", line 353, in step
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m model_output = self.model_executor.sample_tokens(grammar_output)
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/executor/multiproc_executor.py", line 271, in sample_tokens
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m return self.collective_rpc(
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ~~~~~~~~~~~~~~~~~~~^
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m "sample_tokens",
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ...<4 lines>...
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m kv_output_aggregator=self.kv_output_aggregator,
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m )
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ^
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/executor/multiproc_executor.py", line 361, in collective_rpc
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m return aggregate(get_response())
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ~~~~~~~~~~~~^^
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/executor/multiproc_executor.py", line 338, in get_response
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m status, result = mq.dequeue(
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ~~~~~~~~~~^
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m timeout=dequeue_timeout, cancel=shutdown_event
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m )
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ^
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m File "/opt/venv/lib64/python3.13/site-packages/vllm/distributed/device_communicators/shm_broadcast.py", line 616, in dequeue
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m with self.acquire_read(timeout, cancel, indefinite) as buf:
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m File "/usr/lib64/python3.13/contextlib.py", line 141, in __enter__
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m return next(self.gen)
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m File "/opt/venv/lib64/python3.13/site-packages/vllm/distributed/device_communicators/shm_broadcast.py", line 531, in acquire_read
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m raise RuntimeError("cancelled")
|
||
|
|
[0;36m(EngineCore_DP0 pid=77851)[0;0m RuntimeError: cancelled
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [async_llm.py:538] AsyncLLM output_handler failed.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [async_llm.py:538] Traceback (most recent call last):
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [async_llm.py:538] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 490, in output_handler
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [async_llm.py:538] outputs = await engine_core.get_output_async()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [async_llm.py:538] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [async_llm.py:538] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/core_client.py", line 895, in get_output_async
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [async_llm.py:538] raise self._format_exception(outputs) from None
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [async_llm.py:538] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Error in completion stream generator.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Traceback (most recent call last):
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 490, in output_handler
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] outputs = await engine_core.get_output_async()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/core_client.py", line 895, in get_output_async
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise self._format_exception(outputs) from None
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Error in completion stream generator.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Traceback (most recent call last):
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 490, in output_handler
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] outputs = await engine_core.get_output_async()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/core_client.py", line 895, in get_output_async
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise self._format_exception(outputs) from None
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Error in completion stream generator.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Traceback (most recent call last):
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 490, in output_handler
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] outputs = await engine_core.get_output_async()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/core_client.py", line 895, in get_output_async
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise self._format_exception(outputs) from None
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Error in completion stream generator.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Traceback (most recent call last):
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 490, in output_handler
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] outputs = await engine_core.get_output_async()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/core_client.py", line 895, in get_output_async
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise self._format_exception(outputs) from None
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Error in completion stream generator.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Traceback (most recent call last):
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 490, in output_handler
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] outputs = await engine_core.get_output_async()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/core_client.py", line 895, in get_output_async
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise self._format_exception(outputs) from None
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Error in completion stream generator.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Traceback (most recent call last):
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 490, in output_handler
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] outputs = await engine_core.get_output_async()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/core_client.py", line 895, in get_output_async
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise self._format_exception(outputs) from None
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Error in completion stream generator.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Traceback (most recent call last):
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 490, in output_handler
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] outputs = await engine_core.get_output_async()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/core_client.py", line 895, in get_output_async
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise self._format_exception(outputs) from None
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Error in completion stream generator.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Traceback (most recent call last):
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 490, in output_handler
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] outputs = await engine_core.get_output_async()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/core_client.py", line 895, in get_output_async
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise self._format_exception(outputs) from None
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Error in completion stream generator.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Traceback (most recent call last):
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 490, in output_handler
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] outputs = await engine_core.get_output_async()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/core_client.py", line 895, in get_output_async
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise self._format_exception(outputs) from None
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Error in completion stream generator.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Traceback (most recent call last):
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 490, in output_handler
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] outputs = await engine_core.get_output_async()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/core_client.py", line 895, in get_output_async
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise self._format_exception(outputs) from None
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Error in completion stream generator.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Traceback (most recent call last):
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 490, in output_handler
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] outputs = await engine_core.get_output_async()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/core_client.py", line 895, in get_output_async
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise self._format_exception(outputs) from None
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Error in completion stream generator.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Traceback (most recent call last):
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 490, in output_handler
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] outputs = await engine_core.get_output_async()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/core_client.py", line 895, in get_output_async
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise self._format_exception(outputs) from None
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Error in completion stream generator.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Traceback (most recent call last):
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 490, in output_handler
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] outputs = await engine_core.get_output_async()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/core_client.py", line 895, in get_output_async
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise self._format_exception(outputs) from None
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Error in completion stream generator.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Traceback (most recent call last):
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 490, in output_handler
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] outputs = await engine_core.get_output_async()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/core_client.py", line 895, in get_output_async
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise self._format_exception(outputs) from None
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Error in completion stream generator.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Traceback (most recent call last):
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 490, in output_handler
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] outputs = await engine_core.get_output_async()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/core_client.py", line 895, in get_output_async
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise self._format_exception(outputs) from None
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Error in completion stream generator.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Traceback (most recent call last):
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 490, in output_handler
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] outputs = await engine_core.get_output_async()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/core_client.py", line 895, in get_output_async
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise self._format_exception(outputs) from None
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Error in completion stream generator.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Traceback (most recent call last):
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 490, in output_handler
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] outputs = await engine_core.get_output_async()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/core_client.py", line 895, in get_output_async
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise self._format_exception(outputs) from None
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Error in completion stream generator.
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] Traceback (most recent call last):
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] out = q.get_nowait() or await q.get()
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ^^^^^^^^^^^^^
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/output_processor.py", line 70, in get
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] raise output
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/entrypoints/openai/serving_completion.py", line 352, in completion_stream_generator
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for prompt_idx, res in result_generator:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] ...<125 lines>...
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield f"data: {response_json}\n\n"
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/utils/async_utils.py", line 278, in merge_async_iterators
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] async for item in iterators[0]:
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] yield 0, item
|
||
|
|
[0;36m(APIServer pid=77686)[0;0m ERROR 12-19 17:38:20 [serving_completion.py:513] File "/opt/venv/lib64/python3.13/site-packages/vllm/v1/engine/async_llm.py", line 436, in generate
|