Qwen2.5-VL-32B-Instruct SYM_INT4 Launch Failure

[shawn9977]

HW: 4XB60

=================================================
Step 1: Launch a container
docker run --rm -dit
--privileged
--net=host
--device=/dev/dri
--name=lsv-container
-v /home/shawn/:/llm/shawn
-e no_proxy=localhost,127.0.0.1
-e http_proxy=$http_proxy
-e https_proxy=$https_proxy
--shm-size="32g"
--entrypoint /bin/bash
intel/llm-scaler-vllm:0.10.0-b1

Step2: Launch Qwen2.5-VL-32B-Instruct SYM_INT4 model failed

export VLLM_ALLOW_LONG_MAX_MODEL_LEN=1
export VLLM_WORKER_MULTIPROC_METHOD=spawn
export VLLM_OFFLOAD_WEIGHTS_BEFORE_QUANT=0

MODEL_NAME="Qwen2.5-VL-32B-Instruct"
MODEL_PATH="/llm/shawn/models/Qwen/${MODEL_NAME}"

python3 -m vllm.entrypoints.openai.api_server
--model "${MODEL_PATH}"
--served-model-name "${MODEL_NAME}"
--dtype=float16
--enforce-eager
--port 8000
--host 0.0.0.0
--trust-remote-code
--gpu-memory-util=0.95
--no-enable-prefix-caching
--max-num-batched-tokens=4096
--disable-log-requests
--max-model-len=4096
--block-size 64
--quantization sym_int4
-tp=4
--allowed-local-media-path /llm/shawn/ \

Here if we change the paramer --quantization fp8 , the model can launch successfully, but we use --quantization sym_int4, it report error

Below is the error info.

(VllmWorker rank=3 pid=70547) INFO 08-29 00:36:37 [default_loader.py:262] Loading weights took 10.04 seconds
Loading safetensors checkpoint shards: 72% Completed | 13/18 [00:10<00:03, 1.26it/s]
(VllmWorker rank=0 pid=70544)
ERROR 08-29 00:36:39 [core.py:638] EngineCore failed to start.
ERROR 08-29 00:36:39 [core.py:638] Traceback (most recent call last):
ERROR 08-29 00:36:39 [core.py:638] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/engine/core.py", line 629, in run_engine_core
ERROR 08-29 00:36:39 [core.py:638] engine_core = EngineCoreProc(*args, **kwargs)
ERROR 08-29 00:36:39 [core.py:638] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 08-29 00:36:39 [core.py:638] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/engine/core.py", line 447, in init
ERROR 08-29 00:36:39 [core.py:638] super().init(vllm_config, executor_class, log_stats,
ERROR 08-29 00:36:39 [core.py:638] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/engine/core.py", line 77, in init
ERROR 08-29 00:36:39 [core.py:638] self.model_executor = executor_class(vllm_config)
ERROR 08-29 00:36:39 [core.py:638] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 08-29 00:36:39 [core.py:638] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/executor/executor_base.py", line 53, in init
ERROR 08-29 00:36:39 [core.py:638] self._init_executor()
ERROR 08-29 00:36:39 [core.py:638] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/executor/multiproc_executor.py", line 94, in _init_executor
ERROR 08-29 00:36:39 [core.py:638] self.workers = WorkerProc.wait_for_ready(unready_workers)
ERROR 08-29 00:36:39 [core.py:638] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 08-29 00:36:39 [core.py:638] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/executor/multiproc_executor.py", line 446, in wait_for_ready
ERROR 08-29 00:36:39 [core.py:638] raise e from None
ERROR 08-29 00:36:39 [core.py:638] Exception: WorkerProc initialization failed due to an exception in a background process. See stack trace for root cause.
Process EngineCore_0:
Traceback (most recent call last):
File "/usr/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib/python3.12/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/engine/core.py", line 642, in run_engine_core
raise e
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/engine/core.py", line 629, in run_engine_core
engine_core = EngineCoreProc(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/engine/core.py", line 447, in init
super().init(vllm_config, executor_class, log_stats,
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/engine/core.py", line 77, in init
self.model_executor = executor_class(vllm_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/executor/executor_base.py", line 53, in init
self._init_executor()
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/executor/multiproc_executor.py", line 94, in _init_executor
self.workers = WorkerProc.wait_for_ready(unready_workers)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/executor/multiproc_executor.py", line 446, in wait_for_ready
raise e from None
Exception: WorkerProc initialization failed due to an exception in a background process. See stack trace for root cause.
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/entrypoints/openai/api_server.py", line 1856, in
uvloop.run(run_server(args))
File "/usr/local/lib/python3.12/dist-packages/uvloop/init.py", line 109, in run
return __asyncio.run(
^^^^^^^^^^^^^^
File "/usr/lib/python3.12/asyncio/runners.py", line 194, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
File "/usr/local/lib/python3.12/dist-packages/uvloop/init.py", line 61, in wrapper
return await main
^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/entrypoints/openai/api_server.py", line 1791, in run_server
await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/entrypoints/openai/api_server.py", line 1811, in run_server_worker
async with build_async_engine_client(args, client_config) as engine_client:
File "/usr/lib/python3.12/contextlib.py", line 210, in aenter
return await anext(self.gen)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/entrypoints/openai/api_server.py", line 158, in build_async_engine_client
async with build_async_engine_client_from_engine_args(
File "/usr/lib/python3.12/contextlib.py", line 210, in aenter
return await anext(self.gen)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/entrypoints/openai/api_server.py", line 194, in build_async_engine_client_from_engine_args
async_llm = AsyncLLM.from_vllm_config(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/engine/async_llm.py", line 163, in from_vllm_config
return cls(
^^^^
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/engine/async_llm.py", line 117, in init
self.engine_core = EngineCoreClient.make_async_mp_client(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/engine/core_client.py", line 98, in make_async_mp_client
return AsyncMPClient(*client_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/engine/core_client.py", line 677, in init
super().init(
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/engine/core_client.py", line 408, in init
with launch_core_engines(vllm_config, executor_class,
File "/usr/lib/python3.12/contextlib.py", line 144, in exit
next(self.gen)
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/engine/utils.py", line 697, in launch_core_engines
wait_for_engine_startup(
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/engine/utils.py", line 750, in wait_for_engine_startup
raise RuntimeError("Engine core initialization failed. "
RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {'EngineCore_0': 1}
root@intelmc4b60-DS-V-P15SWN:/llm/shawn# /usr/lib/python3.12/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
/usr/lib/python3.12/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 2 leaked shared_memory objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '

[gc-fu]

Hi, this gonna be fixed by a new image. Will post here when the new image is available

[gc-fu]

Hi, you can pull the new image now:

[shawn9977]

It still failed with the new image, please take a closer look.Thank you!

shawn@intelmc4b60-DS-V-P15SWN:~$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
83eb1986c22a intel/llm-scaler-vllm:0.10.0-b1 "/bin/bash" 3 days ago Up 3 days lsv-container

===============================================================
#!/bin/bash

export VLLM_ALLOW_LONG_MAX_MODEL_LEN=1
export VLLM_WORKER_MULTIPROC_METHOD=spawn
export VLLM_OFFLOAD_WEIGHTS_BEFORE_QUANT=1

MODEL_NAME="Qwen2.5-VL-32B-Instruct"
MODEL_PATH="/llm/shawn/models/Qwen/${MODEL_NAME}"

python3 -m vllm.entrypoints.openai.api_server
--model "${MODEL_PATH}"
--served-model-name "${MODEL_NAME}"
--dtype=float16
--enforce-eager
--port 8000
--host 0.0.0.0
--trust-remote-code
--gpu-memory-util=0.95
--no-enable-prefix-caching
--max-num-batched-tokens=8192
--disable-log-requests
--max-model-len=8192
--block-size 64
--quantization sym_int4
-tp=2
--allowed-local-media-path /llm/shawn/ \

===============================================================
Error:

(VllmWorker rank=0 pid=392251)
(VllmWorker rank=0 pid=392251) INFO 09-01 01:04:41 [default_loader.py:262] Loading weights took 19.21 seconds
(VllmWorker rank=1 pid=392252) INFO 09-01 01:04:42 [default_loader.py:262] Loading weights took 19.64 seconds
(VllmWorker rank=0 pid=392251) INFO 09-01 01:05:39 [gpu_model_runner.py:1886] Model loading took 9.3493 GiB and 76.725226 seconds
(VllmWorker rank=1 pid=392252) INFO 09-01 01:05:39 [gpu_model_runner.py:1886] Model loading took 9.3493 GiB and 76.956265 seconds
(VllmWorker rank=1 pid=392252) INFO 09-01 01:05:39 [gpu_model_runner.py:2377] Encoder cache will be initialized with a budget of 16384 tokens, and profiled with 1 image items of the maximum feature size.
(VllmWorker rank=0 pid=392251) INFO 09-01 01:05:39 [gpu_model_runner.py:2377] Encoder cache will be initialized with a budget of 16384 tokens, and profiled with 1 image items of the maximum feature size.
(VllmWorker rank=1 pid=392252) Using a slow image processor as use_fast is unset and a slow processor was saved with this model. use_fast=True will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with use_fast=False.
(VllmWorker rank=0 pid=392251) Using a slow image processor as use_fast is unset and a slow processor was saved with this model. use_fast=True will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with use_fast=False.
(VllmWorker rank=1 pid=392252) You have video processor config saved in preprocessor.json file which is deprecated. Video processor configs should be saved in their own video_preprocessor.json file. You can rename the file or load and save the processor back which renames it automatically. Loading from preprocessor.json will be removed in v5.0.
(VllmWorker rank=0 pid=392251) You have video processor config saved in preprocessor.json file which is deprecated. Video processor configs should be saved in their own video_preprocessor.json file. You can rename the file or load and save the processor back which renames it automatically. Loading from preprocessor.json will be removed in v5.0.
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] WorkerProc hit an exception.
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] Traceback (most recent call last):
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/executor/multiproc_executor.py", line 541, in worker_busy_loop
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] output = func(*args, **kwargs)
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/utils/_contextlib.py", line 120, in decorate_context
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] return func(*args, **kwargs)
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/worker/xpu_worker.py", line 90, in determine_available_memory
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] self.model_runner.profile_run()
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/worker/gpu_model_runner.py", line 2400, in profile_run
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] dummy_encoder_outputs = self.model.get_multimodal_embeddings(
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/model_executor/models/qwen2_5_vl.py", line 1110, in get_multimodal_embeddings
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] vision_embeddings = self._process_image_input(multimodal_input)
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/model_executor/models/qwen2_5_vl.py", line 1045, in _process_image_input
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] image_embeds = self.visual(pixel_values, grid_thw=grid_thw_list)
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1773, in _wrapped_call_impl
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] return self._call_impl(*args, **kwargs)
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1784, in _call_impl
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] return forward_call(*args, **kwargs)
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/model_executor/models/qwen2_5_vl.py", line 794, in forward
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] hidden_states = blk(
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1773, in _wrapped_call_impl
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] return self._call_impl(*args, **kwargs)
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1784, in _call_impl
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] return forward_call(*args, **kwargs)
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/model_executor/models/qwen2_5_vl.py", line 459, in forward
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] x = x + self.attn(self.norm1(x),
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1773, in _wrapped_call_impl
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] return self._call_impl(*args, **kwargs)
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1784, in _call_impl
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] return forward_call(*args, **kwargs)
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/model_executor/models/qwen2_5_vl.py", line 293, in forward
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] x, _ = self.qkv(x)
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1773, in _wrapped_call_impl
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] return self._call_impl(*args, **kwargs)
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1784, in call_impl
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] return forward_call(*args, **kwargs)
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/model_executor/layers/linear.py", line 588, in forward
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] output_parallel = self.quant_method.apply(self, input, bias)
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/model_executor/layers/quantization/sym_int4.py", line 158, in apply
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] out = layer.ipex_qlinear(reshaped_x)
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1962, in getattr
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] raise AttributeError(
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] AttributeError: 'QKVParallelLinear' object has no attribute 'ipex_qlinear'
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] Traceback (most recent call last):
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/executor/multiproc_executor.py", line 541, in worker_busy_loop
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] output = func(*args, **kwargs)
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/utils/_contextlib.py", line 120, in decorate_context
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] return func(*args, **kwargs)
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/worker/xpu_worker.py", line 90, in determine_available_memory
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] self.model_runner.profile_run()
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/worker/gpu_model_runner.py", line 2400, in profile_run
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] dummy_encoder_outputs = self.model.get_multimodal_embeddings(
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/model_executor/models/qwen2_5_vl.py", line 1110, in get_multimodal_embeddings
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] vision_embeddings = self._process_image_input(multimodal_input)
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/model_executor/models/qwen2_5_vl.py", line 1045, in _process_image_input
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] image_embeds = self.visual(pixel_values, grid_thw=grid_thw_list)
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1773, in _wrapped_call_impl
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] return self._call_impl(*args, **kwargs)
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1784, in _call_impl
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] return forward_call(*args, **kwargs)
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/model_executor/models/qwen2_5_vl.py", line 794, in forward
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] hidden_states = blk(
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1773, in _wrapped_call_impl
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] return self._call_impl(*args, **kwargs)
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1784, in _call_impl
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] return forward_call(*args, **kwargs)
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/model_executor/models/qwen2_5_vl.py", line 459, in forward
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] x = x + self.attn(self.norm1(x),
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1773, in _wrapped_call_impl
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] return self._call_impl(*args, **kwargs)
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1784, in _call_impl
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] return forward_call(*args, **kwargs)
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/model_executor/models/qwen2_5_vl.py", line 293, in forward
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] x, _ = self.qkv(x)
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1773, in _wrapped_call_impl
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] return self._call_impl(*args, **kwargs)
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1784, in call_impl
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] return forward_call(*args, **kwargs)
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/model_executor/layers/linear.py", line 588, in forward
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] output_parallel = self.quant_method.apply(self, input, bias)
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/model_executor/layers/quantization/sym_int4.py", line 158, in apply
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] out = layer.ipex_qlinear(reshaped_x)
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1962, in getattr
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] raise AttributeError(
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546] AttributeError: 'QKVParallelLinear' object has no attribute 'ipex_qlinear'
(VllmWorker rank=1 pid=392252) ERROR 09-01 01:05:41 [multiproc_executor.py:546]
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] WorkerProc hit an exception.
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] Traceback (most recent call last):
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/executor/multiproc_executor.py", line 541, in worker_busy_loop
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] output = func(*args, **kwargs)
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/utils/_contextlib.py", line 120, in decorate_context
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] return func(*args, **kwargs)
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/worker/xpu_worker.py", line 90, in determine_available_memory
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] self.model_runner.profile_run()
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/worker/gpu_model_runner.py", line 2400, in profile_run
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] dummy_encoder_outputs = self.model.get_multimodal_embeddings(
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/model_executor/models/qwen2_5_vl.py", line 1110, in get_multimodal_embeddings
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] vision_embeddings = self._process_image_input(multimodal_input)
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/model_executor/models/qwen2_5_vl.py", line 1045, in _process_image_input
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] image_embeds = self.visual(pixel_values, grid_thw=grid_thw_list)
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1773, in _wrapped_call_impl
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] return self._call_impl(*args, **kwargs)
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1784, in _call_impl
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] return forward_call(*args, **kwargs)
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/model_executor/models/qwen2_5_vl.py", line 794, in forward
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] hidden_states = blk(
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1773, in _wrapped_call_impl
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] return self._call_impl(*args, **kwargs)
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1784, in _call_impl
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] return forward_call(*args, **kwargs)
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/model_executor/models/qwen2_5_vl.py", line 459, in forward
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] x = x + self.attn(self.norm1(x),
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1773, in _wrapped_call_impl
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] return self._call_impl(*args, **kwargs)
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1784, in _call_impl
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] return forward_call(*args, **kwargs)
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/model_executor/models/qwen2_5_vl.py", line 293, in forward
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] x, _ = self.qkv(x)
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1773, in _wrapped_call_impl
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] return self._call_impl(*args, **kwargs)
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1784, in call_impl
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] return forward_call(*args, **kwargs)
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/model_executor/layers/linear.py", line 588, in forward
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] output_parallel = self.quant_method.apply(self, input, bias)
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/model_executor/layers/quantization/sym_int4.py", line 158, in apply
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] out = layer.ipex_qlinear(reshaped_x)
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1962, in getattr
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] raise AttributeError(
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] AttributeError: 'QKVParallelLinear' object has no attribute 'ipex_qlinear'
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] Traceback (most recent call last):
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/executor/multiproc_executor.py", line 541, in worker_busy_loop
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] output = func(*args, **kwargs)
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/utils/_contextlib.py", line 120, in decorate_context
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] return func(*args, **kwargs)
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/worker/xpu_worker.py", line 90, in determine_available_memory
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] self.model_runner.profile_run()
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/worker/gpu_model_runner.py", line 2400, in profile_run
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] dummy_encoder_outputs = self.model.get_multimodal_embeddings(
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/model_executor/models/qwen2_5_vl.py", line 1110, in get_multimodal_embeddings
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] vision_embeddings = self._process_image_input(multimodal_input)
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/model_executor/models/qwen2_5_vl.py", line 1045, in _process_image_input
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] image_embeds = self.visual(pixel_values, grid_thw=grid_thw_list)
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1773, in _wrapped_call_impl
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] return self._call_impl(*args, **kwargs)
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1784, in _call_impl
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] return forward_call(*args, **kwargs)
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/model_executor/models/qwen2_5_vl.py", line 794, in forward
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] hidden_states = blk(
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1773, in _wrapped_call_impl
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] return self._call_impl(*args, **kwargs)
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1784, in _call_impl
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] return forward_call(*args, **kwargs)
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/model_executor/models/qwen2_5_vl.py", line 459, in forward
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] x = x + self.attn(self.norm1(x),
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1773, in _wrapped_call_impl
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] return self._call_impl(*args, **kwargs)
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1784, in _call_impl
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] return forward_call(*args, **kwargs)
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/model_executor/models/qwen2_5_vl.py", line 293, in forward
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] x, _ = self.qkv(x)
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1773, in _wrapped_call_impl
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] return self._call_impl(*args, **kwargs)
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1784, in call_impl
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] return forward_call(*args, **kwargs)
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/model_executor/layers/linear.py", line 588, in forward
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] output_parallel = self.quant_method.apply(self, input, bias)
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/model_executor/layers/quantization/sym_int4.py", line 158, in apply
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] out = layer.ipex_qlinear(reshaped_x)
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] ^^^^^^^^^^^^^^^^^^
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1962, in getattr
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] raise AttributeError(
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546] AttributeError: 'QKVParallelLinear' object has no attribute 'ipex_qlinear'
(VllmWorker rank=0 pid=392251) ERROR 09-01 01:05:42 [multiproc_executor.py:546]
ERROR 09-01 01:05:42 [core.py:638] EngineCore failed to start.
ERROR 09-01 01:05:42 [core.py:638] Traceback (most recent call last):
ERROR 09-01 01:05:42 [core.py:638] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/engine/core.py", line 629, in run_engine_core
ERROR 09-01 01:05:42 [core.py:638] engine_core = EngineCoreProc(*args, **kwargs)
ERROR 09-01 01:05:42 [core.py:638] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 09-01 01:05:42 [core.py:638] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/engine/core.py", line 447, in init
ERROR 09-01 01:05:42 [core.py:638] super().init(vllm_config, executor_class, log_stats,
ERROR 09-01 01:05:42 [core.py:638] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/engine/core.py", line 86, in init
ERROR 09-01 01:05:42 [core.py:638] self._initialize_kv_caches(vllm_config)
ERROR 09-01 01:05:42 [core.py:638] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/engine/core.py", line 164, in _initialize_kv_caches
ERROR 09-01 01:05:42 [core.py:638] self.model_executor.determine_available_memory())
ERROR 09-01 01:05:42 [core.py:638] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 09-01 01:05:42 [core.py:638] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/executor/abstract.py", line 77, in determine_available_memory
ERROR 09-01 01:05:42 [core.py:638] output = self.collective_rpc("determine_available_memory")
ERROR 09-01 01:05:42 [core.py:638] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 09-01 01:05:42 [core.py:638] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/executor/multiproc_executor.py", line 237, in collective_rpc
ERROR 09-01 01:05:42 [core.py:638] result = get_response(w, dequeue_timeout)
ERROR 09-01 01:05:42 [core.py:638] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 09-01 01:05:42 [core.py:638] File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/executor/multiproc_executor.py", line 224, in get_response
ERROR 09-01 01:05:42 [core.py:638] raise RuntimeError(
ERROR 09-01 01:05:42 [core.py:638] RuntimeError: Worker failed with error ''QKVParallelLinear' object has no attribute 'ipex_qlinear'', please check the stack trace above for the root cause
ERROR 09-01 01:05:43 [multiproc_executor.py:140] Worker proc VllmWorker-1 died unexpectedly, shutting down executor.
Process EngineCore_0:
Traceback (most recent call last):
File "/usr/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib/python3.12/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/engine/core.py", line 642, in run_engine_core
raise e
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/engine/core.py", line 629, in run_engine_core
engine_core = EngineCoreProc(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/engine/core.py", line 447, in init
super().init(vllm_config, executor_class, log_stats,
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/engine/core.py", line 86, in init
self._initialize_kv_caches(vllm_config)
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/engine/core.py", line 164, in _initialize_kv_caches
self.model_executor.determine_available_memory())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/executor/abstract.py", line 77, in determine_available_memory
output = self.collective_rpc("determine_available_memory")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/executor/multiproc_executor.py", line 237, in collective_rpc
result = get_response(w, dequeue_timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/executor/multiproc_executor.py", line 224, in get_response
raise RuntimeError(
RuntimeError: Worker failed with error ''QKVParallelLinear' object has no attribute 'ipex_qlinear'', please check the stack trace above for the root cause
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/entrypoints/openai/api_server.py", line 1856, in
uvloop.run(run_server(args))
File "/usr/local/lib/python3.12/dist-packages/uvloop/init.py", line 109, in run
return __asyncio.run(
^^^^^^^^^^^^^^
File "/usr/lib/python3.12/asyncio/runners.py", line 194, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
File "/usr/local/lib/python3.12/dist-packages/uvloop/init.py", line 61, in wrapper
return await main
^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/entrypoints/openai/api_server.py", line 1791, in run_server
await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/entrypoints/openai/api_server.py", line 1811, in run_server_worker
async with build_async_engine_client(args, client_config) as engine_client:
File "/usr/lib/python3.12/contextlib.py", line 210, in aenter
return await anext(self.gen)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/entrypoints/openai/api_server.py", line 158, in build_async_engine_client
async with build_async_engine_client_from_engine_args(
File "/usr/lib/python3.12/contextlib.py", line 210, in aenter
return await anext(self.gen)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/entrypoints/openai/api_server.py", line 194, in build_async_engine_client_from_engine_args
async_llm = AsyncLLM.from_vllm_config(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/engine/async_llm.py", line 163, in from_vllm_config
return cls(
^^^^
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/engine/async_llm.py", line 117, in init
self.engine_core = EngineCoreClient.make_async_mp_client(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/engine/core_client.py", line 98, in make_async_mp_client
return AsyncMPClient(*client_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/engine/core_client.py", line 677, in init
super().init(
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/engine/core_client.py", line 408, in init
with launch_core_engines(vllm_config, executor_class,
File "/usr/lib/python3.12/contextlib.py", line 144, in exit
next(self.gen)
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/engine/utils.py", line 697, in launch_core_engines
wait_for_engine_startup(
File "/usr/local/lib/python3.12/dist-packages/vllm-0.10.1.dev0+g6d8d0a24c.d20250827.xpu-py3.12-linux-x86_64.egg/vllm/v1/engine/utils.py", line 750, in wait_for_engine_startup
raise RuntimeError("Engine core initialization failed. "
RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {'EngineCore_0': 1}
root@intelmc4b60-DS-V-P15SWN:/llm/shawn#

[gc-fu]

Hi, I am investigating this problem. I have verified that multi-modal models currently experience failures.

[gc-fu]

Hi, now you can try to pull the latest image intel/llm-scaler-vllm:latest.

The problem should have been solved