VLLM Framework Crashes with Attribute Errors
18 posts since Aug 2025, all from GitHub.
“Your current environment vllm 0.25.1 on OpenShift 🐛 Describe the bug With 0.25.1 and kv cache dtype fp8, the ministral modele is broken : https://huggingface.co/mistralai/Ministral 3 3B Instruct 2512 It runs but it produces glitched token like " " No problem if i use 0.24.0…”
“Prevent timeout based connection close of running vLLM response behind (reverse) proxies (tunnels) with long thinking LLMs Problem When vLLM streams responses via SSE ( text/event stream ), there are periods where no data is sent to the client: 1. Queue wait : Request sits in…”
“Your current environment vllm nightly image 🐛 Describe the bug Serve Qwen3.5 27B FP8 as a openai compatible seveing. Use openai python sdk response format function, found model response doesn't follow the structure required. Before submitting a new issue... [x] Make sure you…”
Users are experiencing crashes and errors within the VLLM framework, specifically related to attribute references and model loading. These issues appear to be tied to recent updates and versions of the software.
SOURCES (18)
“Your current environment <html <body <h2 data pm slice="1 3 []" <span Summary</span </h2 <p <span We observed a severe text generation correctness regression when upgrading from vLLM 0.22.1 to 0.25.1 using the same </span <code dir="ltr" <span RedHatAI/Apertus 70B Instruct 2509 FP8 dynamic</span </code…”
“Your current environment vllm version: 0.31.0(db9527a46873454610df6dbedf79a36d6bf1a7f6) Model: Qwen/Qwen3 ASR 1.7B Python: 3.12.14 PyTorch: 2.13.0 GPU: NVIDIA H100 80GB 🐛 Describe the bug When serving Qwen3 ASR ( Qwen/Qwen3 ASR 1.7B , qwen3 asr.py ) with the Rust frontend , any scheduler step that batches two or more requests carrying audio of different durations kills the EngineCore process: The server then shuts down and all in flight requests fail with HTTP 500 . This is fatal for production”
“Your current environment <details <summary The output of <code python collect env.py</code </summary </details 🐛 Describe the bug Versions tested: vLLM main c5c1161382 (2026 10 08, newer than v0.31.0) , built from source with TORCH CUDA ARCH LIST=10.0 (the collect env output above is from this build). Its version string 0.31.1.dev0+pr59717 is the one we set for the build, and its CUDA Archs line reflects the container's TORCH CUDA ARCH LIST environment variable, not what this build was compiled”
“Your current environment <details <summary The output of <code python collect env.py</code </summary </details 🐛 Describe the bug Summary With vLLM 0.31.0 and Mooncake KV transfer, the Decode EngineCore can crash after a P/D request is cancelled or times out while Decode is unavailable. Environment vLLM: 0.31.0 Mooncake: mooncake transfer engine cuda13==0.3.13.post1 CUDA: 13 KV connector: MooncakeConnector KV protocol: tcp KV role: kv consumer Model: Qwen2.5 0.5B Instruct V1 engine Reproduction”
“Your current environment <details <summary The output of <code python collect env.py</code </summary </details vLLM v0.31.0 (official image, banner version 0.31.0 ) 1x NVIDIA B300 SXM6 AC (SM103, 148 SMs), TP=1 GLM 5.3 Flash, NVFP4 checkpoint ( nvidia/GLM 5.3 Flash NVFP4 ), hidden size=4096 , hc mult=4 🐛 Describe the bug On GLM 5.3 Flash, every prefill step with 129 4736 tokens (148 SMs) JIT compiles a new DeepGEMM sm100 tf32 hc prenorm gemm kernel the first time its split count is seen. Startu”
“Your current environment docker 🐛 Describe the bug vllm tb.txt SimpleCPUOffloadConnector on Qwen3.8 Flash Next while an async CPU→GPU prefix load is in flight Environment Image vllm/vllm openai:nightly e9757321527ca1ecd514c07c1418dd2c53da3d19 ( 0.29.1rc1.dev573+ge97573215 , includes 56810), Model Runner V2 1× RTX PRO 6000 Blackwell (SM120, 96 GB), PCIe Gen3, driver 615.71.09 Model: Inferact/Qwen3.8 Flash Next NVFP4 , TP1 Config Workload: agentic, 4–8 concurrent long context requests (80k–185k t”
“Your current environment <details <summary The output of <code python collect env.py</code </summary </details 🐛 Describe the bug Describe the bug InternVL2 and Qwen2 VL fail during Punica initialization. To Reproduce Run this standalone Python program on the target machine after ensuring the selected model path exists and the GPU has sufficient free memory: python import subprocess VLLM = "/root/anaconda3/envs/vllm/bin/vllm" MODEL = "/root/.cache/huggingface/hub/models OpenGVLab InternVL2 8B/s”
“Your current environment <details <summary The output of <code python collect env.py</code </summary </details Notes on the environment: Image vllm/vllm openai:qwen38 flash next ( sha256:fc120ece0a388cc0aa1caad4a9f1cd92113484ab7ec2fd0efadd62585be05bf8 , built 2026 08 26), the dedicated image the Flash Next recipe requires. Host: Ubuntu 26.04.1, 64 GB RAM. 1× NVIDIA RTX PRO 6000 Blackwell Workstation (96 GB, SM120), driver 595.84. Checkpoint primitive ai/Qwen3.8 Flash Next mixed NVFP4 FP8 (rev ad”
“Your current environment vllm 0.25.1 on OpenShift 🐛 Describe the bug With 0.25.1 and kv cache dtype fp8, the ministral modele is broken : https://huggingface.co/mistralai/Ministral 3 3B Instruct 2512 It runs but it produces glitched token like " " No problem if i use 0.24.0 Before submitting a new issue... [x] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.”
“Prevent timeout based connection close of running vLLM response behind (reverse) proxies (tunnels) with long thinking LLMs Problem When vLLM streams responses via SSE ( text/event stream ), there are periods where no data is sent to the client: 1. Queue wait : Request sits in scheduler queue waiting for GPU resources 2. Prefill : Model processes the input prompt (especially with long contexts) 3. Generation pause : Rare cases where the scheduler preempts a request During these periods, the HTTP”
“Current environment <details <summary The output of <code python collect env.py</code </summary </details Bug description When using vllm 0.23.1rc1.dev207 , tool call requests to the /v1/responses endpoint fail when using Qwen/Qwen3.6 27B with MTP enabled via speculative config '{"method":"mtp", "num speculative tokens":2}' . The same requests complete successfully when speculative config '{"method":"mtp", "num speculative tokens":2}' is removed from the start command. The issue appears to have”
“Your current environment <details </details 🐛 Describe the bug Summary Serving a Qwen3.6 27b model with vLLM + LMCacheMPConnector works initially, but after some successful requests the engine crashes with: AssertionError: External KV connector is not verified yet The crash happens inside vLLM's Mamba scheduler path, not during LMCache startup. Environment vLLM: 0.22.1 LMCache: v0.4.7.dev103 (g7bf82915) Model: Qwen3.6 27B AWQ BF16 INT4 Tensor parallel size: 2 kv connector: LMCacheMPConnector kv”
“Your current environment <details <summary The output of <code python collect env.py</code </summary </details 🐛 Describe the bug When running vllm serve with data parallel size 1 and enable expert parallel on a multi node Ray cluster, the server hangs permanently and never becomes ready. No inference requests are served. Environment: 4 physical nodes, each with 4× NVIDIA RTX A6000 GPUs (PCIe), InfiniBand interconnect Ray cluster spanning all 4 nodes (head node + 3 worker nodes) Each node contr”
“Your current environment <details <summary The output of <code python collect env.py</code </summary </details 🐛 Describe the bug Description Qwen/Qwen3 1.7B on vLLM 0.21.0 gives a wrong answer to the simple arithmetic prompt 2 10 when using tensor parallelism. The model outputs 512 or 81 instead of 1024 . When TP=1, the result is correct. This is a regression from vLLM 0.19.1, which produces correct results on all configs. Reproduction Expected output (all 3 seeds) Actual output (vLLM 0.21.0)”
“Your current environment <details <summary The output of <code python collect env.py</code </summary </details 🐛 Describe the bug the output tool call content: the json string is surrounded with specific chars <|" : the second format error case: start command: Before submitting a new issue... [x] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.”
“Your current environment <details <summary The output of <code python collect env.py</code </summary </details 🐛 Describe the bug I use code below to start the server. The yaml file is and i use lmcache version is 0.4.1 After serveral requests ( about 500 requests each request about 4000 with input len avg in 16 batch size), i encounter an error The important part is old mem objs.append(self.hot cache[key]) (EngineCore DP0 pid=17830) ERROR 03 17 14:38:51 [core.py:938] ^^^^^ (EngineCore DP0 pid=”
“Your current environment vllm nightly image 🐛 Describe the bug Serve Qwen3.5 27B FP8 as a openai compatible seveing. Use openai python sdk response format function, found model response doesn't follow the structure required. Before submitting a new issue... [x] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.”
“Your current environment <details <summary The output of <code python collect env.py</code </summary </details 🐛 Describe the bug As the title described, I tried to deploy the qwen3 30b mo3 model using vllm=v0.10.0 but failed. The error is AttributeError: ' OpNamespace' ' moe C' object has no attribute 'topk softmax' The command is vllm serve /usr/local/storage/models/Qwen3 30B A3B Instruct 2507 FP8 port 9850 served model name "qwen3" max model len 32768 gpu memory utilization 0.85 enable auto”