VLLM users see crashes with HiSparse and moe_backend
11 posts since May 2026, all from GitHub.
“Your current environment 🐛 Describe the bug Description With quantization="mxfp8" and moe backend="humming" , the engine dies after the weights have been loaded , while converting them to the kernel format, raising a bare KeyError with no context at all: Three problems: 1. A…”
“Your current environment B200, upstream main @ 61d9c777bf, Model Runner V2 ( VLLM USE V2 MODEL RUNNER=1 ). 🐛 Describe the bug Found while reviewing 60424. Tracking: 60072. HiSparse needs Model Runner V2, and DBO on V2 needs VLLM USE V2 MODEL RUNNER=1 , DP 1 with EP, FULL CUDA…”
“Your current environment 🐛 Describe the bug Description The mutually inconsistent combination convert="classify" (requires a pooling path) with runner="generate" is not rejected at config validation time . The engine crashes later during model initialization with a bare…”
VLLM users are experiencing crashes when using HiSparse with a specific `moe_backend` configuration. The errors involve assertion failures and type mismatches, indicating incompatibility between the selected configurations and the underlying model architecture.
SOURCES (11)
“Your current environment 🐛 Describe the bug Description With quantization="mxfp8" and moe backend="humming" , the engine dies after the weights have been loaded , while converting them to the kernel format, raising a bare KeyError with no context at all: Three problems: 1. A bare…”
“Your current environment 🐛 Describe the bug Description With online FP8 ( quantization="fp8" or "fp8 per tensor" ) plus an explicit moe backend="cutlass" , model loading dies inside select fp8 moe backend() with: Three problems: 1. The message carries no reason. "disabled for this configuration" does not say whether the compute capability is insufficient, whether the CUTLASS kernels were not compiled into this build, or whether the weight/activation quantization keys do not match. And 'cutlass'”
“Your current environment 🐛 Describe the bug Description With enable lora=True , setting dtype="float32" (a documented, user settable value) makes the engine die during the startup dummy forward, inside the LoRA expand Triton op: Nothing validates the pair ( dtype=float32 , enable lora=True ) at config time; the incompatible combination is only discovered as a bare assertion inside a Triton kernel wrapper during profile run , surfaced to the user as the opaque RuntimeError: Engine core initializ”
“Your current environment 🐛 Describe the bug Description With dtype="float32" on an SM89 GPU, unquantized MoE models pass EngineArgs.create engine config() and then die during the startup memory profiling forward, because the Triton fused MoE kernel selected for the float32 activation dtype requires more shared memory than the device allows: float32 is a documented, user settable dtype . Nothing in config validation or backend selection rejects it, and the failure surfaces only deep inside the p”
“Your current environment B200, upstream main @ 61d9c777bf, Model Runner V2 ( VLLM USE V2 MODEL RUNNER=1 ). 🐛 Describe the bug Found while reviewing 60424. Tracking: 60072. HiSparse needs Model Runner V2, and DBO on V2 needs VLLM USE V2 MODEL RUNNER=1 , DP 1 with EP, FULL CUDA graphs (the default FULL AND PIECEWISE works; HiSparse rejects plain FULL and DBO rejects PIECEWISE ), and DeepEP low latency. With HiSparse on, the first microbatched step fails: Repro Then send enough concurrent requests”
“Your current environment <details <summary The output of <code python collect env.py</code </summary </details 🐛 Describe the bug vllm serve zai org/GLM 5.3 Flash \ tensor parallel size 8 \ enable expert parallel \ enable eplb \ eplb config.num redundant experts 8 \ eplb config.log balancedness true \ eplb config.log balancedness interval 100 \ gpu memory utilization 0.95 \ max model len 1 \ max num seqs 128 \ speculative config.method mtp \ speculative config.num speculative tokens 5 \ kv tran”
“Your current environment 🐛 Describe the bug Description The mutually inconsistent combination convert="classify" (requires a pooling path) with runner="generate" is not rejected at config validation time . The engine crashes later during model initialization with a bare assertion: This is a missing cross field validation: convert=classify implies a pooler capable model configuration, while runner=generate means no pooler config is ever constructed. The invalid combination survives EngineArgs.cr”
“Your current environment 🐛 Describe the bug Description Selecting moe backend="batched triton" for an unquantized MoE model is accepted at config validation time, but the engine crashes during model load with: The root cause is visible in make unquantized moe kernel ( vllm/model executor/layers/fused moe/oracle/unquantized.py ): when prepare finalize.activation format != FusedMoEActivationFormat.BatchedExperts , the else branch constructs experts cls(moe config=..., quant config=...) without ma”
“Your current environment vLLM main @ ad0f67a7ec, 1×B200, FlashInfer sparse MLA, fp8 KV, Model Runner V2. 🐛 Describe the bug With HiSparse, a prompt that the server accepted can crash EngineCore with CUDA OOM, and running requests die with it. Two allocations in the HiSparse sparse MLA prefill path aren't reserved by startup memory profiling. The profiling dummy run is one chunk of short pure prefills. 1. Full history copy. stage prefill rows → HiSparseRuntime.gather prefill cache copies the pre”
“Your current environment vLLM Version : 0.22.0 PyTorch version : 2.11.0+cu130 CUDA used to build PyTorch : 13.0 GPU : NVIDIA RTX PRO 6000 Blackwell Server Edition (compute capability 12.0) Nvidia driver version : 590.44.01 triton : 3.6.0 flashinfer python : 0.6.11.post2 transformers : 5.9.0 Python version : 3.12.3 OS : Linux x86 64 Model : Qwen3.6 35B A3B (qwen3 5 moe, hybrid linear attention MoE) 🐛 Describe the bug On RTX PRO 6000 Blackwell (SM 12.0), serving a Qwen3.6 35B A3B family MoE (qwen”
“Your current environment <details <summary The output of <code python collect env.py</code </summary </details 🐛 Describe the bug Description When sending a /generative scoring request to a vLLM server running DeepSeek V4 Flash with speculative decoding (MTP) enabled, the server crashes with a shape mismatch error inside the rejection sampler. The service becomes completely unresponsive and requires a restart. The error occurs while processing a batch that mixes generative scoring requests and”