vLLM users face errors with ROCm backend and model configurations

“Your current environment <details <summary The output of <code python collect env.py</code </summary </details 🐛 Describe the bug Crashes on MI300 with Sep 30 nightly: vllm/vllm openai rocm:nightly 29468dde8b515031dc6d4d9d06bf0a2fa0442098 . Reproducer Send some requests Before submitting a new issue... [x] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.”

“Your current environment <details <summary The output of <code python collect env.py</code </summary </details 🐛 Describe the bug Issue: gibberish at low concurrency on latest nightly vllm/vllm openai rocm:nightly ac68c3087215e0a4f3cdfa218508c6aada57235d . Tested on MI350 Not an issue on gfx942. Reproducer 8xMI350X. Observed both with and without breakable cudagraphs on. 1. GSM8k conc 1 |Tasks|Version| Filter |n shot| Metric | |Value | |Stderr| | | :| | :| | | :| | :| |gsm8k| 3|flexible extract”

“Your current environment <details <summary The output of <code python collect env.py</code </summary </details Additional context on the environment: vLLM is tag v0.26.0 ( 568afb3a1 ) built from source with VLLM TARGET DEVICE=rocm PYTORCH ROCM ARCH=gfx1151 , against the TheRock per arch wheels from https://rocm.nightlies.amd.com/v2/gfx1151/ (torch / triton / rocm sdk 7.13.0a20260424, all inside the venv; no system ROCm involved). One local source change was needed to make the build link: a ROCm”

“Your current environment This issue has been encountered a long time ago, and it still exists in the latest version. So there's no need to discuss version related issues. When we use the openai library, http, httpx, and other methods to make requests, this phenomenon is triggered randomly and with a very low probability (possibly once every 3 4 days). As shown in the figure below, we made a request to a locally deployed GLM model using streaming. Our logs indicate that the request was sent succe”

“Your current environment <details <summary The output of <code python collect env.py</code </summary </details 🐛 Describe the bug When i request to the model to write some code more complex everything looks good at first, but during the code generation an exception occurs "400 Unterminated string starting at: line 1 column 14 (char 13)" At first i was thinking it was related to this issue [39055] (https://github.com/vllm project/vllm/pull/39055) , but after updating to the last version i am sti”

“Your current environment <details <summary The output of <code python collect env.py</code </summary </details 🐛 Describe the bug When serving Qwen3.5 122B A10B, whenever I add VLLM PP LAYER PARTITION=7,7,7,3,3,3,3,3,3,3,3,3 or values alike, the following exception would raise: I have searched for related issues and most seems to be fixed already, but in my case using an uneven pipeline parallel config will still reproduce it. I had no problem serving Qwen3 MoE so it might be specifically with”

“Your current environment <details <summary The output of <code python collect env.py</code </summary </details 🐛 Describe the bug There is a substantial variance in DeepSeek R1's performance on the MMLU and HumanEval benchmarks when tested with different versions of vLLM. I evaluated the accuracy of DeepSeek R1 on the official MMLU and HumanEval datasets using the EvalScope framework with vLLM versions 0.10.2 and 0.20.2. The results are as follows: There is a noticeable performance regression f”

“Your current environment The vllm version is 0.19.1, which I installed using pip. All packages are those specified during the vllm installation. 🐛 Describe the bug <img width="1925" height="65" alt="Image" src="https://github.com/user attachments/assets/e7a4e3ed 20e1 44e4 a94f 6d59bde3eded" / <img width="1678" height="378" alt="Image" src="https://github.com/user attachments/assets/3890cdbe 79f0 403d bc63 80d5dfe754f9" / Before submitting a new issue... [x] Make sure you already searched for re”

“Your current environment <details <summary The output of <code python collect env.py</code </summary Collecting environment information... ============================== System Info ============================== OS : Ubuntu 22.04.5 LTS (x86 64) GCC version : (Ubuntu 11.4.0 1ubuntu1 22.04.3) 11.4.0 Clang version : Could not collect CMake version : Could not collect Libc version : glibc 2.35 ============================== PyTorch Info ============================== PyTorch version : 2.10.0+cu129”

“Your current environment 🐛 Bug Report Summary Prefill performance regressed 5.5x between vLLM v0.18.1 and v0.18.2 on ROCm . The root cause is a backend selection logic change in rocm.py that gates ROCM ATTN (flash attention) behind a new config flag use prefill decode attention which defaults to False . Even with flash attention installed, v0.18.2 silently falls back to TRITON ATTN . Environment | Component | v0.18.1 (Fast) | v0.18.2 (Slow) | | | | | | vLLM | 0.18.1rc1.dev231 | 0.18.2.dev0 | |”

“Your current environment NA 🐛 Describe the bug 🐛 Describe the bug Recently I submitted a PR that assumed vLLM would always only need 32‑bit indexing. This broke one of our internal models and raised the question of whether that assumption is actually valid in all cases. I’ve since made the assumption configurable to unblock current issues. We discussed several alternatives, and here’s the plan to make this more robust: Turn off assume 32 bit indexing for backed dynamic shapes. For unbacked mod”