Ollama users face issues with structured output and models
12 posts since May 2025, all from GitHub.
“Before Submitting [x] I searched open and closed issues and discussions for an existing report. [x] I reproduced this bug on the latest release AND on the current dev branch, right before submitting this report. I did not just check an old version or rely on a check from days…”
“What is the issue? When OLLAMA CONTEXT LENGTH is less than the model's native context window size, the effective available context length is OLLAMA CONTEXT LENGTH not the model's native. When querying the models list via the /api/show or /api/tags each model lists its native…”
“What is the issue? Observed behavior The installed stock qwen3.5:4b , with request local options.num ctx:65536 , returns HTTP 200 and completes with done:true , done reason:"stop" , and positive eval count , but only message.thinking : no content or tool calls. The attached…”
Users are encountering problems using structured outputs with certain Ollama models, including incompatibility with reasoning modes and difficulties parsing outputs from the OpenAI SDK. These issues hinder the effective use of advanced model features.
SOURCES (12)
“Before Submitting [x] I searched open and closed issues and discussions for an existing report. [x] I reproduced this bug on the latest release AND on the current dev branch, right before submitting this report. I did not just check an old version or rely…”
“What is the issue? When OLLAMA CONTEXT LENGTH is less than the model's native context window size, the effective available context length is OLLAMA CONTEXT LENGTH not the model's native. When querying the models list via the /api/show or /api/tags each model lists its native context window size, instead of ollama's OLLAMA CONTEXT LENGTH cap. This can result in clients assuming a larger context than actually available and running into truncated messages errors. The OLLAMA CONTEXT LENGTH cap is on”
“What is the issue? Observed behavior The installed stock qwen3.5:4b , with request local options.num ctx:65536 , returns HTTP 200 and completes with done:true , done reason:"stop" , and positive eval count , but only message.thinking : no content or tool calls. The attached anonymized request reproduced this 3/3 sequential runs directly against Ollama , without VS Code, Copilot, or the extension provider. Expected: a visible answer to the completed task, or a tool call if more work is needed. Th”
“What is happening? On Windows with Ollama 0.40.0, qwen3.8:27b reproducibly returns HTTP 500 from /api/chat for a large tool aware request: text {"error":"error unmarshalling llama server response: unexpected end of JSON input"} The important observation is that the underlying llama server appears to complete inference normally before Ollama returns the 500. In the reproduced failure: llama server receives approximately 14.8k prompt tokens prompt evaluation completes the model generates 99 tokens”
“What is the issue? The ollama available from homebrew is broken, it cannot provide structured output when using mlx models. Relevant log output OS macOS GPU Apple CPU Apple Ollama version 0.34.2”
“What is the issue? On Ollama 0.34.0, the official qwen3:14b model declares the thinking capability, but a non streaming /api/generate request with top level "think": true returns the complete <think ...</think block inside response and leaves thinking empty. The equivalent /api/chat request correctly returns the reasoning in message.thinking and only the final answer in message.content . Both requests below are deterministic, use the same model, prompt, seed, temperature, and output limit, and f”
“What is the issue? Environment Ollama: 0.34.1 Client: SillyTavern 1.18.0 CPU AMD GPU Radeon RX 9060XT (16GB) Deployment: Ollama and SillyTavern running in Docker containers Model: Reproducible with multiple models Found this PR that is most likely breaking: https://github.com/ollama/ollama/pull/18448 Description After upgrading Ollama to 0.34.1, SillyTavern conversations fail with: The problem is that SillyTavern exposes typical p as part of its AI Response Configuration, but does not provide a”
“What is the issue? Using https://github.com/ollama/ollama python/releases/tag/v0.6.2 <img width="538" height="34" alt="Image" src="https://github.com/user attachments/assets/d1c66acf 6ae6 4438 895a 6536575f5f95" / New error from the server. I saw the PR https://github.com/ollama/ollama/pull/18448 about TP but I thought it was still allowed. Relevant log output OS Windows GPU Nvidia CPU AMD Ollama version v0.34.2 rc0”
“What is the issue? Format output for qwen 3.5 35b model does not count thinking tokens as eval, but adds them as prompt eval. This is the request params: response = client llm.chat(model="qwen3.5:35b", messages=messages, context length= 40000, top p = 0.95, top k = 20, temperature = 1, repeat penalty=1.5, max tokens=6000, stream=True, format=ProcessedTranscription.model json schema(), ) The problem can be identified by changing prompt eval for the same request running several times. Relevant log”
“What is the issue? OpenAI SDK is unable to parse structured output from gpt oss:20b responses. Ollama is supposed to be compatible with OpenAI SDK structured outputs per this Blog Post. Reproducer: Relevant log output OS macOS GPU Apple CPU Apple Ollama version 0.11.0”
“What is the issue? Here is the script to reproduce Output: Note {" occurs twice at the beginning of response . If the thinking mode is turned off, response is a valid JSON: (It seems that when structured output is requested, thinking process is omitted, but that's another topic.) Relevant log output OS macOS GPU Apple CPU Apple Ollama version 0.9.0”
“Problem : As far as I know, structured outputs and reasoning models / thinking mode are currently incompatible. As I understand it, using format suppresses next token probability in the model output for any tokens not complying with the desired output structure, thereby "forcing" structured generation. I assume the reason for the incompatibility with reasoning models is that the probability for <think token also will be set to zero, effectively disabling thinking mode for structured outputs. Pro”