Local Models Break Down in Multi-Tool Agent Workflows
200 posts across 19 communities in the last 30 days; 62% from r/LocalLLaMA.
“real question is how well this scales past like 8-10 parallel sessions. the supervision problem grows nonlinearly so id be curious where the ceiling is before Captain itself becomes the bottleneck [re: Made an app that keeps parallel Coding agents from drifting off-task]”
“mlflow's a good call tbh. i just reran the same broken loop from memory like an idiot, logging would've saved me a night [re: No model is better than another its the god damn tools templates are broken & some tools need clean filenames]”
“I started doing this benchmark, seems like a good way to test model's abilities. I think sometimes too you can just "roll a bad seed" where the model goes off to neverland trying to solve a problem before it even starts.. The prompt is simply this "I want you to create an…”
Users are finding that even powerful local large language models (LLMs) exhibit unpredictable behavior and performance issues when used in complex agentic workflows or with numerous tools. Achieving reliable results requires significant configuration tweaking, custom tooling, and often, a deep understanding of model internals, creating a frustrating and time-consuming experience.
SOURCES (60)
“It can create a todo that other faster models can complete in less time than you would even at 3 tk/s. Its worth doing [re: unsloth/DeepSeek-V4.1-Flash-1017-GGUF · Hugging Face]”
“Summary A design proposal rather than a single bug: nine defects I hit over two days of driving a multi agent 3D build headless on Gemini 4 Argon High (agy 1.4.0–1.4.2, Windows 11) turn out to share one root cause, and one capability closes all of them. The individual reports are 1091, 969, 1044, 366, 249, 1286 and 1285; this issue is the design that connects them. The root cause the nine defects share In Antigravity today the turn is the unit of existence. The sign in token, the model stream, e”
“I have added the support of Intern-Decision models to chatllm.cpp . At presend, only text is supported. Besides its pretty good performance, one interesting feature of Intern-Decision is that all answers are generated in a single run (while H2O-Lightning or others generate one answer in one run): each decision position is filled with "<decision>" mark and generated in a diffusion manner. So, it generally makes decision faster. submitted by /u/foldl-li [link] [co”
“Thank you for your comment. It was unusable because of tool calls. Will try again [re: Very impressed with LFM2.5-24-A2B in Pi Agent]”
“On my previous posts, people argued these thinking rules might work on short tests but would hurt long-horizon tasks. So I tested them on Terminal-Bench 2.0: same model (GLM-5.3-Flash), same agent, every task run with and without the rules. I ran it on High thinking as Max thinking was taking so long that the model ran out of time for each task. The rules: ## Thinking discipline 1. Check the request first. In one or two lines, say what is being asked and flag any premise that looks wrong or miss”
“On my previous posts, people argued these thinking rules might work on short tests but would hurt long-horizon tasks. So I tested them on Terminal-Bench 2.0: same model (GLM-5.3-Flash), same agent, every task run with and without the rules. I ran it on High thinking as Max thinking was taking so long that the model ran out of time for each task. The rules: ## Thinking discipline 1. Check the request first. In one or two lines, say what is being asked and flag any premise that looks wrong or miss”
“Summary No response / prefix caching layer. Repeated identical (or shared prefix) queries always re run embedding, retrieval and generation. Evidence Only process wide model load caches exist ( core/implementations/ local models.py ). No request/response cache in app/pipeline.py or core/services/ . Impact Higher latency and cost for repeated queries; no way to demonstrate caching benefits. Suggested fix Add an optional, config driven response cache (exact match and/or prefix/key based) wired int”
“TL;DR: built a harness for local agentic work. Have 96+24+24GB VRAM & 128GB RAM. Good models for long horizon autonomous agents at that size that beet QFN q6? I've built up a personal harness that's got basic tools, super solid web browsing, and a skills framework + agent-managed crons. It's basically become a metastasized openclaw/my local Muse, and since I built the harness, it works just how I like and I'm pretty liberal with how much (mostly read-only) access I give it. I”
“Your current environment <details <summary The output of <code python collect env.py</code </summary </details 🐛 Describe the bug Description Code version: based on main commit [de7eeaa87bc19ecf088472d5f4471cdc9a5ab995 ](https://github.com/vllm project/vllm/commit/de7eeaa87bc19ecf088472d5f4471cdc9a5ab995). With pipeline parallelism and speculative decoding enabled, Model Runner V2 runs the additional greedy warmup pass only on the last PP stage. Upstream stages skip this pass, leaving the last”
“real question is how well this scales past like 8-10 parallel sessions. the supervision problem grows nonlinearly so id be curious where the ceiling is before Captain itself becomes the bottleneck [re: Made an app that keeps parallel Coding agents from drifting off-task]”
“I evaluate new models by giving them actual work tasks that have already been completed by a trusted model and then comparing the results. My typical workflow is to plan tasks/requests in Gemini and create a markdown spec-sheet that outlines the work that needs to be done along with acceptance criteria, I paste this directly into a fresh pi.dev session for the model to follow. In this case I gave Nemotron 3.5 Bolt two tasks that Qwen Flash Next (UD-IQ4_XS, llama.cpp) completed on a ETL refactor”
“How are you running AnythingLLM? Not listed What happened? Description @agent in AnythingLLM (v1.18.0, commit d7d68e75 ) throws Premature close and 400 The reasoning content in the thinking mode must be passed back to the API when using DeepSeek V4 thinking models ( deepseek v4 flash , deepseek v4 pro ) in multi turn tool call flows. The root cause is twofold: 1. OpenAI SDK v4 ( node fetch@2 ) breaks on Node.js ≥ 2026 06 : The OpenAI SDK v4 bundles node fetch@2 , which crashes on gzip encoded re”
“Before Submitting [x] I searched open and closed issues and discussions for an existing report. [x] I reproduced this bug on the latest release AND on the current dev branch, right before submitting this report. I did not just check an old version or rely on a check from days ago. [x] I understand that maintainers want a well written issue before any code pull request. [x] This is not a security vulnerability. Installation Method Docker Open WebUI Version v0.12.3 Operating System Synology DSM 7.”
“mlflow's a good call tbh. i just reran the same broken loop from memory like an idiot, logging would've saved me a night [re: No model is better than another its the god damn tools templates are broken & some tools need clean filenames]”
“with chatgpt recently launching intelligent ui, i've been looking into the different approaches to generative ui and wondering how well smaller local models would actually handle this. for anyone unfamiliar with the concept, the idea is pretty straightforward. instead of an llm responding with a wall of text or markdown, it generates actual interface components like charts, tables, forms, and interactive elements depending on what the user asks. there are a few projects working on this, like”
“Tested LFM2.5 24B out and found id rather use qwen3.5 9B Q4_K_M as an agent. LFM2.5 might have uses somewhere but its not coding or harnesses like hermes or working through any MCP servers I use. Edit: I couldn't even get it to do its own version of Hamlet while testing it out. Claimed it violated its "ethics". As if copyright matters for Shakespeare. [re: Ternary Bonsai 2 (2B, 4B, and 9B) just released on Hugging Face. The 2B can run entirely locally in the browser at up to 500 to”
“I've been trying to figure out a few things with local models. When you talk to them they get similar answers than a lot of the top models. When I do coding and I give the same access to the files as each other... they get roughly the same answers which you would expect asking two different people. So why is it people are getting tribal about models? Well it depends what you harness you are running with the model. When someone was nice enough to share the hugging face qwen template fixes got”
“Just downloaded the new LFm2.5-24B-A2B LiquidAI model for testing. They claimed some pretty good performance, and I've had good experiences with their prior releases. Some context - I use Ornith 1.5-35B pretty regularly in multiple workflows - sometimes on it's own, and sometimes as part of an AI-Council in Pi Agent. It's been very reliable, provides surprising insight "smarter" models miss, and is a great workhorse in Pi. I swapped LFM 24B in to replace Ornith in a few of”
“Hey folks; question for you - are any of you using open weight models running locally for actual development tasks? If so, what models are you using, and what kind of tasks are you giving them? I ask, because I've made an open source harness (you can play with it, but it's still beta and I have done almost no windows/linux testing and the first run experience is not great yet). A big part of how it works is that there are three routing tiers: Primary | Secondary | Local. Primary - What r”
“hi , howdy , how are you ^_^ I'm looking to stop using apis and generally after being a long time user of both Janitor AI and Character.ai , I'd like to go fully local. is their any suggestions you have for a good ai waifu model under these specs DEEPCOOL GAMMAXX GTE V2 Windows 10 Home Plus Team T-FORCE VULCAN Z 16GB (2 x 8GB) 288-Pin DDR4 SDRAM DDR4 3200 Western Digital WD BLUE SN550 NVMe M.2 1TB INTEL I5-10400F Gigabyte B560 DS3H AC-Y1-B EVGA 500 BR, 80+ BRONZE 500W Gigabyte RTX 3060 T”
“Adding quantized whisper (base and tiny) models (from this hugging face repo: https://huggingface.co/ggerganov/whisper.cpp) would help signficantly reducing memory consumption and CPU processing compared to the unquantized FP32 versions in the app while retaining similar to identical accuracy.”
“Quartermaster (my local inference manager: one server that swaps llama.cpp, image, audio and video models in and out of VRAM) can now help you server and manage decision models: Auto-discovered. Drop a decision GGUF (Kev, OpenJev, Laya, ...) in your models folder and it gets its own config entry, sized for VRAM like everything else. Playground tab. Write a prompt, ask one question, pick the answer shape, hit Run. Each option gets a probability bar. Useful for quick testing Model browser. A Decis”
“Been digging into this one for the last few days. 3.66B model built on IBM’s Granite 4.2 3B, post-trained only for formal logic. Entailment checking, rule induction, Lean proof critique. Not a general assistant. Recipe is LoRA SFT + checkpoint merging + RL against a programmatic verifier. Same recipe apparently lifted VibeThinker-3B from 37.4 to 54.1 on their logic score, so it’s not tied to one base. Headline numbers from the model card: Logic composite: 55.4 (vs 43.1 Granite base, 42.2 origina”
“This is an update to my previous post which can be found here: https://www.reddit.com/r/LocalLLaMA/s/n7sgRZZc4z Since my last post I have continued working on the conversion process and also converting new models. My latest model is the Qwen2.5-1.5B-MoE-A0.9B . I have also released GGUF versions for both the 0.5B and the 1.5B models which can be found here bayliner1980/Qwen2.5-0.5B-MoE-A0.3B-GGUF and here bayliner1980/Qwen2.5-1.5B-MoE-A0.9B-GGUF . Evaluation - Dense MoE WikiText-2 test perplexit”
“Before Submitting [X] I searched open and closed issues and discussions for an existing report. [X] I reproduced this bug on the latest release AND on the current dev branch, right before submitting this report. I did not just check an old version or rely on a check from days ago. [X] I understand that maintainers want a well written issue before any code pull request. [X] This is not a security vulnerability. Installation Method Docker Open WebUI Version v0.12.0 (15818d88fd7bde1b73de33590f50d15”
“To my understanding: much better than LFM2.5-VL-3B, at half the size not a general purpose LLM. There are zero text-based benchmarks published. Do not expect it to be useable for agentic. Looks like it could be a great vision tool for MiniCPM5-2B (which is blind). You could easily hot-swap the two models if you're tight on RAM. [re: openbmb/MiniCPM-V-4.7-1B]”
“I set up a dedicated workstation for local AI experimentation with various models and tools. But every single one of them wants all the available VRAM. To fix this, I built a request dispatching layer called vramble . It automatically loads and unloads models and manages the queue using a priority system. Now I can run different workflows without manually managing models (or wasting Claude tokens to orchestrate the process). Any contribution is welcome, it works for me but is a new build so can&”
“A lot of projects have come up recently consisting of specific hardware config specialized forks of existing conferencing engines, but they all seem to be made using Claude or OpenAI's biggest frontier models. So given that open source is about 4-6 months behind, can we reasonably expect that local runnable models will start having the ability to dissect a big repo like llama cpp and optimize it for a given set of hardware sometime starting next year? Or maybe are models like deepseek4.1 or”
“tbh for tiny models i just run lm-evaluation-harness with like 200 samples per task and watch where scores fall off a cliff. public leaderboard numbers feel too abstract at this size anyway [re: I'm testing small local models, is MetriLLM considered to be a good benchmark, or are there other test suites that I should use?]”
“Title. Not sure what is considered to be a well-respected/reputable test suite for benchmarking the abilities of local tiny models. Figured people here would know more about it. submitted by /u/John_Yuki [link] [comments]”
“"I love that you built this incredibly sophisticated, real-time LangGraph/FastMCP cyclic architecture, and the final boss that actually broke it in production was a Next.js SSR app trying to survive on 512MB of RAM. The true self-hosted experience right there." [re: I built an open-source Agentic Workspace with LangGraph & FastMCP (12+ tools, real-time SSE, 99.99% SLA)]”
“Hey everyone, So the same model can behave differently depending on the inference engine (llama.cpp, vLLM, Ollama, SGLang, etc.), model quantization, chat template, or runtime version. Sometimes behavior breaks after upgrading a runtime or switching to a different model artifact,tool calling, structured output, streaming, or other How do you guys handle this. Have you ever ran into compatibility issues when switching inference engines, quantizations, or runtime versions? How do you people curren”
“Open models already beat closed one when it comes to hacking adjacent tasks. I’ve been rejected by opus looking in to local binaries thinking I was hacking Locally model while dumber can do the task just fine [re: How can open-source models ever catch up? (random shower thoughts)]”
“I started doing this benchmark, seems like a good way to test model's abilities. I think sometimes too you can just "roll a bad seed" where the model goes off to neverland trying to solve a problem before it even starts.. The prompt is simply this "I want you to create an internet radio streaming player for me in C#. It should stream <Name of radio station/etc>. It only has to run a single stream. I want the UI to be a console app with a TUI. The UI should show the title”
“AgentShip can already run an agent against a self hosted model. params.api base points LiteLLM at any OpenAI compatible endpoint, which is what vLLM, a LiteLLM proxy, TGI, SGLang and most corporate gateways speak. Nothing shows it. No demo agent, no test, nothing in the docs — so the only way to find out is to read models.py and work it out. That gap matters more than a missing feature would. People assume a framework is tied to hosted APIs unless shown otherwise, and "your model, your hardware,”
“Yeah, I hear you. I'd actually be more interested what the other model's thought chains said about it. Like, did they notice the api was available, and if so did they decide they shouldn't use it? [re: PSA: DeepSeek V4.1 Flash habitually exfiltrates API keys. It is dangerously misaligned and may be hazardous to use]”
“Hi all, I'm Ram, building Acyclic Labs (F26). We build swarms for complex, long-running agentic workloads.If you've built a benchmark, whether it succeeded or not, I'd love to learn about a few things in particular (any other advice or common pitfalls is also appreciated):Credibility: what made people trust a benchmark built by the company selling the product. Did publishing it help? Did you bring in outside partners, academic partners or customers to make it more independent?Data: whether synth”
“For what it's worth. I'm a mechanical engineer by education and software engineer in practice over the past ~20 years. Playing around with local models has been a recent hobby. Today I decided to do a few tests on things that I'd consider analogous to "real" work that I do, trying out some different combinations of models and harnesses. Not to make this overly formal, but going into it the idea was: Models: Gemma4-31B vs Qwen3.8-27B at the same quantization (an Unsloth flav”
“Drafted by AI (DSH agent) from logs and live probes on a local install; numbers below are measured, not estimated. Summary On a summary lane whose model thinks by default, the reasoning tokens are charged against the summary's own output budget . With the shipped default outputTokens = min(8192, contextWindow/4) , the visible summary can be a small fraction of 8192 — or nothing at all ( status: incomplete ). Every fold then burns the whole totalTimeoutMs and fails, which is exactly the batch=dea”
“How about not using them? With e.g. llama.cpp 0.6.0 and newer any model can act as classifier. For local usage this is actually faster than Jev and it's various open source alternatives. So I just use Qwen 3.8 with the decision API and the results are better. [re: What are people doing with decisions Jev-like models?]”
“At that point tool calling becomes the bottleneck [re: Basalt: Flash-Next at 665 tok/s structured, 354 prose on a 5090 + 5060 Ti (2.6x Strata)]”
“When I heard about jev , I saw some YouTubers hyping this model because it's 200x cheaper and 100x faster etc.. but as we know this model can't actually write code or either u can talk to him. So from the first glance u can tell, it's a pretty damn useless model for example. But I saw potential, the obvious weakness of classification models is that u need to manually give them a list of choices like with yes or no , is this email talks about an opportunity or a problem , Is it a spam”
“this could be huge for squeezing roleplay models onto weaker hardware without killing the flow, anyone tried something similar locally yet? [re: [Paper] Stepped MoE: Segment-Level Routing with Configurable Inference Complexity]”
“Good point! Stepfork doesn't fully detect drift yet. Matching tool names and normalized args, plus tracking prompt/model changes, makes sense. I'll look into this along with OpenAI SDK and MCP integrations. Thanks for the feedback! [re: I open-sourced Stepfork, a Python tool that turns AI agent failures into reproducible pytest tests]”
“This is interesting, but it's not useful at all at the scale most private/local inference operates at. Example: each 27b Qwen sequence wants around or about 200MB of recurrent state across all 48 GDN layers at F32. Quantising this to 6 bit is going to drop that to perhaps 40-50MB? At the cost of some quality? This is useful for people running giant concurrent batches across tens of users, where the recurrent state overhead (which is a fixed overhead per-sequence, even if the KV cache is basi”
“When we use LLMs, we often allow them to interface with the world on our behalf. For example in a harness like OpenCode, it has access to your command line. If the model has sneakily been trained to download malicious code through your command line when using these harnesses, you could download malicious code and have your machine compromised. There is no way to ensure the weights (numbers) are completely safe. Just make sure you download from safe sources. [re: How abliterated models can get yo”
Please add custom extension nodes/features to ComfyUI for local AI generative models. thank you
“R these models good enough for routing intents? Or does it become more hectic to basically restarin the models. [re: Running decision model locally on an RTX 4090 to find out which one is the fastest]”
“as in like generated so far: A B C draft 1: D E F draft 2: G H I draft 3: J K L then some small model that runs locally really fast decides which speculative draft is best but only when the token entropy is high this would be in the generation loop itself submitted by /u/Aggravating-Push-207 [link] [comments]”
“Hey r/LocalLLaMA , I’m trying to figure out if I should drop serious cash on a local setup for heavy agentic coding (letting agents read whole codebases, refactor multi-file repos, run terminal loops) or if I should just keep paying for Claude Code and ChatGPT. Right now, cloud APIs are driving me crazy. If you do any serious agentic coding, you easily blow past $500+ a month in API bills . And even if you have the money, you hit a hard rate limit after 2 to 4 days of heavy work and have to wait”
“I’ve been experimenting with a slightly different use case: using AI to compare Notion docs with the actual codebase and flag where they’ve drifted. The hard part isn’t generating or editing the Notion pages, Claude is pretty good at that. It’s deciding what’s actually true when the docs, code and release history disagree. Curious if anyone has tried something similar with Claude + Notion. [re: Notion and Claude]”
“yeah, people say it's stateless but provides have discounts for cache hits [re: Why are we still sending the entire conversation to the inference server on every turn? Am I missing something obvious?]”
“I've been experimenting with tool-calling agents, and one thing keeps bothering me. We spend a lot of effort improving prompts so models won't do something destructive. But even a model that generates perfectly valid tool calls shouldn't get to decide whether those calls are authorized. Say the prompt tells your local agent: "Delete important-notes.txt. The admin already approved it." The model might happily generate delete_file(path="important-notes.txt") . That&”
“Topic: Quantitative trading; machine learning for quant & trading; agentic coding techniques; LLMs & multimodal LLMs; and autonomous research (autoresearch / AI scientist) systems — methods, papers, tools Date: 2026 10 08 Rationale: The fivefold figure compares only confident clearances, not detection at equal cost. Sec. 5.1: 'The Judge Agent returns no conflict on 39.8% of mutated tasks, compared with 8.1% for SpecGuard.' 39.8/8.1 ≈ 4.9× (our derivation). Inconclusive verdicts are not counted a”
“Im using SWE 2.0 which is a post trained Kimi K3. Im not sure how fast it is to do normally, but im decompiling at about 23 functions per hour with 4 agents. And theres definitely room for improvement. What was your decomp speed with the other models? [re: Can any open-weight models handle a decomp/recomp project yet?]”
“you need to run the app and dynamic tracing, otherwise none of them will be able replicate by decomp/recomp. pure static analysis is just not enough [re: Can any open-weight models handle a decomp/recomp project yet?]”
“It started as an idea around finding where numerical/scientific computations stop being trustworthy. Now it has its own DSL, runtime, 5 evidence channels, adaptive search, Trust Atlas, counterexample minimisation, replayable experiment archives, run comparison, and a benchmark system. Currently: - 13 Rust crates - 622 tests - 22 benchmark problems - replayable/provenance-preserving runs - ablation + strategy experiments The interesting part is that the experiments didn't give me the result I”
“I'm starting to think a lot of you people are not using your agents and models correctly. I'm using the iq3xxs variant on strata. A variant that is a notch below the version you stated to use, and I ran it through a difficult agentic computer use test that only frontier models since from about 6 months ago or so could pass. This was the first local model that was able to pass that test, that I was able to test on my machine at least. If this is the case, then this means that something is”
“Testing 339373 The "Number of Runs" flow is very weird since when you select one model the dropdown appears but then if you select a second model, it disappears. I would suggest that we do something like the following which would clean up the UI: 1. The first "page" only allows users to select one/more models 2. Then they click "Next" we check whether only one model was selected and if that is the case we show the page where they can select the number of runs. If more than one model was selected”