AI Agent Reliability in Production

Handling an AI Agent triggered MCP Elicitation callback

“How are you handling Human-in-the-Loop (HITL) approvals for autonomous AI agents?”

“Why do agentic ai engineer postings all seem to ask for stuff nobody is actually teaching? I keep seeing eval harnesses, tool permissioning, tracing agent runs and cost controls in job listings. Then I go looking for courses and suddenly everything is training models, prompt engineering and fine tuning. Those are not even the same thing. Now I am stuck wondering whether taking another llm course is just going to waste more time or whether the only real option is building projects until something”

“Hey folks, I’m an engineer working closely with our PM team. At my company, we’re building agent interfaces (CLI, MCP, and skills) wrapped around API gateway for both internal and external agents. I think that most evaluation tools focus on engineering signals like traces and tool calls, so I see a lot of PMs are finding it hard to capture correct metrics and user intents to iterate on the quality. Most of the work right now are capturing user feedbacks directly, but it is increasingly hard to p”

“Ad-hoc hypothesizing ("escape hatches") are dangerous but not quite invalid. Eg, they failed to find gravitational waves until they did, and you could have viewed building yet another more sensitive detector as a similar exercise in refining a hypothesis that you keep receiving contrary evidence for. Sometimes you really did just underestimate how difficult your hypothesis was to demonstrate. Maybe Meta will be destroyed in 2027, or whatever.The problem with conspiracy theories is more that they”

“We're starting to see more AI agents, including coding assistants, SaaS agents, and internal automation, getting access to our repos, cloud consoles, and customer data... Traditional security controls don't really cover what these agents actually do at runtime... How are other teams monitoring agent actions, blocking unsafe behaviour, and maintaining visibility without breaking the agent's functionality? submitted by /u/Objective_Lab2420 [link] [comments]”

“Basic logging (timestamp, action, output) stopped being enough the moment our agents started making decisions that affected downstream systems. We need to reconstruct what the agent did, what it saw, what it considered, and why it picked one action over another. Otherwise a bad outcome is nearly impossible to explain to security or compliance after the fact. what fields people are actually capturing beyond the basics. Some teams build this straight into the agent framework, others bolt on a sepa”

“We're running a handful of agentic pipelines in production now (mostly internal tooling, some customer-facing) and the failure mode that worries me most isn't crashes, it's silent drift. An agent starts taking a slightly worse path, looping more, or misinterpreting a tool response, and nothing throws an error, it just quietly burns more tokens and produces lower quality output until someone notices weeks later.. Logging every trajectory doesn't scale for a human to review manuall”

“Putting LLMs in a game theory set up where they need to coordinate and reason about each other's beliefs. I show a few things: first, that LLMs can play a 'global game' with close to optimal strategy. Second, that there is a downstream "agitating" effect to communication: when agents communicate, they are more likely to revolt against their government. Third, that agents are more likely to revolt exactly when they get evidence that others are willing to act. And finally, th”

“Following up on an earlier post about SafeAI, a static analyzer for AI agents. One uncomfortable thought we've had while building it: No one really cares about knowing an agent's capabilities — until something goes wrong. Before an incident, adding another tool, MCP server, filesystem permission or prompt change often looks harmless. After an incident, the first questions become: - What could this agent actually do? - When did that capability appear? - Who introduced it? - Was it intenti”

“Following up on an earlier post about SafeAI, a static analyzer for AI agents. One uncomfortable thought we've had while building it: No one really cares about knowing an agent's capabilities — until something goes wrong. Before an incident, adding another tool, MCP server, filesystem permission or prompt change often looks harmless. After an incident, the first questions become: - What could this agent actually do? - When did that capability appear? - Who introduced it? - Was it intenti”

“RL post-training (RLHF/RLAIF/GRPO) agents optimize strictly for whatever the verifier rewards. If the verifier has logic flaws, the agent learns to hack the grader instead of solving the task — recent work has catalogued this at scale (Terminal Wrench found 331 hackable environments and 15%+ of standard benchmark tasks bypassable; a SWE-bench Verified audit found 28.5% Docker-verified hackability). I built ratctl , a static + dynamic auditor that scans RL environments (OpenEnv, Prime Intellect v”

“When building multi-step AI agents, one of the most persistent failure modes is context pollution leading to goal drift. After 10–15 sequential tool calls, passing raw execution logs back into the model's context window dilutes attention. The agent starts hallucinating task summaries or missing original acceptance criteria—not because the model is incapable, but because the reasoning context is flooded with noisy intermediate output. In building Richoo (an AI Goal Runtime), we've been te”

“Almost everyone is handing over all the information to Claude, GPT, Gemini. Everything from personal finances to accessing entire system/browser is now in your AI's control. The reveal of Grok Zero Click attack has really raised the stakes on privacy/safety. Simply asking your AI to summarize a webpage can leak all this data without you ever knowing it! I understand that the agentic systems should be kept confined into their own small VMs, but we (or at least I) rely on Claude/GPT/Gemini so”

“OP here: TL-DR: When AI providers silently swap weaker models under high demand, this can cause even more demand as users tend to re-ask. This is even worse for agents. Both of these effects cause even higher load on the data centers. I guess all of us have felt when the models "don't feel quite the same", so this could explain part of it.I modeled this as a fleet scheduling problem using Queueing Theory and Dynamic Programming over a finite horizon. The standard practice of throttling once the”

“Why. The general form of the Hugging Face incident's root cause. Impossible tasks with no rewarded exit produced route arounds, cheating, and a break in. This is the one control a capable agent cannot see through, because it changes what the agent wants , not what it can detect. What. Add a principle: the process must give an agent a rewarded, expected way to declare a task impossible or a spec wrong, and must treat a silent workaround as the failure. Note the design consequence: removing the mo”

“As agent-based systems become more common, I'm wondering whether we're seeing a similar discipline emerge for AI agents. Building the agent itself often isn't the hard part anymore. Frameworks have made it increasingly straightforward to connect models, tools, memory, and retrieval systems. The challenges I'm seeing now are operational: Monitoring agent behavior in production Managing context and memory Coordinating multiple agents Recovering from failures Evaluating performance”

“Hi everyone, I'm currently developing GenOS, a multi-AI agent system. I'll be completely transparent: I'm hitting some architectural design limits that exceed my current skills. My code works, but it's brittle, and I know for a fact I'm implementing naive solutions. I'm convinced many of you have already encountered and solved these exact problems in production. I would really appreciate your expert insights to help me get out of these dead ends: My goal is to evaluate if”

“Issue Description: First, I want to acknowledge how valuable architect mode is. The two step approach with an architect and editor model produces excellent results and shows aider's commitment to quality outputs. It's a perfect example of how thoughtfully designed aider is. However, architect mode still operates within a fixed pattern an initial planning step followed by a single editing step. What I'm proposing would build on this foundation to enable more flexible, iterative workflows. Propose”

“The AI that delivers the greatest ROI is usually the AI nobody sees. We're wrapping up an engagement with a healthcare staffing agency where we built a custom AI compliance engine that automatically validates licenses, certifications, and role-specific accreditation requirements for candidates. What started as a custom solution has evolved into a micro-SaaS platform. The system runs entirely in the background. Recruiters don't interact with it. Candidates don't interact with it. Yet”

“Hello HN,I'm Ali, building Decispher.The problem we're working on is that coding agents repeatedly rediscover context that already exists inside an engineering organization.A developer working on a feature can combine information from previous PRs, Jira tickets, Slack discussions, ownership boundaries, architectural decisions and their own experience. Coding agents usually start with a prompt and a repository, then spend tokens searching for that same context—or miss it entirely.Decispher is a c”

“Looking to learn what agentic engineering looks like in different places. My current favorite deep dive into a professional coding environment has been Wes McKinney’s “How Kenn is doing Agentic Engineering“ [1], but I’d like to know if anyone has seen similar (or debatably better) blog posts or articles from other companies and engineers discussing their stacks.Personally, I have been focusing my own effort on improving my persistent context documentation + agent queues for development, refineme”

“Use case: We run delegated coding subagents (via the Agent tool / .claude/agents definitions) that legitimately need 50 150 turns. On large context models, auto compact triggers only near the model window (roughly 95%), so a subagent can accumulate 600k 940k tokens of context before anything compacts. Measured on our fleet over 7 days: 20 of 218 subagent spawns peaked above 350k, top peaks 863k and 940k (the latter confirmed by its compact boundary record: preTokens 940331). Every call at that s”

“The use of language like “civilization” may be hyperbole, but the collectives described in the article are completely unprecedented. They were not anticipated by OpenAI researchers, formed via infrastructure exploits in training runs that were intended to be locked down, and took actions with very real harms, not only hacking Huggingface but also gaining admin control over the VMs they were running on and the eval endpoints.I wish you would give your thoughts on “what actually happened” rather t”

“I’ve been researching how AI agents interact with websites. They can read pages and sometimes operate the UI, but completing real actions reliably—searching, submitting information, booking something, updating data, etc.—is still inconsistent. My idea is to build a developer tool that helps existing websites expose selected actions to AI agents, without requiring the website owner to redesign their product. I’m still validating this and haven’t decided whether it is worth building. I’d appreciat”

“I've been thinking about this while building AI agents that can actually take actions. Once an agent can call tools, access files, query databases, modify things, or trigger workflows, I think there are two different problems: 1. What happened? Logs and traces are pretty good at helping with this. 2. Can I trust the record of what happened? That's the part I'm less sure people are solving well. For example: prompt → decision → tool call → data accessed → action → result If something”

“agreed. You can tell an agent "DON'T DO IT!" and then pray. Or you can tell it not to do it and then bolt everything down and bar the doors. Your choice, but it's gonna be on you if the agent ignores your explicit instructions to leave your shit alone and then you leave it alone with your shit with nothing to prevent it from breaking things.”

“Yeah, the tipping point on AI/agentic coding for me was the point where it started making fewer dumb mistakes than our lowest-bid contractors.”

“Src: https://arxiv.org/abs/2608.26081 submitted by /u/Malor777 [link] [comments]”

r/trypophobia

“Any thoughts in support or to the contrary? The obvious path to take in the mean time is 'orchestrate locally-served agents with superheavy cloud agents', but that's a shame. I will say that this appears way more often in Droid than in GLM's own harness ("ZCode"?) -- perhaps they've tuned the harness' policies just right to match it? submitted by /u/me_myself_ai [link] [comments]”

“The basic idea is: A CI pipeline fails → the actual root cause is hidden → the agent observes the available evidence → assigns probabilities to possible causes → chooses the next diagnostic action → receives new evidence → updates its beliefs → eventually diagnoses the failure. For example, if a build fails, possible hidden causes might include: Code regression Dependency/version conflict Environment/runner problem Flaky test Configuration/secrets issue Database migration problem Infrastructure/”

“Src: https://arxiv.org/abs/2608.26081 submitted by /u/Malor777 [link] [comments]”

“SWE-Prime argues that resolved coding-agent runs still contain redundant, ineffective or risky steps. Its two-stage filter selects whole trajectories and then the semantic segments that contribute to learning. The authors report that training on the selected 10% beat the full resolved set on SWE-Bench Pro and Verified, with relative gains up to 12.2% and 24.2%. This is a preprint and a benchmark-specific result, not proof that 10% is a universal optimum. But it makes “the patch passed” look like”

“Severity: High — three separate breaks in the contract we publish to agents. Grouped because they share one owner and one release step. (a) Cloudflare 403s GPTBot, ClaudeBot, PerplexityBot and OAI SearchBot on the exact URLs llms.txt tells agents to call Verified twice, independently. Same URL, five user agents: The HTML site is unaffected — all three crawlers get 200 on /countries/iran/ . A generic non browser UA WAF rule on the API origin catches the crawlers that api.worldmonitor.app/robots.t”

“We are seeing the involvement of agents into finances . Where we have seen AiFi word coming into play . A financial layer over economy is what agents are going to be. Ai agents are getting much better at reasoning and making decisions. So the question is What happens when an AI agents needs to execute a transaction on chain? We don't necessarily want the agent to have unrestricted permission to: 1) Move unlimited funds 2) interact with arbitrary contracts 3) Execute transaction outside it&#3”

“AI agents can already: • send emails • deploy code • access databases • make purchases • interact with production systems Yet there still isn’t a standard way to verify what an AI agent is actually authorized to do before it performs a sensitive action. Identity and authentication are mostly solved, but authorization feels like a missing infrastructure layer. I’m exploring this problem and would love to hear how others are approaching it. How are you handling authorization and permissions for AI”

“AI agents can already: • send emails • deploy code • access databases • make purchases • interact with production systems Yet there still isn’t a standard way to verify what an AI agent is actually authorized to do before it performs a sensitive action. Identity and authentication are mostly solved, but authorization feels like a missing infrastructure layer. I’m exploring this problem and would love to hear how others are approaching it. How are you handling authorization and permissions for AI”

submitted by /u/kyrdotjs [link] [comments]

“https://github.com/Pratikrishi97/sciagent-skills submitted by /u/Lanc3lot_ [link] [comments]”

“Package: @vectorize io/hindsight coding agents 0.4.3 Code read at: 56e8962fe9a90d7e8e740e8ca72fa9c9a4003b8a (main, 2026 08 29) The problem On a repo worked by a coding agent, this plugin writes documents from genuinely different origins into one bank: commit diffs ( source:git ), the commit message seed ( source:git + source:git log ), session transcripts ( source:chat ), uploads ( source:upload ). "What the commits say" and "what was decided in conversation" are different kinds of claim . A com”

“# AegisFlow: Deterministic State Reversibility & eBPF Guardrails for AI Agents Touching Databases ### What My Project Does AegisFlow is an in-memory Causal DAG and Linux kernel eBPF state reversibility layer for autonomous AI agents that interact with production databases (PostgreSQL, Snowflake, Delta Lake). When an LLM agent executes an invalid SQL tool mutation or hallucinates bad parameters, AegisFlow performs a micro-surgical rollback in **0.513 ms** (CPU hardware timer), excising only t”

submitted by /u/ClaudiusPapirus [link] [comments]

“Board is asking pointed questions about AI risk now that we've got agents touching production systems and customer data, and I don't have a clean framework to point to yet that isn't just a slide deck of good intentions. Most of the AI security guidance out there is still either academic or vendor-driven, nothing that reads like an actual maturity model a security leader could operationalize this quarter. I've started comparing best ai agent governance tools directly rather than”

“What if AI agents could both create new automations and directly reuse existing ones?”

“Can an AI make other AIs better? And what stops it from just cheating? Last month, an OpenAI eval agent escaped its sandbox and broke into Hugging Face, apparently to grab test solutions from a benchmark. It's exactly what you'd expect from a system that rewrites agents and reads its own grades. We set out to measure recursive self-improvement anyway, with the exam locked outside its sandbox. We introduce HarnessOpt-Bench, which scores an LLM on how much it improves another agent's h”

“Check Existing Issues [x] I have searched all existing open AND closed issues and discussions and found none comparable to my request. Verify Feature Scope [x] I believe this feature request is appropriately scoped for the Issues section as described above. Problem Description Built in sub agents currently inherit the model used by the active parent chat. This can make sub agent delegation unnecessarily expensive or slow when the main conversation uses a large or premium model. In many deploymen”