LLM Prompts Get Bloated and Output Gets Worse

“Yeah, I don't think that line quite landed. My point was that LLMs are inherently adversarial (read, "relentlessly proactive") and sandboxing should be a first-class integral feature of your harness, not an afterthought. The problem with dev containers is you still tend to end up with something you care about on the same side of the trust boundary as the LLM.”

“A few months ago, I was building an AI application and ran into a problem I kept seeing everywhere. Getting an LLM to understand a document is easy. Getting it to return clean, predictable data every single time isn’t. I tried a few document parsing APIs. They worked well, but for small projects and startups, the cost adds up quickly. I also wanted more control over the output format instead of writing post-processing code after every API call. So I ended up building ParseFlow ( https://parseflo”

Literally one of the examples I gave where this is relevant.

“Does the conditional part hold up in practice? Missing clause is one thing. “Should be here given X, and should say Y” is where I’d expect noise. Do you trust the flags or is there a category it gets wrong every time?”

“Interesting observation. I've noticed the wrapper can make a bigger difference than people expect. The same underlying model can produce noticeably different results depending on how context is retrieved, organized, and presented.”

“I'm looking for sources on LLM interaction processes using human language to get exactly what you ask for. I'm looking for data spanning from regular prompt interaction, depth of details,whether you use system prompts,how do you handle agent tasks, anything in that category. What does your prompt look like when you add a feature, edit, diff,refactor, debug. If you can post a simple prompt to let's say create an auth feature for a SaaS project, what would be your setup, system prompts”

“> "Lack" isn't the right word. "Lacking" is more like it.Yeah, that's fair.> My takeaway is that text is a far richer modality than anyone has expected - and that high end LLMs are often sharp and flexible enough to recognize their weak points and substitute their strengths. I.e. all the LLMs implementing A* to optimally solve pathfinding in ARC-AGI-3 tasks, often unprompted.I agree and disagree with this. I think we've learned a lot of humans are more text based than we thought, but conversely”

“And you are misunderstanding what the point of LLMs are. You cannot sell "oh its actually so smart that it can only answer super complex questions now!" That literally does not make sense or functions as a narrative.”

“It's interesting that there are a few features in the API that the AI can do that are not available in the user interface. One of them is self-relating properties that are one way. There's no way to change that in the user interface but there is a way by asking the AI to do it. Very interesting.”

“What if a small language model could make a much larger one faster without changing the final answer? That's the idea behind speculative decoding. Instead of generating every token with the expensive model, a smaller draft model predicts several tokens ahead. The larger model then verifies them in parallel. If the prediction is correct, those tokens are accepted instantly. If not, they're discarded and the output remains exactly the same. Link: https://youtu.be/P3r3MPPkM4c This is part o”

“Actually, yes. Small language models are under active development. Focused on specialization in discrete topics - coding, language, visual interpretation. They don't replace frontier models, as SLMs can't answer any question on any topic. But they should require a small fraction of the resources of the current frontier models, while having a similar level of expertise in their single, chosen topic. Open weight SLMs could soon take over in spaces like coding.”

“Using Notion AI is not like using ChatGPT or Claude. Even though it does let you use an LLM inside Notion, if you want it to work properly inside Notion, you need to set it up properly. Here’s a post on Reddit I did about how to structure it to work better: https://www.reddit.com/r/Notion/s/FDXWoeIdPe Here’s a free version of a template you can use: https://www.notion.com/templates/agent-001 I have a full system too but start with these free resources and I’ll bet it dramatically improves your N”

“LLM judges are increasingly common among AI teams due their ability to automate decisions that require complex reasoning and analysis. Pairing their reasoning ability with calibrated confidence scores unlocks entirely new ways to work with AI. For one, active learning enhanced prompt optimization uses low confidence decisions to curate a golden set, allowing judges to learn human expertise with lower annotation effort. Additionally, safety classifiers for agents and chatbots can use confidence s”

“Sorry, which definition do you mean?The problem with "reasoning" is that for the most part it happens outside of training e.g. as CoT. The base model knows what it knows, it knows what it's trained on, and it can't really go much farther than that.Now, starting with o3, "reasoning" models are probably (who knows exactly) trained on traces of reasoning, either from automated systems or from human experts, but that doesn't mean they learn any kind of algorithm, just that they learn to reproduce th”

“this is about any long generation where later output has to stay consistent with earlier output. a serialized knowledge base, a multi-part spec, a long agent transcript that must not contradict its own earlier decisions. i hit it writing multi-chapter fiction, which is where my examples come from, but the problem is not about fiction. the failure that actually bites is not quality of the output, it is that deep into a long generation the model has forgotten a detail it committed to early, revers”

submitted by /u/usernamehere93 [link] [comments]

no i'm afraid

“>and this has been tested by feeding in thousands of texts written before 2020.And these text didn't train the model in the first place? I just want to ensure clarity on that.>pangram currently has a false positive rate of about 1 in 10000Says Panagram.The problem with just looking at old text is language is a living thing. Say for example I make up the world 'oklambroahaha' right today. Both humans and AI pick up that word and start using it. Now lets say the model says that anything that uses”

“I've noticed a lot of my actual thinking/research now happens inside LLM chats. The problem is it's spread across ChatGPT, Claude, and sometimes Grok, and each one is basically a silo. I'll remember "I worked this out in some chat a few weeks ago" and then have to dig through three different apps' terrible search to find it. Curious how people here handle this:- Do you copy the useful parts into your PKM (Obsidian/Notion/whatever) manually? submitted by /u/i”

“This is a good example of where it’s important to be more up front about the role of AI in the making of a thing.Making a language that compiles through LLVM is no small task and takes a lot of expertise. Most of the time people do it because they have a point of view and are highly technical.Making a joke language via AI is an entirely different exercise. Not without value but not the same, especially when evaluating what it means about the author.”

“The problem with this analogy is an LLM isn't very much like a "tool" at all.- In cases of practical relevance, you pay a recurring fee to keep using it - not very tool-like.- Rather than doing a predictable action in response to user inputs, you give it imprecise instructions and it does something unpredictable but vaguely correct in response, which then must be thoroughly inspected.- In cases of practical relevance, its actual operation (not just design) is entirely handled by a third party, n”

“I’m not denying that LLMs are useful or can be thought of as „intelligent” in some definition, I just don’t believe that this stateless architecture is what will make „true AGI” work.”

“Ok I get what you mean (I think). At least for me personally, it still feels like a simulation of persistence though. But this might be primarily because there are many errors in persisting stuff (e.g. the model wrongly remembers things that did not happen/I didn't say).”

“For years, one of the cleanest arguments against relational AI was this: That argument made sense when every conversation effectively died at the edge of the context window. But is it still describing what people are actually interacting with? The underlying model invocation may still be stateless in the narrow engineering sense. It receives an assembled input, generates a response, and stops. But the system around the model is increasingly stateful. It can now retain or retrieve: previous conve”

“I really hate the while Terminator Skynet mentality a lot of people have when it comes to current AI models. I only started properly looking into what LLM is and how it relates to AI, but I still think we are too far from having something from media. Again, no expert on AI, I just think that we haven't moved that far yet since the introduction of SIRI on the iPhone years ago.”

“They are, yes and good point, but then still that doesn’t make them autonomous entities in their essence. Their process and available tools could be more autonomous but the LLMs are still the same models.”

I'm with you on this

“I have been downvoted into oblivion on this sub many times for trying to make the same points. It depends on the thread. Some are filled with relatively rational people who understand what an LLM is and how it works; some are filled with delusional dipshits getting high on their own farts and convincing themselves daily that their chatbot is a real boy. The latter will jump down your throat hard if you try to tell them verifiable facts about how these systems work that contradict their worldview”

“I’ve been experimenting with putting multiple LLMs on the same problem instead of relying on one answer. When models see each other’s responses without much orchestration, they often become too agreeable. The obvious fix is to instruct them to challenge assumptions and disagree when necessary. But then the opposite often happens, as they will disagree just because you ask them to. Once a model is explicitly assigned the role of critic or contrarian, it sometimes seems to manufacture objections b”

“LLM output is stochastic, which makes it hard to treat like anything else in CI. wrote up how i turn it into a number i can gate a merge on with Pydantic evals - repeat runs for pass rates instead of flaky single results, a latency budget, and an LLM judge calibrated against my own labels before i trust its score. https://coles.codes/posts/pydantic-evals/ submitted by /u/mattjcoles [link] [comments]”

“It could be done, I have got some credits from a provider that I could use to generate the self distillation dataset, though I am not sure I’ll be able to tune the model at home. Actually, as soon as I have time I’ll also try to estimate the cost of the self-distillation process in number of tokens (Qwen 4B and Gemma 12B ran on my PC so I didn’t need that before, but 27B would be too slow to work here). I’ll make sure the report back here or as an edit in the post, but I think I’ll start the dis”

“Pairwise tests would work better than a 1–10 score: show two outputs and ask which one made the evaluator physically recoil first.”

“A lot of models output can be quite cringey at times. Somebody should make a benchmark for measuring how well models can identify this behaviour, as well as measuring how cringey a specific model is. submitted by /u/ChippingCoder [link] [comments]”

submitted by /u/bgs11235 [link] [comments]

“I'm having trouble understanding what the tension is. For me, keeping things simple means lots of options.”

“been building llm workflows for about a year now and the thing which pops out as a road block is usually not the quality of prompt, retrieval or model choice rather just not being able to answer the basic questions about what actually happened in a given request and which model handled it and what prompt version was live at that moment. whether a fallback rule kicked in or whether retrieval returned what we expected or something stale. with traditional software you get exceptions, stack traces,”

“Read a long piece on the future of LLM reasoning that makes a provocative claim: Chain of Thought is a useful hack but we've started to confuse a readable trace with the actual computation. All in all, "generating text is not the same as thinking." There are two practical problems here: Faithfulness: CoT style traces can decouple from what the model actually "did." u can get plausible steps with a wrong answer, or messy steps with a right answer (so the trace isnt a relia”

“I’ve come to the conclusion that fighting against the indeterministic nature of an LLM is paradoxical, so I thought, “Shouldn’t it just be better guided?” I ended up building a suite that allows for advanced manipulation and visualization of a knowledge base (KB); I’m also working on it for codebases tracing. I named it Silica, it’s essentially a user-controlled LLM via a state machine: if you’re curious, you can check out the repository. It’s a personal, local knowledge management system, and I”

“> So lets hear them.There are few people delivering more in-depth explanations of their claims, like literally thousands of hours of content. You can just look it up. If you disagree, please disagree. Personally I find what they say articulates almost 100% of my independent experience. And I do systems programming in C++ full-time (file systems / desktop app infrastructure). Personally I've made at least 9 serious attempts of building robust and maintainable systems on top of C++ features like R”

“TLDR: I trained Qwen3.5-4B and gemma-4-12b on self-distilled, compressed reasoning traces; compression was section-aware (compute and verification spans remain, fillers/narration/transitions get dropped/compressed). The models match or beat the originals, often by a large margin, while using 2-3 times fewer tokens. Full study, models and code are available. section-aware compression, visualized Hi! For the past few months I have been exploring reasoning compression: I built datasets, trained (ma”

“dude, lol. We own LLM, UQ claim filtering, RAG, tool calling, benchmarks, supervised fine tuning of small language models, BERT-NER, in addition to many classic ML things.”

“" Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity " This paper was accepted to ICML this year. Its main idea is a very simple prompt-engineering trick: "changing the prompt this way led to more diverse sampling". Naturally, it is difficult to provide a rigorous theoretical analysis for something like this. Even if it works, I’m not sure this kind of prompt engineering belongs at a top-tier machine learning conference. Some people seems to call this”

“using it as a guard loop makes more sense than using it as some magical mind reader. like yeah maybe "21 to 42 to 49" is just a decodable intermediate and not literal scratchpad math, fine. but if the intermediate state predicts the model is drifting toward junk output, thats still a knob you can use.”

“Hello All, Anyone who uses Large Language Models (LLMs) in their daily programming workflow has grown accustomed to certain limitations: sometimes the model misses something, sometimes it needs to be nudged back on track. However, what happened during today's session with one of Google's flagship models goes far beyond the definition of "ordinary technical limitations." It is a textbook example of a deep, systemic flaw in the AI architecture—a total loss of control over the con”

Just spot check and trust your gut. Works most of the time.

Llms are not designed for math…. Jeez

“Anthropic recently published their paper on "Global Workspaces" (J-Space) inside language models, showing that looking at internal "workspace noise" (entropy) can catch hallucinations better than just looking at output logprobs. `solarkyle` followed up with a great open-source implementation showing it could route confident errors on TriviaQA. I wanted to see if this actually holds up as a deployable tool, so I ran a stress-test: extracting J-Space entropy from the late layer”

“I would let Notion AI review, rewrite the formula for me. I don't write my formulas anymore. You'll need to give it more information that you have provided here in terms of context, and what you are expecting it to do. You're going to run into the same issue you would with a human developer. No one really likes to work with anybody else's code. They really want to understand: * What the requirements are * What the constraints are * What you expect as an output If you don't kn”

“Thanks for running those, I appreciate you! :-) Maybe the 2B model will be able to imagine the future...”

“Interesting, but they didn’t include a “control” test between multiple apolitical entities. The closest was the person one at the end, but those examples are somewhat loaded too. The results wouldn’t necessarily surprise me though. When trying to influence a model to avoid spitting out Nazi content or something that would be a PR mess, it could ends up resulting in behavior like this. Also explains why Grok happily promotes Nazisim and makes child porn.”

“You can have a smaller LLM help you with polishing a question or finding alternative words.”

“I’ve been using LLMs to draft proposals, SOWs, solution docs and decks, the hard part has always been reviewing those multi-page drafts: * Are the documents fully compliant with requirements? * Are the facts current and verifiable (not stale training data or hallucination)? * Does the math and calculations check out? * Are the facts researched and latest and grounded in sources? * Is the doc articulated well and actually readable and decision‑ready for the target audience? So I built a [small op”

“Yeah I think the other guy is right, I was looking at it and the main connections to LLMs is ChatGPT and Claude, due to their capability of agentic ai.”

title submitted by /u/greatlove8704 [link] [comments]

“ok, which other open llm do you use. do you use any non chinese llm other than gemma?”

“It's important to be specific about what's actually demonstrated here. The paper is significant additional evidence for "access consciousness" in LLMs, which is a different thing from "phenomenal consciousness." The former is a functional capability related to the availability and use of internal information, which you can objectively observe from the outside. The latter is the kind that involves qualia, the subjective feel of experiencing something that is inaccessib”

“LLMs have gotten pretty good at coding by being trained on huge amounts of existing code and documentation. Why do you think this is the use case bringing in the money? AI can help PMs do more for sure but not as fully automated as a lot of coding related tasks”

“Indeed. You gotta admit though, the illusion is quite fascinating...I remember being utterly confused at the idea that predicting one token at a time can produce all the behavior we see in models today. I now do code reviews with AI where I ask it "what do you think?"...when I know full well that it doesn't think. What a crazy time to be alive :)”