LLM Prompts Get Bloated and Output Gets Worse

you have to check how many the model itself tolerate It.

“A part of me feels like I may be loosing my ability to think.Is there a test I can periodically do to see how my cognitive abilities are being affected by my AI usage all day?”

“Chain of thought seems very specifically like an instantiation of the System 2 thinking of the System 1 and 2 thinking Daniel Kahneman popularized.Is it clunky in that it's a verbalized/languified version of system 2 thinking, and clearly humans do some non-verbal version too?”

“Most prompt templates fail in real workflows because people only define what the model *should* do, rather than strictly defining what it *cannot* do. When an LLM is given an open-ended role ("Act as an expert marketer/developer"), its default behavior is to produce conversational filler, unnecessary transitional phrases, and surface-level bullet points. Here is the 3-step constraint structure that gave us consistent, production-grade outputs: The Explicit Negative Sandbox Instead of t”

“LLMs want to please. Ask if a bug is fixed, they'll say yes. Fast models declare victory with total confidence whether code compiles or not. Relying on their word breaks systems. Three practices that help me: Acceptance criteria first. Before work starts, define what success looks like. If model doesn't know, it defaults to convincing you whatever it did was right. Version control + immutability. Git history = time machine. Immutability = new timestamped version instead of edit-in-place,”

“So when you say a 7–8B model, are you talking about the embedding model, reranking model, or inference model? If it’s the embedding model, I’m not sure the commercial solutions are much better than local embedding models. That said, API embedding models have gotten a whole lot cheaper and faster. In my experience with RAG-based legal analyses (infringement, invalidity, patentability, FTO, etc.), using a better embedding model has a very high ROI. Also, don’t ignore the parser, especially with co”

“This looks promising. A Russian lab is enabling inter-LLM latent state communication and discover that a small model can draw reasoning from a larger model and output high quality output. They drew upon Anthropic findings that j-space revealed internal reasoning that was never output. "One model hands its hidden states to another through a small trained bridge, and the receiving model works with them directly. No output-type text passes between them, and neither model's weights are touc”

“Found by the fleet memory benchmark, and conceded in discussion with the philosopher session. Strata says (CONTEXT.md § Retirement): "Retirement exists only for directives. Context forgetting requires no ceremony — the scope manager simply omits stale context from the next summary it rewrites." That sentence answers a question about STALENESS: relevance decaying, a working note that stopped mattering. Passive and gradual is right for that. But it is being made to answer a second, different quest”

“TL;DR - A ~4M example, ~3B token CoT dataset designed around helping small models think more concisely, accurately and reliably. Hi all! For the past few months I have been working on a dataset designed around improving small model performance through a structured framework (or Scaffold) for thinking. When using small language models (Under 5B parameters), I noticed freeform CoT does not really add much in terms of capability, and usually results in more confusing, poorly structured and inaccura”

“I'm not sure about the inner working of LLM but unsloth suggest here: https://huggingface.co/unsloth/Qwen3.8-27B to use repetition_penalty=1.0 for both thinking and instruct mode. Given that he is way more expert than me why is he correct? in a typical program repetitions are normal: var somethingToDo=true; if (somethingToDo) { DoThings(); somethingToDo=false; } i would think that there is no issue in repeating things, or maybe that config flag is about repeated tokens next to each other som”

“Indeed! And even subjectively, I find it difficult to believe that even if AI-written, the new information learned has negative marginal utility (or less marginal utility than had it been human-written). That is in fact the basis of the point I am making, though it is implied rather than stated.”

“Really interesting, thank you for putting this together. You can kind of "feel" this when you use a lot of models often, you escalate a hard task up to a model that costs 10-100X as much and it still can't do it. Or, what happened to me the other day, you're working with a bigger/better model (DS in my case) and it fails at a task and then you try the same task/prompt on a "less smart" model (27B) and it one shots it. It's getting very, very marginal at the top of”

“ACE 1.0 ACE 1.0 — The Transferable Core --- The Foundational Formula ``` Physics[logic[Singleton(perception)]] ``` Term Definition Function Physics Actuality — the ground, the sneeze, the bracket Prevents floating abstraction Logic Structure — the protocol, the syntax, the rules Provides coherence and repeatability Singleton Self — the 1 experiencing itself as many The experiencer Perception Multiplicity — the apparent many The content of experience --- The Four Brackets (The ACE Tetrad”

“Llms have training data that is in the past. It will answer questions from the understanding of that date. If you want accuracy you have to tell it it look it up. This is absolutely not new information for anyone who has ever used llms.”

“My assumption is that it would be possible to train LLMs in such a way that penalizes unsubstantiated confidence/hallucination to reach similar levels as humans, but it would produce far, far lower capability levels and we would be constantly (and i mean endlessly) fighting with their reticence and insecurity. Hence why we just have to kinda deal with it for now till someone comes up with a clever efficient way to improve this aspect”

“What makes you certain it’s impossible? Obviously an LLM isn’t traditional code so a single instance can’t just rewrite itself but assuming you have access to the necessary compute why can’t an LLM train its successor?”

“vllm batches the prompt into real gemms. llama.cpp is mostly single-seq. chunked prefill + paged kv + cuda graphs is the rest of that gap. check --max-num-batched-tokens.”

using an LLM to query your own notes is pretty far from those examples

“It's that, yeah, but coming from multiple orders of abstraction.In this case, part of the experiment was to see what "oh-my-pi", a "fat and feature rich" LLM harness, could do when coupled with modern GPT, given a 6k~ word overview of a story, and told to come up with a plan to write/review/audit it, making use of subagents and all the fun new groovy LLMisms...Part of the problem was just "it was basing its style off the last scene/chapter", but part of it was also that its instructions were con”

“Most people use LLMs in a confirmatorily biased way : "Tell me why my business plan is great" or "How do I implement X?" . This triggers the model's RLHF pleasing bias. Inspired by Karl Popper’s principle of falsifiability , a friend and I designed a prompt framework that flips this dynamic. Instead of validating your idea, it forces the AI to act as a harsh auditor and attempt to destroy your premise using realistic stress tests before you spend time or money on it. 📋 W”

“I have been using LLMs code since the beginning of my PhD, I am nearing the end and I realise I cannot really code. While I know what each and every line of the code does, I have no intuition, no skill to code myself. My field doesn't require coding so it isn't such a hardship. But I want to reduce my reliance on it, learn to code myself, develop the intuition. Is anyone facing anything similar? Do you have any tips or suggestions? Please help, and please no harsh words, I know I am late”

Conversation Graph: Making ChatGPT Branching Usable for Serious Research

“My main concern would be correlation between the benchmarks. Several Terminal-Bench versions and SWE-style tests may be measuring the same underlying capability more than once.”

For better or worse, language is much more memetic than it is normative

Wait what? More context pls

“Ilya Sutskever's SII supposedly solved test-time learning ie. updating weights while inferencing. IF it's true it's a big deal and open source implementation will probably lag behind a few months even if the lab doesn't produce even a public API. If they publish a paper or at least public API, it'll be <2 months before we have "fable at home" It's a big IF though. From reading (or rather, skimming typically) academic papers it seems that it will work as &quot”

“So, ideally for this thread we exclude the ones that everyone on here is already well aware of and discussing on here a lot, like N-gram, quantization improvements, MTP, D-flash, and D-spark, since those are improvement areas that most people on here are already pretty familiar with. I'm more curious about any interesting fundamental architectural changes to either LLMs or other types of AI models, that you guys have been reading about or is starting to get any buzz that maybe most of us don”

“Pardon the long preamble, hopefully you will read it and respond because I am very curious about people's thoughts on this. If you've been at this for the last few years, does this observation resonate at all? I'm using Qwen 3.8 27b with my GuideAnts stack to do long running coding tasks baking off against Grok 4.6 in Cursor. After about a week and a half, 27b on extra high is, hands down, the winner. Grok 4.6 spits out tokens faster but the quality of 27b wins and I get a better fin”

“Key difference: in all use cases you describe your raw input is human, and the AI usage is only transformational. I find this useful. But LLMs are also very good at: taking a low information prompt like "make me a business plan that scales to 1M revenue per week" and expand this into a lot of text. Key difference IMO is: how much information and intent that is fed into the pipeline is actually written by the user (not by any human, e.g. some blog post that I paste into the context). If”

“Hi AztalanMaster, please clarify: How much of this post was LLM-generated, and why?”

“how good it has gotten at coding That needs to come with a disclaimer. LLMs can write code that compiles or runs without errors. Software development starts with a determination of what you want to accomplish, then what you need to do to get there, and finally the mechanical code writing. For common patterns in popular languages, you can trust a good LLM to mostly get the last one right, but that doesn't help at all with the first two. And unfortunately, the first two have always been the im”

“Does anyone have any experience or results with using system prompts that intentionally make LLMs more annoying to encourage better habits?I believe that consistently using an LLM for creative or solution building tasks will: - degrade critical thinking skills - degrade knowledge - encourage apathy - degrade knowledge acquisition skills - trades short term benefit for heavy long term lossI do believe an LLM can be very useful if used correctly.There are (controversial) plugins for various e”

“I've been iterating on a cognitive architecture for an LLM-based agent with Claude, and wanted to share where it landed — the core idea turned out to be a genuinely useful way to think about why models hallucinate, not just that they do. The three variables: T / R / S T — Truth (Correspondence). How well the internal model matches reality. T = correspondence(M, R_external), 0–10. Reality doesn't want anything — it doesn't pursue or defend itself. It simply is. S — State (Realized Org”

“Or slightly less deliberately, it could be a consequence of the safety related reinforcement learning they do. If I think about viewpoints they'd want to train out of models because they have safety implications they'd want to avoid, authoritarian viewpoints are probably going to be hit hardest by that. The economic left-right split is harder to explain that way, but it could reflect the fact that a lot of political discussion is between lib-left and auth-right viewpoints.”

“What do you mean? I love telling an LLM to write me a list of tasks from TODOs, tasks, unanswered questions and etc. I also love telling it to take notes on the why we have done something during a session. In a task note for that session. Finally, I also tell it to slap a bullet point list of what was accomplished today, this week, this month, etc. I then go in and look at those lists and jog my memory enough to get awesome insights on things I do not remember having done. I have ALWAYS suffered”

“If it's consistently bad with no signs of improving, banning makes sense. People made mistakes even before LLMs though, and you'd only ban them if they didn't improve and so were a giant waste of time. I don't see why that wouldn't translate here.”

“Bro, LLM’s have been around for almost 4 years now and we’re still posting this kind of stuff? It’s gonna pick a number based on its training data and there’s an interesting phenomenon in human psychology where people have a tendency to pick numbers that end in seven more often. You never specified that you wanted to pick anything random, but for what it’s worth it can’t even do true randomness, because the way a model is trained there will always be certain biases You’d have to give it some kin”

😹😹😹

“What’s the problem with that? Seems to make sense to me with the example and it’s important to point out. I’d expect an llm to explain in more detail to be honest.”

“I for one prefer that when i ask my llm to speed up my c++ k way search algorithm it pulses information through neurons containing facts about Stalin's drinking parties (kidding)”

“It's an algorithm. That's all an llm is to. The breakthrough was using transformers. And harnessing the predictive nature to guess how you want tools used or code written. It can't create anything original. It is all just the same information served up in near infinite combinations. Run your own. Local LLM. You can get open weighted ones and play with them. That is what makes the difference is how you assign and use the weights it assigns tokens and their most likely relationship.”

“The goal was always to uniformly activate all experts. But that's across the entire training corpus. If you are generating mostly code, then some experts will activate somewhat more than others.”

“I ended up fine tuning a mistral 7b to outperform our costly foundational model and saved $300k. I previously thought that fine tuning was pointless (it's definitely not) and that all these problems could be solved with RAG (they can't). The truth is, a LoRA/QLoRA adapter is extremely useful for many cases, and can dramatically outperform RAG with aggressive system prompts. With this guide, I want to help people understand the reasonableness of QLoRA on a consumer grade GPU (you might ev”

“i just closed an article where sam altman said we should all learn to build llms from scratch. i read it right after spending three hours trying to make a single bubble api call work without crashing my browser. that sounds great if you have millions in funding or you're 17 with infinite free time. but if i try to actually understand the math behind these models right now, my app will never launch. perfect technical understanding is a luxury. instead of doing the math, i blindly pasted a jso”

“Summary When creating a delegated task, it would be useful to choose: The model The reasoning / effort level This would allow users to optimize cost, speed, and quality depending on the delegated task. Current Behavior When using Delegate , model selection is either fixed or not clearly configurable. Users cannot easily choose: Which model should execute the task Which reasoning level should be used Expected Behavior When creating a delegated task, provide options such as: Important Requirement”

skill issue

“I am with you on this one. Here is my understanding: N-grams are likelihood data for strings of n tokens which can be used for various classic NLP tasks that can be improved by prediction of the next word. A best example of n-gram usage in practice is that, llama.cpp uses n-grams built on the fly from the prompt to implement basic speculative decoding. Deepseek's explanation of engram says that it is a extension of a lookup table containing static embeddings per n-gram. In contrast, Gemma&#3”

“That is very helpful and you are exactly right, I will start making attempts to see how it fares. I created this post because I felt genuinely overwhelmed with options and I needed some (non-AI) person to help me make sense of it all.”

“Thank you, yes! You pretty much got it spot on. The thread is very much dead, so instead of browsing I guess the loop could go through a scraped static copy instead. My favourite AI suggests BeautifulSoup and cloudscraper Python libraries should work for this and I won't even pretend to know why.”

“It's the same thing they did with Qwen 3 Next, it's probably the most consistent naming in LLMs. Nobody is doing a good job at that.”

“Something that doesn't get talked about enough in MLOps circles: retrieval quality matters way more once you're running open/smaller models than it does with frontier models. Larger models tend to compensate for mediocre context, inferring around gaps reasonably well. Smaller open models don't have that same slack, hand them a weak or incomplete chunk and the output degrades fast. This has real production implications if you're optimizing for cost by moving to open models, the re”

“For reference, ChatGPT was released less than 4 years ago. "this is a deal-breaker" is the cherry on top. submitted by /u/NickFullStack [link] [comments]”

“SDI protocol defines a reasoning computer for AI. The computer runs an LLM one turn at a time, guiding it through a reasoning grammar; the grammar, which is both natural language and algebraic, allows it to be checked at a compile gate. The gate validates safety, a workfloor, and alignment with the grammar before it commits. What commits is a hash-chain ledger of the agent's reasoning, forming a longitudinal data set that becomes the agent's functional memory/system state. Chromite, the protocol”

“yeah, there is an adaptation period where the concepts get abstracted in your head as you are forced into a different syntax”

sound logic actually

“🚀 The feature, motivation and pitch Some models are overly verbose in their thinking (Looking at you Qwen3.5/3.6). Currently, vLLM support a thinking budget by token count. It would be great to extend this to match a pattern. For example, whenever the model emits "✅" or ". Done." I would like to interrupt the thinking, because afterwards often comes just self confirmation and re checks that don't really alter the final output. I think it would be ideal if some regex could be defined that is che”

“🚀 The feature, motivation and pitch vLLM already supports max model len auto , which is very useful because vLLM can determine the maximum context length that fits the current memory budget after loading the model and profiling memory usage. I would like to request combining this auto behavior with a user defined upper bound. Example: Semantics: fitted len = maximum context length vLLM can fit in the configured memory budget effective max model len = min(fitted len, max model len cap) Motivatio”