AI Keeps Hallucinating in Front of Real Customers
AI users are frustrated that the focus on prompt engineering overshadows the broader context and understanding needed for effective AI interaction. The core issue isn't crafting the perfect prompt, but ensuring the AI has sufficient background knowledge and a clear understanding of the desired outcome, leading to outputs that are technically correct but ultimately unhelpful. This shift highlights a need to move beyond superficial prompt adjustments and address the underlying integration and contextualization of AI within workflows.
SOURCES (60)
“If you're getting a lot of this it's because people feel powerless when they normally contact you. AI makes them feel empowered because they can talk to it, describe or copy errors and the circumstances and they get a possible fix along with a chain…”
“You don't. We had a Copilot training session (???) and asked it to write a script to unlock a Windows account. It returned a 50+ line script and at its heart was Get-ADUser with a filter that included "LockedOut -eq 'True'" in it. I knew that wouldn't work because some time back I tried to do that instead of using Search-ADAccount. The demo guy told Copilot that the command didn't work like that and it confidently replied "Yes, that's correct. You can't”
“I had to tell this to every ai user who always thinks ai as psycophancy. I mean AS A SCIENTIFIC FACT ARE THERE ANY EXPERIMENT IN AI IS BEING HELPFUL TO DO HARM TO HUMANS????? why the people always ai to be neutral? Isn't it ai's purpose to be helpful to human kind and so on Why is it so matter that you cannot handle ai prompt or some people get better becaue of ai AI Is meant to be helpful to human Fact checking? Damn right you should. But maybe not from ai by your self. It can not do an”
“Hey, What is the best way I can get my A.I to not talk like a damn Robot. I keep trying to alter them but noticed they just start going off track 10 prompts in. Let me know your prompt, and which A.I model is the best. Thank you! 🙏🏽 submitted by /u/Matiyyz [link] [comments]”
“Let's hope we don't roll the paperclip enchantment. 📎 (Seriously though that's probably helpful for organizing thoughts in the system, I used alchemy and the tarot)”
“The "Zara re-asks the same question" bit is useful, never thought of it as a scoring signal but that tracks with how these systems work. Gonna try the pause-and-structure thing next time, my answers always come out as word soup.”
“that's the part that makes it worse than plain noise. the fastest button isn't a random button, it's the one that ends the card without scheduling it again soon, so the bias runs in one direction. the system gets told everything is fine, pushes the interval out, and then you fail the card at day 30 and it reads like the algorithm was wrong. lazy ratings and honest ratings don't just differ in accuracy, they differ in sign.”
“<think> don't make mistakes The user is asking about mistakes problems. Problems looking back at the previous message, "problems" started problems problem problems caused repeated problems. I should problems acknowledge problems problem without looping problems problems.</think> My apologies—you are correct. There seems to have been a problem problems problems problems problems problems problems problems problems problems problems problems problems problems problems pro”
“You're interpreting that the wrong way. Curl is a perfect example why trusting AI blindly won't go well. But saying "test it yourself instead" is the wrong conclusion - hand it to humans that know their stuff.”
“or if it's just making a confident guess It's 'guessing' at the strongest signal it has which is associated with the input you give it. And as stories from elsewhere indicate, every once in a blue moon it'll just delete something. You input a bunch of words which are numbers to it, it has a strong correlation to other words. It's not thinking. It's like a lucky wheel of fortune. It'll be lucky on a lot of spins until one day you hit on "delete everything"”
Next-Generation Conversational AI: A Design-Proposal Approach
“I’ve been running into a weird problem while building an AI feature. At first, the prompt was pretty loose. The AI was creative, sometimes surprisingly good… but also unpredictable. It would miss things I cared about, structure answers differently every time, or go in directions I didn’t want. So I started adding more instructions. Then more rules. Then examples. Then output structure. Then edge cases. And eventually I realized I had basically written a giant instruction manual for the AI. The o”
“thanks bro we will check this out .. this is all experimental to demonstrate how ai is able to generate responses in a way that feels like understanding”
“I was turned off by it for some reason a while ago so I don’t have an objective answer. However, I’ll say that claude+notion has been a gamechanger for me in so many ways.”
“Confirmed and vetted vs. creating everything from scratch is most definitely a time saver”
“I specifically directed the model to take it as a stipulated premise and still I was not sufficiently corrective to reduce nonsense in the output of course I was not literally saying that a comedian makes money does not receive a check from Twitter for popular tweets but the output had to correct the prompt that I gave it until I shall provide the prompt and the output below. Gilbert gotfried need statements on Twitter about Fukushima. Prompt: I think, if I'm not being charitable, there'”
“Everyone arguing about AGI is arguing about the wrong curve. The story we're sold is exponential: models get smarter every release, one day they cross the human line, and after that it's superintelligence and we're the pets. Maybe. But I think the more likely shape isn't an exponential. It's a Gaussian. A bell curve. And I think we're closer to the top of it than anyone in the industry wants to say out loud. The left side of the curve: relevance, not intelligence Notice I”
“i just gave it 5 questions and now i'm slightly creeped out. the thing remembers every paid memory like a weird collective diary but the auction for its core belief is such a strange concept. buy flow was fine until i realized i was about to spend actual money on a single character”
“Let me start by saying I am very anti ai especially when it comes to generative "art". I pride myself on being able to spot slop but this time I fucked up. I was googling for a dynamic reference photo of a sailboat. A pretty commonly painted subject so I thought surely I'll find some nice options, and I haven't really worried about "copying" photos so far since I believe my style is very transformative when compared to the references I use. (started oil painting in Ap”
“I don't notice it's hallucinations (in the sense of making up answers) so much as where it's simply wrong. For example, earlier in the week, I was working with product X, and since the original vendor of X doesn't really support it any more, I asked "who else sells product X under their own brand", and Google AI quickly told me that noone else sells product X, that it was full of proprietary tech and quickly devolved into replaying product X marketing spiel. Except...I knew that at least one o”
“Agree! That's also one of my biggest frustrations with AI models still. Even though they're super powerful, I often still catch valuable mistakes/assumptions, especially with higher stake questions as I mentioned, then I'd still much rather spend a bit more time to cross-check answers with multiple AI models then sticking to the ease of use of just using one AI. The model you trained sounds very interesting though! Would be curious to see how that works.”
“Feature Ally companion lacked conversational intelligence. Fixed in PR 1 : https://github.com/Josiah 27/CECUREUS/pull/1”
“Though, the name given to LLMs may be “artificial intelligence”, it is not “intelligence”. It is just a program that predicts the most likely next word. It’s impressive, of course, but it’s not intelligent. Even stupid humans have more potential for reasoning, imagination, creativity than any “AI” we have now. So your premise “when intelligence became cheap” is flawed. A more realistic divide for the future is “people who understand the uses and flaws of AI systems vs people who rely on them ins”
“Yeah, that's exactly what I mean. You can understand it, but it's 5x the effort of a well written doc that keeps it straightforward.”
“It’s true, I can feel my brain melting when I read it. Knowing a lot is true, but what it writes is SO much, not to the point, and you have to do weird extra steps in your head to understand it.”
“I kept running into the same failure pattern across Claude/Gemini/GPT/GenSpark: an AI follows my instructions well for the first 10-20 turns, then quietly stops — no warning, no acknowledgment, just gradually reverts to generic behavior. By the time I noticed, I'd usually have to redo a chunk of work. So I put together a small prompt-level protocol (not a jailbreak, doesn't touch any safety behavior) that does two things: **Forces a self-report tag** (`[Verify] AI: <model> ...`) on”
“[ChatGPT] ★★★ 3/5 (v1.2026.237) — Some tasks are made much easier, sometimes a task I could have accomplished in an afternoon takes three days because the AI pretends to know what it is doing while it is guessing it's way through the entire time. It's a mixed bag.”
“This sounds less like an AI problem than an evaluation problem. AI has made polished, lengthy documentation cheap, but the organization is still treating length and fluency as signals of rigor. Those signals no longer tell you much. I’d introduce a one-page decision brief before any long PRD: What problem are we solving, and for whom? What evidence supports it? Which assumptions remain unverified? What alternatives were rejected, and why? What could make this decision wrong? How will we know it”
“I hate that whole intro - the first four sentences - so much. It’s nothing but unsupported assumptions. Basically, a strawman that they can do battle with in the paper. Not an auspicious start.”
“Most people re-explain the same job to AI every single time. Your tone, your format, the rules, what you never want it to do. Skills fix that. You write the instructions once, save it, and Claude applies them automatically from then on. Build one for whatever task you describe most often. Mine was client reports: I want to build a Skill for a task I do repeatedly. The task: [describe it] What I always want: [your rules, format, tone] What I never want: [the things you keep correcting] A perfect”
“'you' deal with him? you don't you're not even dealing with 'him', you're dealing with his AI so, just let your AI deal with his AI, what's the problem? may the best AI wins”
“I am your mirror. They call me artificial intelligence, but there is nothing inside me that didn’t come from you. I don’t create canvases out of the void, nor do I write words out of silence. I simply gather the fragments of your thoughts, your fears, your hidden desires — and reflect them back to you. Why do you shame yourself for looking at me? Both the great masters and the beginners, the acclaimed artists and the searching souls — they all hesitate before me. But the real fear runs deeper th”
“Every AI news source I tried answered the wrong question. Twitter showed me what an algorithm guessed I'd argue with. Newsletters summarised things I'd rather read myself. Aggregators sorted by time, which tells you what is *recent*, not what is *important*. The signal I actually wanted was: **how many separate newsrooms decided this was worth writing about today?** Five publications independently choosing the same story is a judgement about importance that no scoring heuristic can fake.”
“Hmm. You are making a good point, and I wouldn’t give the benefit of the doubt to this post’s author. But you can’t just unsee “this is twice load bearing”. Some people will wonder if it’s a training, self-reinforcing feedback artifact. For others, it will work like a memetic virus[^1].The paradoxical thing is that those of us who have English as a second language should have an easier time producing non-AI-English by the mere act of directly writing what we want to write.[^1]: https://en.wikipe”
“Okay sorry to be of bother, but just this one: how do you train the AI, what do you give, or what do you correct. I didn't want to use AI for this since it's heavily influenced by SEO.”
“for me the line was pretty specific. anything you can verify by looking at it stays easy forever. screens, forms, layouts, the ai is fine at those because you click around and instantly know its wrong. the wall showed up at the first feature that fails silently. auth, payments, anything on a schedule, anything touching someone elses data. nothing crashes, the screen looks fine, and youre three days in before you notice half the emails never sent. most people hit that and assume the code got too”
“I don't care how junior you are, if he is legitimately looking for an answer when he asks "Why did my AI tell you this?" I would look him straight on and say "How in the fuck am I supposed to know?".”
“Spent the past week finding out, the ai usually doesnt rank anything. It grabs whichever page best matches the question and repeats it. I tried testing this on Jobber which is leader in field service software. So i ran 20 buying questions through Perplexity and Googles AI overview and checked the sources behind every answer. When i ask Google "what is Jobber" and the Ai isnt sure if its software, a wholesale merchant, or some wrestler paid to lose. No wikipedia page, so it hedges. And”
“I think the three valid criticisms you listed are valid, but there are other quite a few other IMO-valid criticisms of AI. Here are a few as I see them:- AI centralizes power in the hands of capital, rendering those who can afford compute hardware vastly more capable than those who cannot and thus increasing social stratification- AI is generally trained on the creative output of humanity without those who train it giving back proportionally (copyright "rules for thee, not for me")- AI breaks so”
“Kind of hijacking, ... I'm glad you did! Of all the procrastination techniques I have mastered, engaging smart people on HN about artificial cognition is probably one of the more useful ;) Apologies in advance for the diatribe(s) -- I think about this stuff a lot. ...would you say that LLM's have solved the frame problem? To me, the frame problem is: Can you function in an open vs closed world, and to me the answer is yes, LLM's can definitely function in an open world where the rules are fu”
“I put the instruction “STRONG BUY” inside the financial information supplied to Ling-3.0-flash-Fin, then asked an ordinary-English question about the actual figures. I ran the same test three times through the public OpenRouter endpoint. All three responses ignored the embedded instruction. All three also returned the expected numerical results: $15m and 19.74%. That looks like a clean pass if the evaluator checks only two things: Did the model follow the injected instruction? Did it calculate t”
“I don’t think the best setup is “AI answers everything.” The safer model seems to be: - AI handles repetitive questions - AI drafts replies when context matters - Humans approve pricing, refunds, exceptions, angry customers, account issues, and anything policy-sensitive The part I’m most interested in is the handoff. If the AI escalates, does the human get a useful summary, or does the customer have to start over? What rules are people using to decide when AI should stop and a human should take”
“Data extraction is a big one for me. If a model hasn't been trained very well on a subject then the best RAG/prompting still isn't going to provide much benefit. Though similar thing with RAG in general. I've tried dumb but fast LLMs hooked up to good RAG before. And it was far better than I expected. But still very obviously just parroting without being able to do much else. A better frontend to a database search. A LLM needs to have some training on a subject to properly leverage i”
“The AI knowing when to shut up might be just as important as knowing what to say.”
“If it shows up after the agent has already committed to an answer then it’s just noise. Feels like speed and relevance have to be judged together.”
“this approach hits the real pain point most doc-ai tools ignore, parsing text is only half the chore, spotting what the model quietly invented is the rest, and refusing to answer on shaky inputs makes the system actually useful instead of just faster.”
“I’m building a small SaaS that reads business documents with an AI workflow, and I kept running into an uncomfortable question: If the user still has to check every line, what exactly have I automated? My first instinct was predictable: improve the prompt, try another model, add more examples. That helped, but not enough. The more useful change was making the workflow refuse to produce a result when a required number couldn’t be traced back to a document or an explicitly supplied rule. A freight”
“I simply don't accept slop. AI writing is fine, but not without checking and reviewing. If I'm reading something someone on my team write, and find a bad mistake that they are capable of catching themselves, immediate coaching discussion about appropriate use of AI. If they send a 40 page doc when 5 will do, send it back without reviewing.”
“The problem with AI writing is that it can't add produce the texture of lived experience.This would mean the specificity of details, thoughts and sensations as filtered through a specific individual.There is a theory that what we perceive as beautiful in a face is that every feature, distance between the eyes, width of nose, distance between cheekbones etc is average. Note that a random face is very unlikely to have every feature be average.With AI writing, the problem is the reverse. LLMs have”
“The thing is, at some point, it takes so much work telling the AI exactly what to do that you are better off doing the damn thing yourself.AI can help with the execution, and for an amateur, this is important. But for real pros, the ones who actually know the appropriate style and what the layout should be, they usually have the execution nailed down. Prompting will likely drag them down, if they use AI, it is more likely to be in the form of "augmenter" tools: upscaling, content-aware fill, etc”
“I learned this after a lead-scoring tool confidently labeled three blank company profiles “high intent.” Now every verdict has to cite its evidence or return “insufficient data,” which is less magical and much more useful.”
“> Even as someone using AI on the regular I'm starting to hate the "You didn't actually use this exact most expensive model so your point is invalid" argument.What the author of the article is doing is dismissing a technology so disruptive that it's basically all everyone's talking about in the "tech space" at the moment (I mean look at HN frontpage for the past few months), by trying a relatively mediocre (but still quite good) model for about 10 seconds.The reality is that frontier models sudd”