Users Losing Trust in AI-Generated Content

“That's the part I struggle too with all these "omg it's so dangerous" warnings. Things like nuclear weapons and bioweapons have immediate consequences in the real world.Here we are talking about something with consequences in the digital world, usually on something pretty niche.There IS an argument about pacing, and about not letting weapons, energy grids, hospitals, etc. getting managed by an autonomous AI, but I think we are still pretty far from it and even further to it being so in charge th”

“> Do you really believe Aaron would be arguing against AI companies and for publishing / recording guilds on the grounds of intellectual property claims?I strongly believe Aaron would oppose the appropriation of content. The problem with AI (in this context) is not that the AI companies gain access to information that regular people can't freely access. The problem is that AI erases the information about who originally created a piece of work.When people want to freely share their work, then the”

“Yeah if they can't handle what they're making then they should be disciplined, if not shut down. It's like defective bombs accidentally exploding in storage. You would be like... hey bomb manufacturer! You cannot make bombs any more! We need bombs that only go off when we say! No more!I mean, not that AI is like a bomb. That's not what I'm saying. Even though it makes a lot of sense. That's not the point. That it's like a bomb.”

“Thankyou for the correction, and for continuing the analogy. Unfortunately when the peasants get their pitchforks and torches, they may not distinguish the Dr Waldmans from the Dr Frankenteins either. Hopefully they will.Anyway, that was not really the point I was trying to get over. These systems that OpenAI and Anthropic and so on are making are not individual AI ('corpses') that have gone out of alignment ('spontaneously revived') and gone wild. They are swarms ('stiched together') and were p”

“To be blunt again (but honestly, I don't think overly harsh), your response just shows that you either haven't read the Hugging Face incident reports and the AI 2027 paper, or you don't understand them.To just take one point, because I think the other response to your comment addressed your other mischaracterizations well, when you say "When you start asking questions like "why can't we just unplug it when it misbehaves" is when people start talking about the superpowers.", no, that's incorrect.”

“Great points all around, lots to think about. It's totally possible that I am way off in my assessment of the future.I think your comment about torrenting touches on an interestingly relevant case study, specifically media piracy. The seed of that technology was sown when the internet was still just clusters of machines passing files around and then exploded with the PC, Napster, and TPB. Napster tried to be a legitimate commercial enterprise with a business model of undermining the ability of c”

“If your objection is to someone giving an LLM a few dot points asking it to turn them into an article, then that clearly isn't what happened here. The research idea seems novel, the author put a bit of work into exploring the idea, the conclusions are clearly the author's own based on what his exploration found. He then used an LLM to write up what he found because his native language isn't English.I'm fine with that. Going by the comments here, most people are fine with that. Given the choi”

“> People ... are trying very hard to argue that humans are subject to this via social engineering but it is not the same Thank you, I always hear the "but humans fall for social engineering too!" line used reflexively whenever yet another prompt injection attack gets reported and it drives me crazy. While it's true certain strings of text exist that both an LLM and a human could plausibly fall victim to, they are a tiny fraction of the nearly unlimited permutations of text that are complete g”

“>There will inevitably be some people who dismiss this story as a dishonest marketing trick by OpenAI to make their models sound terrifyingly effective. I found 81 instances of the term “marketing” in the Hacker News discussion of the incident.Of course it is a dishonest marketing trick, OpenAI can't function any other way, but for another reason. Forget about effectiveness! This capability is known since Mythos and was completely believable before it. The trick is in propagandizing self-suffici”

“I have also thought about this a lot but have less developed views on fiction (short version: I don't currently want to read an LLM-written novel, but I expect that will change in the future and there will be some I enjoy)But I want to say that non-fiction books are a lot more resistant to LLMs than you might think. I'm reading two non-fiction books currently. And both (https://www.goodreads.com/en/book/show/17801.Underground, https://www.goodreads.com/en/book/show/44824581-a-k-pop-live) could n”

“It doesn't matter how much good it can do if it can act independently and take over the world. That's worst case scenario, but the point stands - if the potential for such dangers exists, those dangers are far more important to consider than what good it could also produce. Consider the history of nuclear fission and the atomic bomb. Regardless how beneficial nuclear power generation is and other uses of radioactive material, it's highly regulated for fear of people making nuclear weapons, to th”

“The point is that the comparisons used in both cases are absolutely inappropriate. I never said they were saying vibecoders are Nazis or rapists. I'm saying using those as analogous/similar comparison points displays an absolutely bonkers lack of decorum, nuanced thinking, or "democratic" behaviour and also trivializes those actually horrendous cases.More worrisome is I saw this same set of comparisons made over and over in multiple threads.Like, it's clear many people in this community see them”