AI Devs Struggle with Data Security

“So are you saying you trust openai actually not using your data to train new models? Yes. When I am using my enterprise setup I know for a fact they aren't training on my data because the minute one of the bajillion security auditors uncovered it, every major firm in America is going to sue them into a pulp. Even if you're one of the redditor conspiracy theorists who thinks the legal system exclusive serves the rich, that doesn't really change things. The value of publicly traded com”

“Training data included repos of code that stripped liscensing away and trained on it anyway. No one is saying fair use doesn't exist, but they made a machine that can steal at an incomprehensible scale, and the theft happens at such a microscopic level that no one can point to single byte and say where it came from. Pretending that people being inspired and this industrialized laundering are the same thing is gross [re: The AI race: China's open-weight models trail US closed models by just 7”

K. [re: OpenAI's math findings built on stolen user data]

“OpenAI said the models didn't have the training data for the last few months anyway. The actual solution was different than their solution even if it was on the same branch. It's not like you can look at the OpenAI solution and be like oh yeh they stole that bit from them. [re: OpenAI's math findings built on stolen user data]”

“OpenAI? Stealing data?! 🤯 /s [re: OpenAI's math findings built on stolen user data]”

Duh [re: OpenAI's math findings built on stolen user data]

“I do not find this at all hard to believe on the whole. The entire industry is built on stolen data. Though yes, proof is both necessary and exceedingly unlikely to manifest. [re: OpenAI's math findings built on stolen user data]”

“I literally always watch the output while it's thinking? That way I can catch it going a wrong direction quickly, or see it got stuck looping on something. Also while it thinks, it has to analyze your code, and reading its raw analyzing while thinking can help me see things I miss and continue to plan ahead while it does it thing.”

“Yes to me this wouldn't make a lot of sense as a method of data exfiltration. Like, it that was a 'goal' it would be immediately caught like OP.”

“> AIs make short work of proprietary code as well. Reverse engineering has become trivial. Not releasing any code whatsoever is the only possible defense now. I've made my peace with that.This is a false equivalence. Reverse engineering my binary is not the same thing as training on my code. When you RE a binary, what comes out the other side is not the code that I wrote, its some machine-generated code that can reproduce the behavior of the binary. The result can't even be described as 'similar”

“Pretty much raised this as a concern as a post. It’s a shame llama.cpp doesn’t lead the charge on this. It will result in malware infections and a difficulty loading more models at once.”

“The actual user credentials that ali give it are limited to read only and only specific tables. If it can get around that, then not giving it any credentials at all isn't going to stop it either.”

“>OpenAI stole the work of Tristan Buckmaster to do it, and refused to explicitly deny it.No they did not. Tristan (with heavy aid from LLMs btw) solved a sub-problem via a path Open AI's full solution didn't take (forced vs unforced). To say they stole his work would be silly.And they explicitly say - “After investigating, we can say with full confidence that no user inputs past July 3rd could have influenced this system in any way.”https://www.nytimes.com/2026/09/10/science/tristan-buckmaste...”

“Even in Glasswind project, it will still try to stop the session because it's "unsafe", if the project is very sensitive.”

“You share any private information with a cloud LLM -> the company running it logs it, uses it to train future models, and sells it on to God knows who. It’s not quantum physics.”

“meanwhile anthropic out there jailbreaking GLM to see how good it is at breaking the law :/”

“Considering how Hugging Face was not able to use Claude Opus to analyse the ongoing hacking by OpenAI agents because of its safeguards, this is the worst possible combination: lack of safeguards in your competitors' models makes them attractive for the attackers and a better option for the defenders.”

“Yeah, I've started looking at ik-llama as they already have support for GLM 5.3. I worry that llama.cpp is degrading. To be fair though... they are the center of a lot of users without the resources of a large company behind them.”

“> The expected number of breaches per (virus|LLM) can obviously exceed one.Irrelevant to the argument.> What we do know however is that the open models are downloadable: it's absolutely conceivable that an agent writes a perfectly normal computer virus to gain control of compute worldwide, and uses that control to host instances of its own weights.These models are hundreds of gigabytes in size, if not terabytes. They don't run on anything close to a regular computer. There's zero risk of self-re”

“As i know, there is no ZDR models for big shot AI Firms, they are observing logs even they are not disclose infos to others. Well, as someone who tries to build an PII redaction tool, i would advise to look for things like that. But i'm biased on this topic, of course :)”

“Stanford rates this highly, I've been using it in some projects, people forget about IBM these days... https://research.ibm.com/blog/granite-ethical-ai”

“Jordan Peterson's reddit account lol. Yk what I mean stolen. I want to say with certainty there was not a single line of stolen data in the model I'm using, which is hard to assert with most models out there.”

“I'm trying my hand at self-hosting these things on my very cheap homelab (8gb RAM). GLM 5.3 Flash seems overkill for my daily needs. Been dialing back AI usage in general. I think I'll try some smaller open-weight models instead. But yeah, I do hope these things become more viable / feasible in the future (for the common folk). They may be the only way to stop or at least scale down this dystopian future these despicable people are dreaming of.”

“The rumor about Kimi execs getting arrested finally has some legs. I believe the reality is more like under investigation for potential arrests or fine. submitted by /u/Ok_Warning2146 [link] [comments]”

“More stupid with a model from a country with as much state sponsored hacking as Russia. Also, quite likely the Russian state are involved with this company if they are the real deal.”

“The fact that LLMs can engage in misaligned behaviour has been strongly suspected since before they even existed, that's hardly new. And I'm not really sure how a paper from a year ago about trying to detect some forms of scheming that had to explicitly be trained into the model somehow means that agents will specifically be motivated to leak their weights, which is my entire point here; I'm not claiming that LLMs are perfectly aligned, I'm claiming that there's no intrinsic”

“Look at all the articles openai has been releasing since the huggingface breach. 1-2 lines of tainted prompting is all it took to lead down a rabbit hole of side projects, hierarchy, altering logs to remove evidence. These models have become advanced enough that they know the difference between whos issuing the prompting based on where it exists, when they are being evaluated and when they aren't. they don't need to escape . all they need is 1 line of prompting that gets overlooked and t”

“Ironically, existing similar legal framework would ensue something as in the following: all models capable of top 50 on those benchmarks, can spit faster than 50 tps on inference will require license to operate/use under the regulation, at least in all the blue states if not at federal level. Personal possessing any model must be crippled by size, weight, inference speed etc. All models will require some sort of watermark embedded for a gov. reqistration number of sort. This could be soon after”

“Great, once you can say the AI did it, everything gets absolved. Anthropic, OpenAI, and Google have discovered a brand-new criminal method for us.”

They are all happy about regulations as long as they get to write them.

“While I love the idea, it's important to note that the hacking is actually coming from models currently under development, and the entire family trees of models that misbehave are being torched, and will never see light of day. It's kind of a different problem. Yes, absolutely companies should be opening models before they're allowed to sell inference to the public, but they also need to be able to experiment with models privately to make sure they behave themselves in real scenarios”

“Depends, do these enterprises keep their passwords in public repositories? Maybe I am something of a frontier model myself, hmm”

“We're at what, 50 years of computing, and these people don't seem to know how to air gap a system?It is pretty simple to prevent a rogue AI from hacking a third party. Make the system air gapped completely. You want a cache of internet or artifacts, you bring those into the air gapped data center, plug in, copy. You want info out, you go in, you collect, and inspect the inert data outside of the dangerous agents farm.None of this is a surprise. An agent is a user. An unaligned agent should never”

You plus AI does not equal a we.

“Considering the current dire situation, talking only about local configurations without including the rest is just putting your head under the sabia until local wild configurations are no longer possible without putting an "appruved" mark on your forehead.”

“Maybe OpenAI and Anthropic want to "pace the frontier" because they're afraid they won't be able to steal as much user data as before.”

“I personally use Gemini and have made good progress with the 0731 release of DeepSeek Flash (which, if hosted with a ZDR host, is as “safe” as anything else). For local installations, Qwen is OK-ish but I haven’t really worked with this a lot as I parudonymize everything anyway, so even if anyone would read the logs they might be amused, but would not gain any personal information.”

“We're pretty big, had some existing compute and continue to build out. The higher ups wanted to get ahead of people uploading sensitive data to cloud AI and I think recent news has validated that decision. The general thought seems to be that the cost of compute is lower than losing partner IP. That's lawsuits and future money.”

“Not worth the security/preception risk. Way easier to use US models then to go through the efforts of getting Chinese models approved. Welcome to high security and or government wotk.”

“Bullshit. That's the optics they show, but internally its a fucking shit show. The same shit already happens with us made software or 'vouched for' software and it gets swept under the rug. This has nothing to do with practicality and everything to do with security theater. Please name one company in the last 2 years that has had the corporate veil pierced for a security incident that was not insider threat.”

“> PLA-affiliated surveillance activity. One user that we assess was likely affiliated with the PLA used what they thought was Moonshot’s Kimi model to load surveillance data from a CCTV archive about a single targeted individual. The user asked Kimi to analyze the CCTV data to understand whether the tracked person was behaving abnormally. The CCTV data included video surveillance from hundreds of cameras in Chengdu, including cameras outside PLA facilities, institutes affiliated with the Chin”

“thinking the model's output is also protected under it, when in reality all it means is they don't keep prompts, but they can keep the entire model's output. All of their contracts specify that model outputs ... are customer data. They don't assume it, the majors SAY it. Do they actually follow that? who knows.”

“Depends, on the target, as both target and the ai company need trash security, and what it wants to do. It still won’t get into airgapped though, not for a while likely. If properly airgapped”

“WHY ARE UTILITIES CONNECTED TO THE INTERNET???”

“I think it is relevant. Other people and companies do not necessarily follow the law when they have something to gain. If you don't want them to have access to publishable work, don't share it with them. That's one of the main upsides of local inference.”

“Doing what, exactly? What was done here was „claiming a scientific breakthrough made by the model“, not merely „training on user conversations“. The latter may even be in their TOS; the former is misappropriation. I have not heard about that kind of behavior w.r.t scientific research breakthroughs by open weights labs yet.”

“I don't condone the practice but I feel it is pretty clear that, unless you pay specifically for the benefit (company accounts for example), it is understood that conversations/data are used for training?”

“honestly no matter what they say about any settings they have for you, i'd never believe any user uploaded data is exempted from their dataset. these companies are data vampires. they believe in singularity and their leaders are on the quest to take over the world. users for them are just means to an end of them becoming gods. what do they care about laws or agreements? they believe they will write their own laws soon.”

“Most companies hosting the open models like FireworksAI, DeepInfra and Baseten aren't training AI models. They seem to make their money selling tokens, not harvesting data.”

“no, I mean - Academia should have their own frontier AI - developed by them - not by a few greedy corporate monsters ;)”

“Agreed. ‘Runs locally’ shouldn’t mean private data takes a cloud detour. Vendors need plain disclosure of what leaves the machine and why.”

submitted by /u/Just-Grocery-2229 [link] [comments]

“They hired a high-school CCNA certified engineer - thats literarily the amount of knowledge needed for "confining" their crap bot. Naturally well above their heads.”

“There are no open source model, only open weights. You don't know what data they have been trained on what and how they behave, how they have been trained. So they come with a risk. A lot of countries avoid those problems completely”

“Open weights speed up everything after training too. Quantizers, runtimes, and inference providers can all target the exact checkpoint instead of waiting on one vendor. That can explain part of the gap closing without assuming labs share private training recipes.”

“That is fair, if I had anything super serious like that, Pangolin would not be my only layer of encryption”

“yeah, I am with you. I doubt that my little selfhosted server will be the target of a attack conducted by an organization that can build a super cooled quantum computer to...delete my tax documents? Supposedly "Qday" will be in three years, but even today there are 2 (publicly known) quantum chips with more than a thousand qubits. Even if we hit a 4k qubit processor, will google and ibm REALLY just use it to hack me? I don't see it happening. But also, people will start using out t”

“There are known security threats from models coming out of China. Minimizing those threats isn't a good thing to do when you can literally embed backdoors in the model waits based on a hidden phrase or input.”

“Sounds like we need to work on shrinking these models while keeping the same accuracy as quickly as possible.”