Local LLM Users Face Memory Capacity Shortages
12 posts across 3 communities in the last 30 days; 83% from r/LocalLLaMA.
“it is. I don't know how you can't see it. Because it's still pushing customers to their API to use the larger models they produce that most people can't run. My god you're dead set against the idea that a company that's not American can't be as shitty as our businesses can you…”
“This chip is just a refresh with more memory, they can't compete on performance, but the frenzy may help then selling more units. The next generation looks more promising, up to 512bit bus, rdna5 (udna)”
“Lord, thats terrible! My internet is fast, but I only have 12gb storage, but I cant run anything bigger than 30B MoE at any speed, so each model isnt too big, I just ahve a lot”
Users are concerned about reported shortages in memory capacity for Local Large Language Models (LLMs), with some predicting issues until 2030 and others noting high prices.
SOURCES (60)
“If you're like me and saw the price increase for the 128gb DGX Spark go from 4.7k -> 7k while a new version with 64GB launch for 5k, you'll have been very disappointed and every right to be so, it's just plain sad for localAI.…”
“No intention of running local llms as I just added in ps Been offered a used air m4 with 24GB, I think it might be such a sweat spot..”
“as a bro who's lucky to have a 5090, I think I'll just up my system ram to 128gb from 32gb for $1k for offload use. does anyone know how much worse it would be compared to these?”
“This whole thing is fucking up the entire sector, like even IMUs, MCUs and other ICs are having massive delays, PCB pricing has also been shooting up in price. There isn't enough capacity and fabs are prioritizing the big spenders”
“My work had 100 laptops on order, we had to reduce it to 60, and push out our upgrade schedule by a year for everyone else. I would imagine a lot of companies are doing a lot worse.”
“its already been happening unfortunately. remember when the Pro Max phones were $1000?”
“So is your point that people will download a 100GB AI model and have absolutely no intention to use it? Has that been your point all along? No. Of course people intend to use it. That was never the question. The NJ law doesn't punish intent to use a file. It punishes intent to 3D print a firearm, receiver, magazine, or firearm component without a license . And that act was already a crime in New Jersey before the files law existed: N.J.S.A. 2C:39-9(l)(1). The files law just moved the line fr”
who is watching these idiots install olmx for 45 minutes.
“As an old guy, 2028 will be here before we know it. When I think “this won’t last that long” I am thinking with 5-10 years.”
“i mean sure but no, it makes sense if a flagship folding phone was 700-1000 dollars, but they arent they are upwards of 2k”
“See previous comment. 405b sense lol. I'm sure there's bigger. But ever since Moe. Dense was not rational. With dense... Most expensive hardware is too slow with 400, 3t parameters... Kimi k3 shows you need 2.8t to complete with proprietary. Rumor is that Claude is 5t to 10t. that just won't run even with most expensive hardware dense. Needs MOE.”
“Yeah, I'd rather have something I own and build than give anything up for convenience”
“it is. I don't know how you can't see it. Because it's still pushing customers to their API to use the larger models they produce that most people can't run. My god you're dead set against the idea that a company that's not American can't be as shitty as our businesses can you. It's literally all a fucking ploy to make money by the higher ups.”
“That said, if the new mac is what its hyped to be, i will prob just toss all my old stuff. I am tired of the massive rigs with 1000 cables, it would be nice to a a 20w box just work.”
“people don't understand the small models are just a proof of concept. "here's what our model can do at this size. Play with it. But wait if you want the best version of our model, you have to pay the API price to get it. Oh you don't have the money? Stop being poor"”
“I have to assume that at that price point, they're going to realistically target folks with a 19 in their birthyear. We even talked about it in teams and it's just a whole lot of money and in general compromise for a use case that we aren't really confident can be justified against all that. But I'm not FAANG and I don't have the market research and team of experts telling me which way the wind is blowing in every corner of the world, so....we'll see what happens. Definit”
“True ! but a few years ago we also had 400B dense models, the 500B model u talking about has 8/16b activated so it's pricing per 1M is still in the range of "flash price"”
“A few years ago a 100GB was considered a very large language model. What do we call under 100GB models now? Tiny models? haha submitted by /u/Terminator857 [link] [comments]”
“They are for multitaskers. People who need Netflix or YouTube to be running in the background while they check their stock portfolio while shopping on temu while betting on kalshi while posting to twitter while logged into onlyfans while on the toilet.”
“I've been thinking about this. Remember when Steve Jobs presented the iPhone for the first time? He jokingly positioned that they were selling three new products - a widescreen iPod with touch controls, a revolutionary mobile phone and a breakthrough internet communicator. If he were around to introduce this phone in a similar way (but we know he wouldn't reuse the same trick), he would probably present the dual screen fold-out feature as "an expansive, cinematic, pocket-sized video”
“This chip is just a refresh with more memory, they can't compete on performance, but the frenzy may help then selling more units. The next generation looks more promising, up to 512bit bus, rdna5 (udna)”
“https://preview.redd.it/5xxyekt9tfnh1.png?width=1596&format=png&auto=webp&s=d6ee8f1d5d8ca71fb28534e4e3c6f144ee29a03a submitted by /u/Only_Voice569 [link] [comments]”
“curious if those 5090s would handle running a couple companion models at once without issues”
It’s not my opinion! I’m just referencing a tweet form Boris Cherny
“The fact that you might be able to scrounge up one or two doesn't mean they are "available" in the market sense of this discussion. Unless you think the world revolves around you specifically, in which case yeah you can get one if you act really fast.”
Wait with the download, there will be newer better models
“Eventually, we're all dead. But there's a difference between having to replace the whole system or buy an entire 2nd system because you need just a bit more resources and between being able to swap CPU, add memory, or add another GPU. This has been the way the computer industry has been operating since the invention of the thing. It's only after the Mac studio that suddenly people forgot upgradability existed.”
“Yep, if I didn’t already have a M4 max 128GB, with hindsight that’s where I would be going to.”
“I bought a mini barbones with an h255 (got an 780M) and put 96GB in when it was at least more affordable. It's not great, but at least I can play with the models up to 120B. Even did v4 flash at 1 Quant. Got 2.5 TPS for very short prompts, but pp was atrocious. Qwen 27B at 7-9 TPS and 35B3A at 30 tps for very short prompts. I think 35B3A would be good on your hardware.”
“I'd settle for being able to buy 64GB DDR5 sticks at 2025 prices. They looked crazy expensive in 2025 but then 2026 happened.”
“any local model i could run up until now were not smart enough to keep around after the newer/better models are out, we're simply not at that stage yet for 8g+16g setups, i'll probably be satisfied with a 20B moe model as smart as qwen 3.8 27B at q4 but until then no model is worth keeping imo”
“Lord, thats terrible! My internet is fast, but I only have 12gb storage, but I cant run anything bigger than 30B MoE at any speed, so each model isnt too big, I just ahve a lot”
“You should daydream about being too pour to have hardware capable of running your Q6”
“That's why we're not waiting for it, we're here lamenting and complaining about it like little beggar children until someone takes pity on us. 😝🤣”
“sounds good. next week i will receive the feedback from my beta testers and i will need a week or theee to fix or implement their sugestions. we can talk after that if you wnat”
Woah. This is the first time I’ve heard this take. This is awesome. Legit can you explain further? You think that these other companies are buying up memory with a legit factor…
“When M5 Ultras come out lol. No one's letting Apple get the memory needed to threaten the cloud AI industry like that again. They drowned it in the bathtub by way of market forces, and they are not about to let it come back. New Apple's a total bitch. They'll roll over, take it, and give their abuser a golden statue for it.”
“Does 1 billion downloads really count if 90% of them were forced on people through their browsers and nobody's even wanted nor used it yet?”
“I wouldn't be shocked if the prompt processing performance was still in the toilet...”
“32Gb MI50s just got too expensive, I remember a a year ago - I could've gottem a 32Gb version for like 5$ more than the 16Gb version - and I just wish I had bought a pile of them, cause it was about 90$ ea. here is some data: https://www.reddit.com/r/LocalLLaMA/comments/1vr7wz5/comment/p4d20xb/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button it gets slower as context fills as expected with small context <16K hitting around 700tks pre”
“The passage of time has not been kind on 8GB of RAM. I still remember when 8GB of RAM was the sweet spot for gaming. On a different note, I still own a laptop with an 8th gen Intel Core i5 and 8GB of RAM and I've just continued to run it on Windows 10. Runs almost as fast as the original factory image of 1809 and is completely useable. Meanwhile Windows 11 on an Intel Ultra 7 155U and 16GB of RAM feels like a dog to use as a work machine. Windows 11 is just very poorly optimised and it sucks”
“No, its more like: tires on an lemans car (bf16, q8) -> tires on a regular nurburgring track car day (q6) -> regular high performance tires (q5) -> all-seasons tires (q4) -> we still have treads (q3) -> fucking bald-ass tires, but I gots to use it 'cause I can't afford better (q2)”
“20Gb on claude ? Damn I should download the app and see how much I'm using on my side”