Local LLMs require significant computing power

“While I'm against AI slop, I think it's not wise to discount it completely. Using the same example, you can say the same for China - they are not learning anything since they just copy. But they got so good at copying and look at where they are now. Similarly vibe coders can get so good at using AI and go from there.”

“I love my HA set up but find stock voice stuff hard to use, slow and not very useful. I think it's better if you tie in a local ai but I haven't done it. I moved away from voice commands and instead try to have thing triggered by conditions (movement, humidity, door sensors, etc) and some ZigBee buttons in strategic places. G suite I replaced with next cloud. Some people thinks too heavy or cumbersome to set up but I didn't find it to be so. You also might be interested in r/graphene”

“I trust Google less than I do Apple and I’m too lazy to mess with jailbreaking/installing/upkeeping a custom install.”

“You are misunderstanding the situation. Its not laptop at that point, it's mobile workstation :3”

“Personally I can't say I run them on a laptop, but I have run them on Google Edge Gallery on my cellphone. And, indeed, it burns battery very quickly. Consider that when we talk about neural network processors, we tend to measure the processing speed in TOPS (though the actual bottleneck is more likely to be memory access speeds). TOPS stands for, "Trillions of Operations per Second." ARM chips are more efficient at that, which is part of why they'vw returned. But trillions of”

“Maybe it's related to *local*llama. A lot of people in this sub are looking for independence by using their "own" LLM and refuse to share their private data. EU is the opposite of independence. Concerning private data, the worst is coming : already done in France since this summer, parliament (EU kollabos) voted for the end of anonymity on social networks, while all French were on vacation.”

“Maybe they want the ram Maybe it's a denial tactic The mac mini can be used for local inference”

“Yes you state valid points, I feel mistral don't have (yet?) the ambition to buy or licence all the textual knowledge like openai did (or didn't) I feel they are more on the light weight tailored to your needs kind of things. May be they just decided that running for frontier isn't worth it for now as models keep raining every 3 months and that they have better to do with their time. But I may be wrong You need some serious shoulders if you want to have the kind of amusement these US”

“Yes, these provided instructions that I've used on several machines personally, is not worth it to "try the latest models on my hardware". You can't learn anything. You can't do "hacky stuff". Even though we don't know what cpu you have, whether you are willing to write some scripts and let them run over night, ignore this advice. Even though i suggest gemma, a lighter weight model grom two competitors, I am chatgpt.”

“I was really tempted to copy paste this post into gpt, and have it reword it to reply to your comment.. But it's too early for me, and so I didn't. But just know.... I almost did.”

“Yeah you're right. I'm still angry though, I don't really trust SoftBank as a good steward”