Developers Pushing Large Language Model Limits
126 posts across 11 communities in the last 30 days; 72% from r/LocalLLaMA.
“Thanks, I just got it set up! 1 thing, I couldn't get my Deepseek API to work. Looks to be a json_object/json_schema issue. [re: Thingport is officially v1.0.0]”
“Tracked my first run of MiMo Pro on a Jetson AGX Thor at a scorching 0.646 tokens/sec,.. but it DID technically load and run, so it counts. Every decoder layer, routed expert, SSD read (..a good bit more) traced and replayable step by step. Here we can't what-now?.. it's ~566GB…”
“A couple of weeks ago I read this Wccftech article about Patrick Moorhead's 4 Spark setup. I meant to post this a few days ago but ran some more tests first, and then my workstation had a problem after a kernel and NVIDIA driver update. This post is long enough already, so that…”
Developers are experimenting with DeepSeek V4 to understand its capabilities, cost, and hardware requirements. They're focused on maximizing performance with limited resources, exploring context window lengths, and seeking ways to distill the model for broader accessibility. The core pain is the complexity and resource intensity of deploying and optimizing these advanced models.
SOURCES (60)
“DeepSeek-V4.1-Flash-1017's Terminal bench 4.0 score: - Model card reported 42.3% - AA reports 34% - Vals index reports 31.82% AA Vals Index Model card Model card quotes: We use the Claude Code harness for Terminal-Bench 4.0 I am guessing AA & Vals index uses mini-swe-agent…”
“Yep, I'm running this quant I built on ds4 on my M5 Max, and it works a charm. Not as fast as Flash Next at 70tps/1100pp (also in ds4, my own Q4-Q8 quant ) but it's quite usable. Crazy times. [re: 72 hours, four big open weights drops. The only one that fits in 24GB came from NVIDIA, of all companies.]”
“Another little challenge I seen on a DeepSeek Thread. I thought I'd try out it out on Qwen FN NvFp4. It was a pretty straight forward prompt with 4 images attached in the doc. The thread was testing different harness and models and showing their results. Op of that thread has two other threads on the same Challenge just one with DeepSeek Harness so I used it as well. It took about 27 min(Not bad) and was surprised how it planned everything out, also I do have vision on this Model. I think th”
“I've been using DeepSeek V4 0731 and Qwen 3.8 Flash Next both for a while and in a way they seem complementary. Lately I lean more towards Qwen 3.8 FN, mainly because of speed and visual strengths. Soon I'll have 256GB and try GLM 5.3 and DS 4.1 Flash 1017. What you can do with 128GB is amazing already. [re: Qwen 3.8 Flash Next, DeepSeekV4 Flash and Time]”
“Thanks for sharing your DS4.1 optimized inference engine! Can it run the original size if there is enough RAM? [re: Xolotl (Strata based) - DeepSeek-V4.1-Flash (552B) on a single 4090 at ~27 tok/s, Uncensored 1017 quant coming in a couple of days]”
“2x6000pro, nvfp4. It's a tight squeeze I think I maneuvered to about 700k total context. [re: As we know, currently Qwen-flash-next is the local king, but is it better than opus 4.6 max in coding?]”
“no document or announcement from unslot team yet submitted by /u/MundanePercentage674 [link] [comments]”
“Thanks, I just got it set up! 1 thing, I couldn't get my Deepseek API to work. Looks to be a json_object/json_schema issue. [re: Thingport is officially v1.0.0]”
“I have a searxng instances but stopped using it after doing a comparison vs actual search provider APIs.searxng gets rate limited fast and the results are inconsistent depending on what providers it can reach at any given time / isn't captcha'd out.Meanwhile there are multiple providers with generous free tiers so no point in feeding LLMs inferior dataAlso did a couple of side by side tests asking LLMs directly which feed results it prefer for some sample technical searches. Exa and tavily #1, b”
“not compared yet. Newer checkpoint vs a smaller quant, so I'd rather measure. But this version feels better. [re: DeepSeek V4.1 Flash 1017 on 2x DGX Spark: 57 tok/s text, 92 tok/s code with TensorFold (2.1-2.3x the vLLM kit)]”
Hey y'all, just following up on my PR to get the new vision tower for 1017 working in ds4 with the quant I promised (the language model loads already, but vision needed…
“Instead of counting 'hmm' on DeepSeek v4.1 Flash 1017, I today clamped Muse Spark Open 816B-A42B to ds4. This was the first successful run, FTW, that made it to decode... 4 t/s! #RoastMe. I'm hoping it will 4-5x with any luck. I'll obvs be running larger prefill chunks too (pre-metal leftovers), but I'm not expecting such leaps. submitted by /u/challis88ocarina [link] [comments]”
“**TL;DR:** DeepSeek-V4.1-Flash-1017 (EXL3 2.9bpw, 1M context) on 2x DGX Spark, TP=2 over the CX7 cable. TensorFold is 2.1-2.3x faster than the vLLM kit on the same pair, same weights, same prompts, same day. vLLM kit TensorFold Decode, text 27.0 tok/s **56.8 tok/s** Decode, code 39.9 tok/s **92.2 tok/s** Needle in 64K 80.4 s **35.9 s** 4 requests, code, total 1 at a time **140 tok/s** Cold prefill (random-word prompts, no cache hit, bertholomus's kit_bench client), measured tonight: Prompt v”
“it would be great. Looking forward to it. [re: Xolotl (Strata based) - DeepSeek-V4.1-Flash (552B) on a single 4090 at ~27 tok/s, Uncensored 1017 quant coming in a couple of days]”
“dealignai here. Would like to know more information about what you mean when you say you’ll abliterate the model yourself. [re: Xolotl (Strata based) - DeepSeek-V4.1-Flash (552B) on a single 4090 at ~27 tok/s, Uncensored 1017 quant coming in a couple of days]”
“This will probably get lost in the sea of praise, but my testing has shown it to still hallucinate quite a lot more than gpt-sol does. [re: deepseek-ai/DeepSeek-V4.1-Flash-1017 · Hugging Face]”
“I did initially, but given that DSV4.1 was not far off, I decided to do that instead. But DSV4 is supported. [re: Xolotl (Strata based) - DeepSeek-V4.1-Flash (552B) on a single 4090 at ~27 tok/s, Uncensored 1017 quant coming in a couple of days]”
“Cheapseek. [re: DeepSeek V4.1 Flash 1017 vs V4.1 Flash, Pi vs DeepSeek Harness: results on my 10 coding tasks]”
“TL;DR: I ran the new DeepSeek V4.1 Flash 1017 on my personal coding benchmark, and it’s really good: 100% on high for under 5 cents a task. Tokens, cache hit, time and more for all 34 setups are on the leaderboard. Hi everyone. I work on coding agents and evals at Nebius. I keep a small benchmark built from my own agent sessions: 10 tasks from my daily work, 3 runs each, checked by hidden tests. I’ve posted results for Opus, GPT and Sonnet before. This time I tried the new DeepSeek V4.1 Flash 10”
“Hey all, been meaning to post this for a while. I've been building an inference engine based off Strata called Xolotl that runs DeepSeek-V4.1-Flash, all 552B of it, on one gaming PC. My setup is a single RTX 4090 (24GB), a Ryzen 7 5700X, 90GB of DDR4 and an NVMe drive, nothing fancy. The weights get spread across VRAM, pinned RAM and NVMe, and the engine pulls in the experts each token actually needs. What I'm getting on that machine: 27 tok/s decode, 180-320 tok/s prefill 512K context 2”
it worked for me quite easily once the weights were quantized... it's a fine tune: they're identical. [re: DeepSeek V4.1 Flash 1017 Support for ds4 · PR #1225 · antirez/ds4]
“Just pushed a PR for dwarfstar that adds support for Deepseek-v4.1-Flash-1017 with its new engrams and updated vision tower. The PR basically just adds compatibility for the file revisions and checks to make sure the vision+language model file combination matches, raising on mixed pairings. Any files quantized with the ds4 quantization suite will load without any special changes, and it introduces no regressions for the previous checkpoint. This is a major revision, with ~40% of the routed exper”
“Hi all, as many of you are probably aware DeepSeek V4.1 Flash 1017 was released yesterday. I put up an abliterated model + GGUF quants for both the base and abliterated version (no imatrix yet that's WIP) For the abliterated model, most refusals are successfully removed. On the original V4.1 model, 97/100 prompts were refused; after abliteration only 4/100 are. In think mode refusals are down from 98/100 to 9/100. I optimized for keeping the model intact rather than trying to remove every la”
“any idea what a "good" context window to work with this model seem to be? [re: DeepSeek V4.1 Flash (1017) Now Leads Open-Weight Models]”
“Need at least a 7200rpm hdd for this one [re: DeepSeek V4.1 Flash (1017) Now Leads Open-Weight Models]”
“They have done some slightly dodgy stuff, like when Qwen3.8 Max was at the top of their aggregate score for a few hours before they coincidentally overhauled the aggregation [re: DeepSeek V4.1 Flash (1017) Now Leads Open-Weight Models]”
“https://preview.redd.it/thtq5y7zr1wh1.png?width=3706&format=png&auto=webp&s=e75b7f205d4c7d9bed8b45bf964e9535f3492a72 I only found one sloppy-ass slop channel that published see pic attached. Obviously they celebrate its cost efficiency and their benchmark is some nonsense but jesus fucking christ that's a big indication it's a tokenslopper Is there a better token efficiency benchmark available yet? I'm not sure AA will retest this one (I sure hope so though) Mainly I care”
“Not as long as they are making money serving the big ones. [re: deepseek-ai/DeepSeek-V4.1-Flash-1017 · Hugging Face]”
“why qwen would wipe 100b? its more expensive to serve than ds flash [re: DeepSeek 4.1 Flash 1017 Incoming? --> DeepSeek users report a possible V4.1-Flash-1017 refresh across its API]”
“Just curious, what would be the cheapest self host setup to run this? [re: deepseek-ai/DeepSeek-V4.1-Flash-1017 · Hugging Face]”
“That is an incredible stretch [re: deepseek-ai/DeepSeek-V4.1-Flash-1017 · Hugging Face]”
“I tried deepseek v4.1 flash via API yesterday and honestly it felt pretty dumb Try it with OMP harness and https://www.aihero.dev/skills 1) get with best model you have grill -> to spec -> to tickets. I use here opus $20 plan 2) then run each ticket with DS. I use my skill to chain run spec tickets 3) define in OMP glm as reviewer model 4) if spec is really complex then use SOTA to double verify after all sub tickets are done In this kind of tasks DS41F is amazing [re: DeepSeek 4.1 Flash 1”
Colibri when? [re: deepseek-ai/DeepSeek-V4.1-Flash-1017 · Hugging Face]
lulz [re: deepseek-ai/DeepSeek-V4.1-Flash-1017 · Hugging Face]
“Love it. I was so happy to read about it just after I finally landed with new motherboard and platform as a whole with 4x RTX 6000 pro on 4 x16 PCIE slots :) [re: DeepSeek 4.1 Flash 1017 Incoming? --> DeepSeek users report a possible V4.1-Flash-1017 refresh across its API]”
“Probably a week ago someone posted a thread with the title "Why isn't the industry freaking out about DeepSeek 4.1 Flash?"I have been intrigued about Chinese models for a long time, but never tried anything except an early Qwen. My daily driver was Luna, with occasional elevation to Terra or Sol - all on 5.6 as I found the output of 6 jarringly incomprehensible.I still had 95 USD on my OpenRouter account, so I thought I would give DeepSeek Flash 4.1 a try.I think now I can understand the person”
“Great. Terminal v4 seems similar to glm 5.3 [re: deepseek-ai/DeepSeek-V4.1-Flash-1017 · Hugging Face]”
“Was? Isn't it anymore? [re: deepseek-ai/DeepSeek-V4.1-Flash-1017 · Hugging Face]”
“他们应该在训练一个非常小的模型，目标是在5070ti这样的显卡上运行，不一定是v4.1这一代的架构 [re: deepseek-ai/DeepSeek-V4.1-Flash-1017 · Hugging Face]”
“im just lurking , hugging face is like github for LLMs? The models can get really big though, so HuggingFace is more like PirateBay for LLM's. [re: deepseek-ai/DeepSeek-V4.1-Flash-1017 · Hugging Face]”
Was waiting for this. submitted by /u/chemist_slime [link] [comments]
“Tracked my first run of MiMo Pro on a Jetson AGX Thor at a scorching 0.646 tokens/sec,.. but it DID technically load and run, so it counts. Every decoder layer, routed expert, SSD read (..a good bit more) traced and replayable step by step. Here we can't what-now?.. it's ~566GB of model (FP8), at the blinding speed and architecture of hand-extruded hotdog manufacturing, read from disk mid-inference, painfully shat through 128GB of memory. I think it's fine but decided to maybe take a”
“English version of 63. See the original issue for the Chinese discussion. Some background: 1. OpenCode Go uses multiple upstream providers, including proxies to model vendors’ own APIs, third party deployments, and deployments hosted by OpenCode Go itself. 2. When using the Chat Completions protocol, OpenCode Go routes requests across these providers based on load. 3. The Go documentation recommends Chat Completions for DeepSeek V4.1 Flash, making it the protocol most users are likely to use. Co”
“Deepseek v4.1 flash is basically free on the web/api. I would not spend 30grand or more chasing a local version of this model. I love local llm's but i love telling people to keep their money more [re: Numbers for Deepseek v4.1 Flash on Studio M5 Ultra]”
“Hi, anybody ran this yet? I'm thinking about getting the 512GB when it's out, but I wonder if I would be better off running 4x DGX Spark instead? submitted by /u/AppealSame4367 [link] [comments]”
“nice [re: DeepSeek V4.1 Flash on 4x DGX Spark: my numbers vs Patrick Moorhead's results via Wccftech.]”
“A couple of weeks ago I read this Wccftech article about Patrick Moorhead's 4 Spark setup. I meant to post this a few days ago but ran some more tests first, and then my workstation had a problem after a kernel and NVIDIA driver update. This post is long enough already, so that will be another post. Initially I thought this Patrick Moorhead guy ran these benchmarks and was interested in the setup. I was just surprised at how much information was left out. Neither the tweet nor the article sa”
“Follow-up to u/neuromacmd 's post from a few weeks back ( https://www.reddit.com/r/LocalLLaMA/comments/1wod07t/ ). The affinity engine ( https://codeberg.org/StillDeadcode/affinity ) refuses to start on more than two cards because the author only had two, simulated four ranks on them, and got repeated-token garbage they couldn't explain. Nobody had tried it on four real R9700s. I have four, so tonight I did. Short version: it just works. Setup: Threadripper PRO 9955WX, 128 GB RAM, four R”
“MacStudio 128GB M1 ultra I'm curious how people who can run both models are deciding which one to work with. DeepSeekv4 benchmarks right below GPT 5.5 high. Flash Next benchmarks right above 5.4/5.5 xhigh Both seem capable but it seems like Qwen 3.8 Flash next is taking much longer to actually finish anything. From the chart, it seems like I can do 5 tasks with DeepSeekv4 in the time it takes to do 1 with Qwen 3.8 Flash Next and Flash Next isn't 5x better than DeepSeekv4. In my personal”
“Look I love using Deepseek's API as much as the next guy and I have no complaints about GLM 5.3 Flash either (I use it from time to time). All I was pointing out, is that with the adequate hardware, and prompts that RP is fine on Qwen 3.8 27b. [re: Character.AI files for bankruptcy, to sell some assets to Disney for $20 million]”
“Lmao the past few months we got qwen 3.8 QFN, 27b, deepseek v4/flash/vison exp/0731/4.1, GLM 5.3/flash, KIMI K3 and the list goes on. Do you have kvcache turned off? [re: Anyone else notice Chinese model releases slowed way down after distillation protections went in place?]”
“DeepSeek guys, thank you so much! Don't give this up. mac setup [pi.dev] -> [ds 4 flash 0731 - brain] -> [qwen 3.6 35b a3b - body] - works perfect submitted by /u/AleksandrNikitin [link] [comments]”
“Coming soon! I'm waiting on RAM upgrades to test it out and release! [re: Basalt: Fast Blackwell inference engine for Qwen 3.8 Flash Next (and how we got vision for free)]”
“DeepSeek 4.1 Flash and GLM 5.3 Flash dropped like a month ago and have been extremely successful. Models already release too damn fast! Kimi has been a bit quiet though. [re: It's been too quiet on the other side of the Pacific]”