Developers Pushing Large Language Model Limits
Developers are experimenting with DeepSeek V4 to understand its capabilities, cost, and hardware requirements. They're focused on maximizing performance with limited resources, exploring context window lengths, and seeking ways to distill the model for broader accessibility. The core pain is the complexity and resource intensity of deploying and optimizing these advanced models.
SOURCES (60)
“Oh okay sorry about that. I'd recommend the Deepseek v4 flash. It's priced like an American nano model but is intelligent on the level of the previous generations of frontier models. Meta's Muse spark models are also relatively cheap for the level of intelligence. Gemini…”
“I'm now on embedded tunneling epic, a spike. NGL my AI crafted this question to you. We're building Kin: a mobile app connecting securely to a user's self-hosted local Bridge/OpenClaw instance. We're specifically investigating how to eliminate dependency on Cloudflare/Tailscale while keeping the connection reliable. I'm interested in the networking architecture you chose for KeySync: how do two machines establish/re-establish connectivity, what requires infrastructure, and wh”
“https://preview.redd.it/5rs3d7vtgbnh1.png?width=950&format=png&auto=webp&s=df828c1146bc6167bd09d3300018aba426fc1ab2 Hey, r/LocalLLaMA ! We are releasing SupraGDN-5M. It's a tiny GatedDeltaNet (GDN1) being trained on 2B tokens. https://preview.redd.it/8gzkf0kggbnh1.png?width=735&format=png&auto=webp&s=9517d3eb88582a57b911bacd8386fca6c59f4fe9 As you can see in the benchmark table above, SupraGDN-5M ("Supra-5M-GatedDeltaNet" in the image!) is almost as good as”
“Upgraded my work LLM server from DSV4-Flash-0731 to the new Vision-Exp model and did this little test for fun to see quality comparison. Test was performed in Openweb UI connected to my vLLM container serving the vision model. Was able to fix the previous KV cache mem issues, getting 2.5million KV cache now running 350k context @ 7 concurrent. ~120t/g ~3000-7000p/p with this model. Prompt: Write a single self-contained HTML file using Three.js (loaded via CDN import map) that renders an animated”
“Qwen3.8 flash next's implementation is very unstable in many inference engine. Give it more time.”
“I went from Deepseek Flash locally falling back to APIs like DS4/Kimi for complex tasks it failed at to GLM 5.3 Flash for everything. Deepseek Flash is good at implementation, but I wasn't always a fan of how it planned things. It's only been a week or two, but I haven't felt the need to use anything but GLM 5.3 Flash since it's been out. I've seen some people say they like Deepseek Flash better, so it might depend on use case.”
“GLM 5.3 Flash runs on 256 GB with community weights. Official weights would require to raise the Sparks from 2 to 4 devices. At the current price point, a pill hard to swallow.”
“Give a try to github copilot with custom endpoint in vs code. I found the build-in tools or prompts are good”
“I can't contribute to the question at hand but have been extremely curious about this as well. I run DSV4 as my daily driver but debating on adding more VRAM to upgrade(?) to GLM.”
“> dsv4 vision Do you use the vision it for screenshots, or scanned/photographed pages? DS4F Vision could not read normal pages on my setup.”
“I've tried both and been having this debate with myself for the last few days, on two Asus Ascent GX10s (effectively the same as 2x DGX Spark): DeepSeek-V4-Flash-0731 (official weights) GLM-5.3-Flash (RedHatAI/GLM-5.3-Flash-NVFP4) Have any of you guys also tried both on this hardware (2x DGX Spark / Asus Ascent GX10), and what are your use cases and findings? DeepSeek runs with more tokens/s… but GLM feels like the better tool for how I actually work. I'll share my experience. Where Deep”
“Thanks for the update. I am running it now - using the same unsloth dspark from the non-vision version with similar performance. If you're using DSH as the harness, make sure to update settings.yaml in ~/.dsh for the vision-toolkit. Something like vision-toolkit: provider: baseUrl: http://127.0.0.1:8000/v1 credential: LOCAL_API_KEY # resolves via your .credentials.yaml model: DeepSeek-V4-Flash-Vision-Exp-UD-IQ3_XXS protocol: openai language: en timeoutMs: 180000”
“deepseek flash is very good at vulnerability discover, covering 70% of pro results. glm flash is at the bottom”
“I think right now they're somewhat focused on model architecture optimization rather than getting great post training on enormous corpuses of high quality training data. All the big labs have lots of coding agent training data to train on, and deepseek doesn't really have that, so most of their headlines come from cool efficiency gains rather than saturating benchmarks.”
“When will they catdh up? They were one of the top labs when ds v3.2 and v3 came out , but now ds v4 pro is worse than qwen 3,8 next in benchmarks. It seems like ds v4 pro is not trained to its full potential , but v4 flash is pretty good. Maybe the kv cache compaction and efficient hybrid attention are affecting its performance. Also they lost some talents to xiaomi and other labs. Will ds beat qwen And be on par with kimi and glm In one or two months? submitted by /u/power97992 [lin”
“a bit late but you configured the context length for each model in the custom provider?”
“Unsloth GGUFs and Vision support https://huggingface.co/unsloth/DeepSeek-V4-Flash-Vision-Exp-GGUF submitted by /u/fmillar [link] [comments]”
“Summary Serving DeepSeek V2/V3 through the OpenAI continuous batching server ( api server v2 ) produces incoherent multi token output : the first generated token is correct, then it collapses into a repeated token. The failure is silent (no error/warning) and deterministic , and is unrelated to MOE DISABLE FUSED KERNELS (verified byte identical A/B). Reproduction Model: deepseek ai/DeepSeek V2 Lite Chat , greedy ( temperature=0 ), via api server v2 /v1/completions : | Prompt | Output | | | | | T”
“Extended reasoning and post-training appear to be the keys used by DeepSeek, Qwen, and GLM to boost performance (leveraging higher token counts). And Qwen 4 hasn't even been released yet. Of course, we don't know if that release will be open-sourced, but I am optimistic about future models, featuring "engrams", that could soon match or surpass 2.4T parameter models on specific tasks. submitted by /u/LegacyRemaster [link] [comments]”
“Anybody else struggling with deepseek after the initial prompt? Somehow it is getting mixed up very easily, even button functionality has been PITA when doing stuff. Never had these problems with Opencode. With Opencode, a continuation prompt on the software or task is pretty much magic at this point. But the inital result isnt as strong imo, and needs a lot of follow up even if a a detailed spec is written. The oneshot potential of deepseek is goated imo, it is a relentless harness that just ke”
“I maintain KeySync, so I want to put the affiliation up front. Rather than call it "the best" in the abstract, I compared it with six other ways to reach DeepSeek Harness using the same acceptance criteria. The target user here is not a network engineer. It is someone running DSH on a work PC or headless Linux server who wants the complete native Web UI from a phone or another computer. I used nine checks: no SSH, VPN, or reverse-proxy knowledge required headless Linux support access f”
“I’ve been working on getting DeepSeek-V4-Flash-Vision-Exp running properly under SGLang on a 2× RTX PRO 6000 Blackwell setup, and figured I’d post the results here in case anyone else is pushing this model on SM120 hardware. Current setup: 2× NVIDIA RTX PRO 6000 Blackwell Max-Q (~96 GB each) Ubuntu 24.04 Driver 610.43.02 / CUDA UMD 13.3 SGLang Vision preview lineage FlashInfer 0.6.18 TP=2 MXFP4 MoE DSPARK speculative decoding context-length=278528 chunked-prefill-size=8192 max-running-requests=8”
“Hi people of LocalLLaMa, I have been wondering for quite some time now - and this all started after I read some comments complaining about the pricing on Qwen 3.8 27B as opposed to DSV4 Flash that it mainly was driven by how massive its KV cache overhead was. And while I did agree with that, what I did wonder later on was why could we not finetune that away. Apparently, I am not the only who has thought of that - Arcee, an open source friendly company that does a lot of neat work and gave us AFM”
“hello, I'm using Deepseek flash regularly and from time to time i see it deviating and start doomlooping or generating gibberish. It's somethign i already saw in heavily quantized model buthere i used official deepseek release https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731 . I would be curious to know if anyone encountered such thing and how they solved it . Here is my config : vllm serve deepseek-ai/DeepSeek-V4-Flash-0731 \ --trust-remote-code \ --safetensors-load-strategy pr”
“IIRC similar observations in other tests, "no thinking" creates additional overhead/error/loop tokens whereas "thinking" nails it down at first shot”
“Hi! I noticed that DeepSeek V4 models always run with thinking enabled at the default high effort, and there's no way to change that through Koog. DeepSeekModels for V4 Flash/Pro declare LLMCapability.Thinking , but DeepSeekParams has no field for it — so I can't disable thinking (to save latency/tokens) or pick a different effort. The API supports this via Chat Completions: thinking: {"type": "enabled" | "disabled"} (enabled by default) reasoning effort: "low" | "high" | "max" (default high ) D”
“I think this is what I had in mind. Correct me if I’m wrong, but the weights themselves are 4bit, but the activations, optimizer states, and gradients are in higher precision so you can have fine-grained nuanced updates to the 4-bit weights?”
“DeepSeek has released the weights and reference implementation for deepseek ai/DeepSeek V4 Flash Vision Exp , the first experimental multimodal model in the DeepSeek V4 family: https://huggingface.co/deepseek ai/DeepSeek V4 Flash Vision Exp Would you consider adding local inference support for this model to ds4? The model is based on DeepSeek V4 Flash, but adds a vision encoder, an aligner, and multimodal prompt encoding. The upstream repository provides: MIT licensed model weights a minimal PyT”
“Just saw this. https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp submitted by /u/Key_Solid_1696 [link] [comments]”
“Deepseek seems like the obvious winner especially when they’re a thirtieth of the price of Claude”
“pi.dev for customization opencode for out of the box working claude code if you love a 30T IPO reasonix if you want to use deepseek”
“around 150tg and 7000pp at 50% acceptance rate MTP, scales well. Deepseek won't fit as many tokens GLM... good luck with running that at IQ4 KS for multi concurrency via llama.cpp (too slow) and you have 14k worth of GPUs (I guess 28k now)”
“Not the specific DSpark version, target this one: https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731 (it's the latest version and also ships with the DSpark head)”
“I crossed 1 billion tokens on Deepseek this month. The cost is shockingly low! https://preview.redd.it/7kwidsr44lmh1.png?width=1431&format=png&auto=webp&s=b28358c77e7b6214437ca15f2dd58ef061794167 submitted by /u/chuckbeasley02 [link] [comments]”
“Awesome! A question from a newly minted Local AI'er, why docker and not just run directly on the server/pc?”
“Makes you appreciate how Deepseek is both cost-effective and capable vs GPT 5.6 ! Thanks for posting the extra comparisons.”
“This is comparable to DGX spark. All of these cheaper inference solutions use LPDDR5X”
“HBF gives 8x - 16x more capacity than HBM at same cost, and with bandwidth till 3 tb/s. submitted by /u/9r4n4y [link] [comments]”
“glm5.3-flash beats deepseekv4flash-0731. both are great and only time will show their strengths. i'm sure there are tasks tht dsv4 will perform much better at, as you mention benchmark doesn't often tell the whole story.”
“Please add EXL3 to your test. I'm running that instead of NVFP4 with the belief that it is even better. What's amazing about your results is how GLM 5.3 Flash solved tasks FASTER.”
“I love this sub - people legit get excited when somebody gets one of these monsters running at 1TPM, even if practical use is ask a question and it may be done thinking in a week or so.”
“I have a 2 DGX Spark setup recently and I have been happily running Deepseek V4 Flash 0731. Since the release of GLM5.3 Flash and Qwen 3.8 Flash Next this week, a lot of folks are still waiting to see what model to run given their own hardware situations. I am very interested in running a GLM model locally and looks like nvfp4 would be a good option for my setup, but I have been hearing a lot of conflicting opinions (mostly negative) about GLM5.3 Flash on nvfp4 quant which gave me pause. So I de”
“Finally achieved usable results with 2 gx10 at over 65 tokens a second sustained. The 2570 prompt eval is really crucial for me as well. Overall stoked 10/10 edit: I followed this setup with 2 ASUS GX10 DGX computers :) https://github.com/tonyd2wild/DeepSeek-v4-Flash-0731-DSpark-1M-NVFP4-KV-2x-DGX-Spark submitted by /u/koalfied-coder [link] [comments]”
“Most of my work is done with Luna on High. If I need a little more brains, I'll use Terra on Medium. I've been experimenting with Deepseek V4 Flash and Kimi K3 a bit, but they seem to stall a bit sometimes. My work is mostly administrative and not very complex. What are all of you using? submitted by /u/Topherho [link] [comments]”
“Your current environment Worker TP7 pid=345) ERROR 08 01 14:43:38 [multiproc executor.py:1007] RuntimeError: Assertion error (/workspace/.deps/flashmla src/csrc/sm90/prefill/sparse/instantiations/../phase1.cuh:614): Assertion res == CUresult::CUDA SUCCESS failed. detail+full log : 123.txt 🐛 Describe the bug vllm/vllm openai:v0.26.0 h20 3e start command vllm serve deepseek ai/DeepSeek V4 Flash DSpark \ trust remote code \ kv cache dtype fp8 \ block size 256 \ enable expert parallel \ tensor para”
“Your current environment [BUG][Regression] Assertion res == CUresult::CUDA SUCCESS failed in FlashMLA ( phase1.cuh ) for DeepSeek V4 on v0.26.0 (Works in v0.25.0) Describe the bug This is a regression issue on vLLM v0.26.0 . When serving DeepSeek V4 Pro with long context prompts (e.g., prompt token ids len=678133 ), the vLLM engine crashes during the prefill stage with a CUDA assertion failure in FlashMLA's sparse prefill fwd kernel. Important Note on Regression: This exact same setup and worklo”
“Your current environment <details <summary The output of <code python collect env.py</code </summary </details 🐛 Describe the bug Background: Long Context & Multi turn Interactions This issue is particularly prevalent in complex agentic workflows characterized by: 1. Long Context: As the conversation history grows (e.g., 20k+ tokens), the parser's reliability in detecting DeepSeek specific protocol tags seems to degrade. 2. Multi turn Tool Calling: In scenarios where the model needs to call mul”
“🚀 The feature, motivation and pitch We will drop support for V0 in the very near future, and if we want to continue supporting this model in vLLM, it needs to be ported to V1. The main work here is in: 1. Adapting the custom Mamba layer to match how V1 manages the mamba state. The MambaMixer, MambaMixer2, MinimaxLinearAttention and ShortConv can all be used as a reference for how to do this. 2. Porting the differential attention backend to V1. Alternatives Drop model support in vLLM Additional”