AI Researchers, Sharing Open Source Work

“I maintain TensorSharp , an open-source local inference engine and agent runtime. I’ve added Video Studio to bring local video generation, an editable timeline, and an optional local LLM assistant into the same project. The attached demo runs in TensorAgent on macOS . The actual run Machine: MacBook Pro with an Apple M5 Pro chip and 48GB of memory. Model: LTX 2.5 Distilled, loaded locally. Generation: Image-to-video at 1024 × 1024. Output: 145 frames at 24 fps, approximately 6.04 seconds, with a”

“Worked on this for about a month. I experimented a lot with resnets (pretrained on Imagenet) which performed quite well, pure ViT which performed worse. ConvNext DINOv3 performed the best. For data I rendered about 40000 varied blender scenes, and used 8000 labelled real images. I had to label most of these real images by hand, which was a huge pain in the ass and about 20 hours of my life I'll never get back. Both registration and detection networks were trained on about 1 million samples (”

“Hi folks! We are HardenedLinux, an old-school free software community. SonicBoom is our open-source (GPLv3) inference runtime: a compiler-first alternative to the usual Python stack. No Python at build or run time — one shared library, a C ABI, and Guile Scheme as the first-class language. What v0.0.3 does on a desktop RTX 3050 8 GB (same session, same GGUF, vs llama.cpp -ngl 99): model decode (tg64) prefill (pp512) Gemma 4 E4B Q3_K_M 1.18× 1.00× Qwen2.5-0.5B Q4_K_M 1.12× 1.09× On real multi-tur”

“I built GPT-2 small and a KV cache in PyTorch as a learning project, then compared the mathematical savings with measured latency and memory. I have published the implementation and a detailed write-up, and would appreciate feedback on the methodology. Setup: pretrained GPT-2 small (124M), NVIDIA A40, fp32 with TF32 off, batch size 1, greedy decoding. Timings are bracketed by torch.cuda.synchronize(), after warmup, using the median of five runs per context length. Correctness: maximum absolute l”

“A solver can give you an answer without giving someone else an easy way to check its accuracy. I built Certloom to package the equations, proposed answer and proof witness together. The new sparse example uses a captured NIST FiPy diffusion solve: 400 unknowns and 1,920 coefficients. In the browser, select “FiPy · steady state”, then “Check with exact arithmetic”. It bounds every solution component and establishes whether component 19 is above 0.98. “When weights fail” refuses the proof; “A chan”

“Total Parameters 4B Architecture Looped (latent-recurrent) normalized-GPT with interleaved sliding-window attention (SWA) and full-attention layers Precision BF16 Context Length Up to 256K tokens Supported Languages English Best For General-purpose reasoning on a small footprint License OpenMDW License Agreement, version 1.1 Release Date October 16, 2026 NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16 is a looped (latent-recurrent) large language model (LLM) trained by NVIDIA. The model employs a normalize”

“hey, CS student here. for the past few days I've been building DenseCore, a small deep learning framework in rust with a pytorch-like python API. the thing I'm kind of proud of: if you point `import torch` at it and run karpathy's nanoGPT unchanged, you get the exact same numbers as pytorch. same initial weights from manual_seed, same loss, same gradients, same weights, bit for bit, for all 50 steps I tested, even with dropout on. not "close enough", literally identical. ge”

“Diffusion language models are the future! submitted by /u/billy_booboo [link] [comments]”

“Seeking arXiv cs.CV endorsement: quantized YOLOv8 drone detector on FPGA, code and weights open”

“Went down the rabbit hole to map the different Post-Transformer architectures. Various directions include Dragon Hatchling (BDH), Mamba, RWKV, xLSTM, Titans, and a bunch of hybrid models coming from different AI Labs. My interest areas are architectural changes that can improve reasoning, memory, and adaptation within practical compute and energy budgets. IMO, the next frontier might be less about scaling alone and more about getting more intelligence out of limited resources. That's what go”

“Hi all, so im making a binary noise classification model. I have training data with timestamps of noise and some with verified some with unverified. The performance parameters are F1 and Dice scores and their combined score. I am following AST backbone (Stage 1 ast frozen and stage 2 ast unfrozen along with BiGRU/CRNN head in stage 1 and then the mean teacher in stage 2). So the sound event detection model is primarily struggling with precise temporal boundary localization, where the majority of”

“I've seen this one floating around quite a bit now and since this is something that can be done without the parent and or other dependencies i thought i'd share something. Usually when you fine-tune a model you can't: - train against the quantized GGUF file itself - do it without PyTorch, transformers or a separate Python setup - use the exact same GGUF for inference afterwards, without converting the model (QLoRA does train through a 4-bit base, but in PyTorch and on its own NF4 for”

“I built pytest-torch to make testing PyTorch models easier Writing the same device setup, reproducibility checks, and output comparisons across projects gets repetitive, so I built pytest-torch, an open-source pytest plugin. It includes inference checks, compile/export parity, checkpoint tests, accuracy evaluation, and layer-by-layer diagnostics to help locate where two models diverge. It’s currently in alpha, and I’d appreciate feedback—especially examples of model bugs that are frustrating to”

“I've been working on an inference engine called jitllm. The idea came from my background in operating systems and JIT compilers: generate kernels for whatever hardware is there, and treat layers a bit like processes the engine can place and move around. In the video, a cloud VM with T4s: mid-reply, all layers of Llama-3.2-3B move from one GPU to the other and back. And my puny laptop where the model is split across CPU, CUDA and Vulkan (iGPU), rearranged while it's generating. The KV cac”

“I kept wanting small, custom models for my own tasks without renting GPUs or sending my data anywhere. So I built streamweights: give it a CSV of examples (input and the output you want), and it trains a small model on your own Mac or Linux machine. It's designed to be honest with you. Every build is compared against the untrained model and a dead-simple baseline. On one test, sorting messages into categories, the simple baseline won, and the tool said so. On another, pulling structured data”

“I built pytest-torch to make testing PyTorch models easier Writing the same device setup, reproducibility checks, and output comparisons across projects gets repetitive, so I built pytest-torch, an open-source pytest plugin. It includes inference checks, compile/export parity, checkpoint tests, accuracy evaluation, and layer-by-layer diagnostics to help locate where two models diverge. It’s currently in alpha, and I’d appreciate feedback—especially examples of model bugs that are frustrating to”

“https://preview.redd.it/m5e9anluo8vh1.png?width=1080&format=png&auto=webp&s=4c80687e6fe40036843a97ed8975ce46eaab0662 I am excited to share a project I have been working on during the past few years during my free time. I started this project after being unable to use torch-points3d at the time (too many installation errors). Note that this project is still in active development and in alpha. I am looking for early users and feedback! What it is: torch-pointcloud is a plain-PyTorch li”

“OK, here's a different tack, then.It is almost completely impossible, and therefore a waste of time, to attempt to gatekeep what other people wish to spend their time and energy on.There's no single hand holding the pursestrings of research. There's no single path to implementation of theory IRL.Talking about what "should" be studied is arrogant at best.”

“The problem with "it's hard to believe" is that this is an area where I'm not even sure how to effectively estimate the counterposition, I've seen things that I thought were interesting get cancelled or have no follow up all the time, and it makes sense, no idea if it will matter, or even work, and even if it does, how much will it cost to realise any value?Who knows how often really impactful tech is gated behind some non-obvious thing which we never follow up on.I mean Douglas Engelbart basica”

“Heyo!I'm Adit, an undergrad in Finland. This is a preview of a side project I worked on last weekend. Essentially, Dragonfly is a LoRA fine-tune of Qwen3-VL with typed output heads that answers many questions about an image in one pass, with a probability for every answer. Unlike a VLM that generates an answer to each question, Dragonfly reads the image once and scores every question in the same pass. Questions can't see each other, so each answer is the same as asking it alone, in any order.Ins”

“I’ve been building SonicBoom, a native tensor runtime written in C++/CUDA with a C ABI and no Python anywhere in the stack. I just tagged v0.0.1, and the first target is local LLM inference on consumer GPUs. Numbers (Gemma 4 E4B-it Q3_K_M on a laptop RTX 3050 8 GB, both runtimes using the same GGUF on the same machine, llama.cpp at commit 43fe9c64281 ): SonicBoom llama.cpp Decode (tg64) 51.65 tok/s 45.20 tok/s Decode @ depth 2048 47.13 tok/s 45.21 tok/s Prefill 4100 tokens 1051 tok/s ~1944 tok/s”

“Pytorch, the literal name sake of this sub. And they are the only US lab to give some shit [re: It's been 2 months]”

“A few weeks ago we posted the open-source Graphsignal profiler here: wrap vllm serve / sglang serve / whatever in graphsignal-run and it serves everything it measures as JSON, mainly so an agent can read it in a loop. Turns out we kept opening the JSON ourselves to see what the GPU was doing. So now there is graphsignal-top : graphsignal-run vllm serve <model> --port 8000 # one terminal graphsignal-top # another What you get: per-GPU util over time, memory, power, throttling; the engine&#3”

“I am an independent researcher. Due to budget constraints, I shifted my 21B MoE pre-training—which I had originally planned to run on 64 H100s—to a single NVIDIA GH200 ($2.29 per hour), and I have compiled the metrics I measured during that process into a paper. https://preview.redd.it/77womlml91vh1.png?width=655&format=png&auto=webp&s=5478597dcc07a74c07877ab792e996e13a6d49c5 I hope this will be helpful to those starting out with LLMs on a limited budget. These are the metrics I reco”

“Hi HN,I'm a big fan of the liquid-dsp C library and thought it'd be fun to bring it over to Rust. I'm calling this library Yagi DSP. The browser demo I've linked here is just a very basic example to show how different modems work in the presence of noise, but it really does use yagi via wasm-bindgen.liquid-dsp has more than 1300 tests and yagi now has ported and passes more than 95% of them. I've just finished a major refactoring of the library with lots of API churn now behind me so I thought i”

“Awesome work. I had Claude and ChatGPT have a go at similar optimisations the other week but yours blew mine out of the water. The things I tried was n-gram reuse, tuning DFlash, some kernel tuning and what I called "card shaped quant", that is quantising such that the weights fit the card's quirks. I ran a head-to-head test of your version against mine (on the exact same weights, yours) and your megakernel approach is slightly better. So I think for now one approach to improve you”

“I wrote a small boid simulator (12 birds), recorded it flying, and trained a transformer to predict each bird's next move without it knowing about any boid rules. (built in Python and PyTorch: a transformer written from scratch, one token per bird, two attention layers, about 414k weights, trained on a laptop CPU) what I found: it flies the flock: R² 0.990 to 0.994 on clips it never trained on, across 4 full runs. It also flies 50 birds (it has only trained on 12). for comparison, a small ne”

“https://github.com/ggml-org/llama.cpp/pull/27694 Now merged. Update your llama if you haven't done so yet. Optimal draft-n-max / draft-p-min seem to be in line with greedy sampling. Main gain seems to be on prose generation. Tests above ran with thinking off, ngram-mod off. submitted by /u/Dreeew84 [link] [comments]”

“Open weight AI is like open source software. Users not only run the weights, but also modify the weights. It matters to develop training framework for local hardware.The method to train large AI models on local hardware is generally called QLoRA (Low Rank Adaptation over Quantized base model). In the last few years it's usually done with HuggingFace Transformers (which is the basis of training frameworks such as Unsloth and Axolotl) and bnb 4-bit base model. However, bnb does not yet support MoE”

“Quite a few real world data problems look like this: many related time series, few noisy readings, and long gaps. Local price indices, sensor networks, or small area stats. I built a small PyTorch model to estimate a smooth underlying level for every location, day, and target even where nothing was observed. Key ideas: Mask, don’t drop missing targets in the loss Weight observations by how many raw readings they average Share information across locations (area + station offsets) and across relat”

“ALHR - Adaptive Learnable Hierarchical Routing, uses static binary trees and learnable functions to minimize the amount of keys to be read. It does use a dense teacher while phase 1 of training however. MQAR TEST AT 1024 TOKENS - Average keys read per query by dense - 512 Keys Average keys read per query by ALHR - 30 keys Top - 1 accuracy of dense - 94.9% Top - 1 accuracy of ALHR - 92.1% KV Compression of dense - 1x(100% read) KV Compression of ALHR - 35.3x(2.83% read) Peak VRAM of dense - 57 MB”

“I built MaRN (Mapping Networks) , a PyTorch library that lets you optimize a compact latent representation instead of directly training every model parameter. Some results from my current benchmarks: MNIST CNN: 107,998 → 1,872 trainable parameters ( 57.7× reduction ), with 91.80% accuracy. LSTM forecasting: 12,051 → 2,048 parameters, with validation MSE of 0.00006. CNN2 + pruning: 204 trainable parameters, with 81.25% accuracy. These results come with trade-offs: mapped models can train substant”

“TornadoVM: The Java to CUDA engine: https://github.com/beehive-lab/TornadoVM jitLLM: The inference engine: https://github.com/beehive-lab/jitllm Deep dive talk: https://www.youtube.com/watch?v=HO5CpETzywk submitted by /u/mikebmx1 [link] [comments]”

“Hey everyone, I have new in brackets above because I'm not necessarily inventing anything innovative in terms of the actual mathematics or optimizations behind some quant techniques, but I'm pretty happy with how things are coming. What I'm working with is basically a "poor man's" RCO (the quant method from IST Austria dAS lAB). Exact same concept: choose a type per tensor under a byte budget while optimizing task KL on the whole model. I worked on GSQ but I don't h”

“I started playing with neural networks in the 90s, on a 386 DX 40MHz. I think I was using Pascal. To me, even the image classification neural networks from 2012 were insane, and LLMs are just magic. And the ability to run an LLM locally is insane magic.”

“I have a somewhat specific question in case anyone has experience with this. I am a big fan of CS, math, and physics. While I was able to get formal instruction for the first two, I was never able to get the opportunity to learn physics. The most wonderful part about llms for me is that I can pursue that now without the costs of college tuition. My current process is the following. I pick up a textbook. I read it section by section and almost always don’t understand on the first read. Then I ope”

Doesn’t this only trade blows with the older jina Omni?

“I kept running into the same thing while learning. You follow a tutorial that says implement cross entropy from scratch. You write four lines. It works on the toy example. Then you open the actual torch source and the real function has ignore_index and label_smoothing and reduction modes, and it handles 2D and 4D inputs, and none of that was in the tutorial. So you think you understand cross entropy. You don't. You understand a version of it that someone simplified so the blog post would fit”

“Looped language models repeatedly apply a weight-shared block to increase effective depth without increasing parameter count, but the resulting T sequential recurrent-block calls per generated token substantially increase decoding latency. To address the issue, we introduce Wavefront Decoding (WFD), a training-free self-speculative decoding framework designed for looped language models. WFD exploits two properties of these architectures: intermediate recurrence outputs provide effective draft pr”

“Public data only (5 datasets, 9 tasks, frozen quiz of 2,199 eval items, paired McNemar tests). Overall accuracy: 4B 56.0% -> 71.6% (2h39 of training), 27B 68.8% -> 77.9%. The tuned 4B beats the base 27B (209 items gained, 149 lost, p = 0.002). Biggest gains on report extraction/classification (biomarker status 97.5% on test for the 4B). Weak spots: exact ICD-10 code (37.6% for the 27B), and MCQ accuracy collapses from val to test for all models, base included (cause unknown). What I would”

submitted by /u/dfbaggins [link] [comments]

“GitHub (llama.cpp fork) : https://github.com/sky-is-green/prism-ml-llama.cpp (Prebuilt binaries there to try instantly) Draft model : https://huggingface.co/SkyIsNotGreen/Scion-35B-A3B-mtp-drafter submitted by /u/pmttyji [link] [comments]”

“Thanks for making pylmcf. I just adopted it in my project kamui, a phase unwrapping library. I have a feature request: expose the node potentials (dual values) that LEMON's NetworkSimplex already computes. What's already there: LEMON provides them through NetworkSimplex::potential(node) and potentialMap(map) . pylmcf's solver keeps them in pi , and already uses them for warm restarts. So this should mostly be a matter of copying pi into a NumPy array. Why we need it: one of kamui's formulations”

“Nice..! When we train small model on same dataset it would never achieve such good benchmark score peasant like me always do 50m - 100m model always end up scrap..😂”

“In our #NeurIPS2026 paper “A Minimal Interpretable Architecture for Zero-Shot Reconstruction of Dynamical Systems (DS)” (preprint: https://arxiv.org/abs/2607.14937 ) we reduce a DS foundation model to the ingredients minimally necessary to faithfully reproduce long-term statistical and geometrical properties of DS : 1) A piecewise affine map with only a single (!!) parameter α that controls local con-/divergence rates, and … 2) … a context selector that chooses from the provided context signal t”

“I was curious about what classifiers could do. It turns out, a lot, sometimes even play Doom.Jeffy is currently a collection of simple classifiers that can do spam detection, intent routing, topic classification, sentiment analysis and even play DoomJeffy works locally on a regular computer, no GPU needed. It comes with 13 already-trained classifiers, and you can also use Jeffy to train your own classifiersAnd there’s a lot that can be built on top of this style of architectureCurious what tasks”

make sure to tell me what you think!

“🚀 The feature, motivation and pitch In current vLLM main, online weight updates use the regular model loading path for fused MoE weights: This path is correct because the model loader understands expert ownership and the runtime specific fused representation. However, every generator worker receives the complete logical expert weights before selecting the experts it owns. PR 51520 adds NCCL M2N weight transfer so trainer shards can be resharded directly into their destination layouts. Ordinary”

“Hi Yunhee, I have spent the last week trying to optimize methylbert for my project. I have tumor vs. normal EM Seq data (12 tumors vs 9 normal; 30x) derived from FFPE, which have pretty solid DMRs (generated through DMRichR and DSS). Whether I look at the areaStat or average DMR difference, the deltas are around 35 42 . The DMRs are also sufficiently large with adequate numbers of CpGs in them. I noticed that your example has extremely differential methylation, which is not something my dataset”

Llamacpp at least, or vLLM/SGlang or Exllama3 with TabbyAPI.

“Hey everyone, If you run local models via Ollama in production or personal projects, you've probably run into the hallucination problem: how do you know when a model is hallucinating without burning extra VRAM or waiting 5 seconds for a heavy judge model? The standard academic approach for this is Semantic Entropy (from an Oxford team's Nature paper last year). You sample $K$ responses at temperature 0.7, run them through a secondary NLI cross-encoder like DeBERTa to cluster equivalent m”

submitted by /u/rm-rf-rm [link] [comments]

“I'll be soon publishing our work on STM32 https://github.com/geastack here! I'd love to hear your thoughts and recommendations. Board in the video is STM32F746G-Discovery. submitted by /u/arbayi [link] [comments]”

“I do a lot of late-night coding and find typical Pomodoro timers either too bright or distracting with repetitive anime loops and loud ads. ​Decided to write an N-body particle system in Python using NumPy and Matplotlib that simulates a 2-arm spiral galaxy collapsing into an accretion core over 25 minutes. ​Rendered the full 25:00 session at 1080p with pitch-black OLED backgrounds and soft space ambient audio. ​Full 25-min session is here if anyone needs a focus sprint tonight: https://youtube.”

Maple is also the name of a loop-happy 20B-A1B ternary model.

“The collection includes kernels for more than 200 common ML operations, all of which can run entirely locally in your browser on WebGPU. We're also working to upstream these optimizations to Transformers.js, ONNX Runtime Web, LiteRT.js, and more! Kernels: https://huggingface.co/kernels?platform=webgpu Blog: https://huggingface.co/blog/webgpu-kernels submitted by /u/xenovatech [link] [comments]”

“Your current environment vLLM Version : 0.22.0 PyTorch version : 2.11.0+cu130 CUDA used to build PyTorch : 13.0 GPU : NVIDIA RTX PRO 6000 Blackwell Server Edition (compute capability 12.0) Nvidia driver version : 590.44.01 triton : 3.6.0 flashinfer python : 0.6.11.post2 transformers : 5.9.0 Python version : 3.12.3 OS : Linux x86 64 Model : Qwen3.6 35B A3B (qwen3 5 moe, hybrid linear attention MoE) 🐛 Describe the bug NVIDIA published ModelOpt NVFP4 checkpoints (e.g. nvidia/Qwen3.6 35B A3B NVFP4,”

“🚀 The feature, motivation and pitch 1. Motivation PR 36287 introduced TPA (tensor parallel size attention) to vLLM, and review feedback from Lucas Wilkinson — "the design shouldn't be attention specific" — pointed to a more general need: a per layer parallelism policy that lets layers declare what they are and have the framework resolve their plan. 2. The idea A single function, owned by the framework, that turns a layer's identity into its parallelism plan: Layers consult the resolver in init”

“Your current environment vLLM : v0.19.1+rocm721 (also reproduced on v0.19.2rc1.dev215+g32e45636e) PyTorch : 2.11.0 (built from source for gfx1201) ROCm : 7.2.1 RCCL : 2.27.7 (built from source with amdgpu targets gfx1201 ) OS : Fedora 43 (toolbox container, Ubuntu 24.04 base image) GPUs : 2x AMD Radeon R9700 (gfx1201), 16 GB VRAM each CPU : AMD Ryzen 9 9900X3D Also present : 1x iGPU gfx1036 (excluded via HIP VISIBLE DEVICES=0,1 ) 🐛 Describe the bug Summary Tensor Parallel = 2 across two AMD Rad”