AI Researchers, Sharing Open Source Work
These complaints highlight the experience of individual AI researchers releasing new tools and optimizations. They are driven by personal passion and a desire to contribute to the field, but lack a clear path for broader adoption or recognition. The focus is on technical innovation and sharing, not necessarily commercial success.
SOURCES (60)
“I initially saw CEA as just an efficiency improvement, but the more I read about it, the more it feels like an inference architecture leap. The encoder/decoder split has some pretty interesting implications for GPU pooling. Instead of treating every GPU the same, you could…”
“Hello this is my fifth small language model I've made and apart of my third series and it has been a lot of work but it payed off: **348M parameters, 22.7B tokens**, then fine-tuned into a math model that solves arithmetic by *showing the work* — column addition with carries, borrow chains, partial-product multiplication — rather than guessing at an answer. Last time I posted a 326M model trained on 10B tokens. This has about 2.3× the data, and the math side is WAY better than my previous tw”
“I wanted to see how far you could push CPU-only inference of a model way bigger than RAM, so I wrote an inference engine in C11 (MSVC + gcc, bit-exact across both) that streams a 744B MoE model (GLM-5.2, 202GB GGUF) off a USB SSD instead of loading it. The parts that actually mattered: Quantization: experts stored int4 group-64, router and correction biases kept at f32. Everywhere else lost too much precision, but this combination didn't. Streaming: experts are fetched per-token through a cu”
“(Quick note: English is not my native language, so I used AI to help translate my thoughts clearly. The code, the 38 unit tests, and the experiments on my RTX 5060 Ti are my own work.) Hey everyone, Lately I've been trying to train small models locally on an RTX 5060 Ti (8GB), and like many people, I kept running into frustrating CUDA OOMs whenever memory pressure fluctuated mid-run. Instead of just sticking to a static batch size and hoping the run survives, I started experimenting with a l”
“Blender has a full Python API built in via bpy, so you can just have your model write Python scripts and run them directly instead of needing MCP.”
“A few weeks ago, I saw this post where the author trained a SIREN MLP to implicitly memorize Bad Apple as a coordinate function: (t, y, x) to pixel. That got me curious about a slightly different formulation: instead of handing the network a timestamp t , could a small recurrent dynamical system (RNN-ish) learn the continuous temporal flow in latent space and generate the entire ~6,500-frame full resolution video autonomously from a single initial condition (h_0, c_0)? The code, weights, and ana”
“Hi everyone! We just released Rustuna (GitHub: https://github.com/optuna/rustuna/ ), a high-speed, memory-efficient implementation of Optuna built in Rust. Optuna-Compatible Design: Keeps the familiar API and concept of Optuna. Zero Python Dependencies: Mitigating the risk of supply chain attacks. Lower Memory Footprint: Optimized memory management natively in Rust. For details, please check out the following blog post. https://medium.com/optuna/announcing-rustuna-cc82a6815bf7 submitted by”
“Summary PyPTO currently provides only low level Tile primitives (similar to Triton's tl.load/tl.store/tl.dot ), with no high level operator library. We developed 6 neural network models (ViT Frame, ViT MultiView, ViT Temporal, BigTransformer MoE, Planner, VAE Encode) using the Tile level DSL and found that all common neural network operators must be hand composed from primitives . This is extremely labor intensive and error prone. We request: 1. A new gather primitive — the only missing primitiv”
“Godot has way less training data than web languages so models nail the syntax but can't track engine state across physics ticks. That's why AD7GD burned 23 hours and anemoDuck26 hits walls with every model.”
“When I first started working in scientific machine learning, I understood the physics much better than the coding. Every time I wanted to try a new physics-informed neural network problem, I had to start almost from scratch: changing the PDE, updating boundary conditions, modifying the architecture, tweaking the training schedule, debugging errors, and generating plots—all by hand. That frustration pushed me to build PINNStudio . It is a free, open-source no-code GUI designed to eliminate boiler”
“Failure 1 — CUDA silently fell back to CPU My training job launched on Vast.ai and ran to completion. Iteration time was 24 seconds instead of 0.4 seconds. CUDA had fallen back to CPU silently. PyTorch logged nothing. I had been billed for six hours of GPU compute while training on an unaccelerated CPU thread the entire time. Failure 2 — HF_HOME on ephemeral disk Every fresh pod re-downloaded base model weights to /root/.cache the ephemeral container disk wiped on pod shutdown. Same download, sa”
“I've been experimenting with Sliding Window Attention (SWA) as a way to reduce the KV-cache memory cost of long-context LLM inference. Instead of keeping the entire KV cache, the implementation keeps: a small number of attention sink tokens a bounded recent-token window a circular/ring-buffer KV cache streaming/chunked prefill normal autoregressive decoding I turned the experiment into a reusable project so you can test it with Hugging Face causal LLMs: 🔗 https://github.com/oraby8/SWA For e”
“From the original paper, this wasn't without losses, but I guess if you can use LoRa finetuning on a few mid-late layers with similar end-results, you could mitigate damage in that regard.”
“Hi I have a fork that caches symbolic factorization in between calls to PardisoSolver::PreProcess. I think this relies on the fact that my model has no contacts, but basically gives me a 1.85x speedup. I don't know how general this will be for other people. My change is here, it's AI written, and I've read it but not reviewed it as I would review production code. (And I'm not qualified to assess the linear algebra claims). Do you want me to actually review this and submit it as a real PR? Or not”
“With RAM price inflation, I wonder if FPGAs can be used to offload inference processing without keeping weights in RAM? The available RAM would be for activations, KV Cache, context but not static weights. Weights could be streamed from disk. This approach is not for tokens/second but throughput at a lower cost. Possibly better answers/kHh? I've started researching this, but wonder if others have considered/tried this?”
“I start with some informations gathered thorough endless posts reading on this sub and online: Inference and hardware optimization projects https://dwarfstar.sh/ - inference engine optimization proposal https://github.com/JustVugg/colibri - Treats VRAM + RAM + storage as one managed inference memory hierarchy, with expert streaming, caches and a strong focus on MoE. https://openfreedom.it/ - agentic harness proposal https://github.com/exo-explore/exo - Core source for heterogeneous/topology-awar”
“124B total parameters, 5.1B activated parameters, and a 256K context window submitted by /u/Bestlife73 [link] [comments]”
“Thanks for releasing ReDimNet2. The README lists the license as MIT , but there's no LICENSE file in the repo, so GitHub doesn't detect one and no copyright holder is named. Could you commit a standard MIT LICENSE file? Also, since the pretrained checkpoints are hosted as v1.0.0 Release assets rather than in the git tree, could you confirm whether the MIT grant covers the weights themselves as well as the code — including modification and redistribution? One note: the vb2+vox2 v0 checkpoints are”
“Hi, everyone, I’ve just released VoxGen, a lightweight native inference engine for VoxCPM2, written in Rust and using Vulkan compute instead of Python/PyTorch/CUDA. Why VoxGen? The main reason I started the project was because I needed a decent local text-to-speech solution. I therefore saw VoxCPM 2 as a reasonable solution. However, most frameworks are NVIDIA-first, and VoxCPM 2 is no exception; as a result, my card was severely stuttering, and my GPU was always spiking. Also, having Python and”
“I'm a biomedical-to-CS student, and I wanted to see how far a genuinely explainable (Use of heatmapping) medical imaging prototype could get on near-zero infrastructure. It's a DenseNet-121 trained on BTXRD (3,746 radiographs), three classes: normal, benign, malignant. Malignant is only 9.1% of the data, so a model predicting "not malignant" scores 90.9% accuracy while missing every cancer. Focal loss with inverse-frequency alpha, thresholds picked on validation only, probabili”
“I've spent the last month building a local C++ machine learning library called Deepity to test alternative credit assignment algorithms; specifically Predictive Coding Networks (PCNs). While PCNs are fascinating for biological plausibility and continual learning, naive implementations are painfully slow. By implementing recent research ( Accelerated PCNs via Direct Kolen-Pollack Feedback Alignment ) and utilizing algorithmic caching to bypass redundant forward projections during the inferenc”
“YOLO26 ships a depth-estimation model — dense, full-resolution, per-pixel regression, a task architecturally much closer to image restoration than to detection. I wanted to know whether the backbone+neck weights it learns through depth training transfer to a different dense-regression task (deraining), compared with training the same architecture from scratch. The deraining model that came out of it is a useful byproduct, but the transfer result is the part I think is worth discussing. What I in”
“I'm proposing TransformerModule: the transformer parallel of the LSTMModule, built on the same four aspects. Collection runs one step at a time against a fixed shape KV cache carried per env; training processes full windows under a block diagonal causal mask so attention never crosses episode boundaries. The load bearing invariant, enforced by a parity test, is that the two paths produce identical outputs. One deliberate rule throughout: the KV cache is inference state, not experience. It lives”
“You should post this over in r/moderndotnet this sort of project is sorely lacking in the .net world.”
“I’ve been experimenting with MiniMax H3 and finally have video generation working in TensorSharp. TensorSharp started primarily as a local GGUF/LLM inference engine, so getting a video-generation pipeline working in the same runtime has been an interesting change of direction. The attached demo is image-to-video: an image is provided as context, followed by a prompt describing the motion/scene, and H3 generates the resulting video locally through TensorSharp. What interests me most here isn’t re”
“Your current environment <details <summary The output of <code python collect env.py</code </summary </details 🐛 Describe the bug Description llm compressor can now quantize embeddings in an effective and nearly transparent way that can be served by vLLM, as in: https://github.com/vllm project/llm compressor/tree/main/examples/quantization embedding This works well for some model types, but many others fail. Reproduction Generate a model with quantized embeddings of one of the affected models (”
“Hi everyone, I’ve been experimenting with alternative language-model architectures for a while, and I recently finished the first complete pretraining run of a new architecture I’m calling WarpState . This is still an experimental proof of concept, not a claim that it beats Transformers or existing state-space models. The model has 150.13M parameters and was trained from scratch on roughly 300 million English tokens from Ultra-FineWeb L2 . The full run completed successfully: Parameters: 150.13M”
“learnt about vector space , fields , and applications whatever i could then went on with learning python libraries like numpy , scikit learn , pandas , also had a bit of knowledge about tensorflow and how to use pytorch but now i stand so clueless when i try to apply my knowledge in my first project. So idk why but i decided somehow that i wanna make a project in "marine reasoning " and i feel clueless about is it just supervised learning reinforcement learning or mix of both and how m”
“Hey everyone! I’m excited to announce the release of py-evoFE (v0.3.0) — an open-source Python library that uses genetic algorithms to automatically discover, combine, and optimize feature transformations for tabular datasets. GitHub: https://github.com/tanopereira/py-evoFE PyPI: pip install py-evoFE License: MIT The Problem It Solves Feature engineering is still where most tabular ML competitions and production models are won or lost. While GBDTs like LightGBM and XGBoost excel on raw tabular d”
“I'm sorry if this is a bit unrelated to the subreddit but I thought since so many hobbyists and professionals gather here and tinkerers with frameworks and everything, I thought I might share some interesting and fun findings, well the definition of fun might not be the same. Anyways, I was investigating whether the resumption of a training run was correct before I launched some experiments and I went into a deep rabbit hole (again...). This is not necessarily that much helpful except in ver”
“lots of interesting merges this week :) submitted by /u/jacek2023 [link] [comments]”
“It would be neat to do "expert-aware" training runs where some experts are fairly narrow and with a known doman specificity, so depending on workloads you would be able to put some in vram, some in ram, and turn off others to save on both.”
“> Will be interesting to see if inference chips are here to stayTo me, the efficiency gains of inference chips are so significant that they are certainly here to stay — barring a revolution of sorts that leads to a world devoid of AI as we know it.”
“I've been working on Python bindings for Contrek, the C++ raster-to-vector library I've been developing, and they're now in a usable state. The idea was to keep the common case simple, while still exposing the lower-level features of the C++ engine when needed. For example, this processes a 10240x10240 PNG using 8 threads and 8 tiles, extracting everything that isn't white: import contrek color = contrek.rgb_to_target_color(255, 255, 255) # white result = contrek.contour( image_p”
“TL;DR - A ~4M example, ~3B token CoT dataset designed around helping small models think more concisely, accurately and reliably. Hi all! For the past few months I have been working on a dataset designed around improving small model performance through a structured framework (or Scaffold) for thinking. When using small models (Under 5B parameters), I noticed freeform CoT does not really add much in terms of capability, and usually results in more confusing, poorly structured and inaccurate respon”
“Very cool work and hoping you push it further. My own attempts at from scratch training of looping models have been pretty disappointing (negligible benchmark gains and losses on some) at the sizes I can pretrain easily at home (50M to 200M) but a decent bit of research does suggest that it only becomes particularly beneficial once you reach those much larger sizes. Looking forward to seeing how yours turns out and if the additional training makes for a clear win over the base model.”
“I was fascinated by Nanbeige's outstanding performance for its size, so I started digging into how much a model can improve its own representation just by looping over itself (for fun). My prototype was a Qwen3-0.6B with a full dual loop in the middle layers, inspired by the Nanbeige 4.2 architecture. Digging further, I found that the Nanbeige team has a paper describing their 4.5 architecture, which uses a triple loop in the middle layers — that made sense to me, so I tried it. Lordnyx/qwen”
“Mandelascii is a fast, multithreaded ascii art mandelbrot zoomer in common lisp. ultralisp: https://ultralisp.org/projects/bonkzwonil/mandelascii github: https://github.com/bonkzwonil/mandelascii/ license: BSD 3”
“Hi, I've been exploring LibreYOLO, and so far I find it very impressive and easy to use. I also appreciate that it is open source and license free. During my testing, I successfully trained a YOLOX model using LibreYOLO and then tried to export the trained model to TFLite with INT8 quantization for post training quantization. However, I found that TFLite INT8 quantization is currently not supported. I received the following error: Command Used bash libreyolo export \ model /home/gokul/devlopment”
“I've been doing some research on this question: At inference time a large part of a model's working memory lives in the KV cache, plus whatever external memory the harness bolts on. I've been poking at the storage-and-retrieval side of this, treating that cache as an index, and what stands out is that it isn't a flat list. It's a structured set of vectors with a navigable geometry, since the keys carry the model's learned sense of what relates to what. Because that geomet”
“I trained three LLMs from scratch in raw PyTorch then post-trained each one with SFT and then GRPO. Same process every time: same synthetic arithmetic curriculum, same reward function, same hyperparameters, same KL coefficient. Pre-training went as expected, the val loss went down as the model got more modern techniques (V1 to V2) and bigger (V3 being the biggest). However, GRPO hurt both V2 and V3 and I'm not sure why. Setup V1 V2 V3 Params 353M 316M 672M d_model / layers 1024 / 24 1024 / 2”
“ornith lab dropped new ornith 1.5 today, a 9B dense with vision and a 35B-A3B MoE, both MIT, trained on a loop that generates its own tasks. in addition there was giant 397b model, but we didn't quantize it (but if you want to try - we will do it) we made our AD (Atomic Dynamic) quants for both, 9B (14 builds) and 35B-A3B (13 builds), and measured them against stock llama.cpp quants (on the same imatrix) their mean KLD and top-1 against our own BF16 conversion Ornith-1.5-9B file size mean KL”
“I’ve been looking at a fairly basic question in weight-space learning that I don’t think gets separated cleanly enough: Why does reading semantics directly from neural network weights work pretty well when the networks share an initialization, but collapse when the networks are fitted independently? The usual explanation is parameter symmetry. Permute hidden units, flip equivalent signs, etc., and two parameter vectors can represent the same function while looking completely different to a downs”
“None has undergone post-training, giving researchers flexible starting points for continued pre-training, fine-tuning, and further research. Two key highlights: - They use WSM to replace LR decay with weighted checkpoint merging, making the training process better suited for continual pre-training while enabling offline exploration of different LR decay strategies. - With one shared training recipe, the community can validate strategies on tiny-base, then scale them to flash-base. #1- Ling-3.0-t”
“Aloha! 🌺Introducing Ornith-1.5, a family of open-source LLMs spanning 9B Dense, 35B MoE, and 397B MoE, trained with self-improving strategies. It achieves state-of-the-art performance among open-source models of comparable size and delivers performance comparable to Claude Opus 4.8 across reasoning, agentic, and coding tasks: ✅Terminal-Bench 2.1 (86.1) ✅SWE-Bench (86 on verified, 65.1 on pro, 79.6 on Multilingual) ✅DeepSWE (56) ✅HLE (44.6) ✅ClawEval (81.4) ✅Tool Decathlon (71.2) https://hugging”
“Hey everyone! I'm a data scientist and I've been pretty fascinated by reinforcement learning for a while. A few days ago, my friends showed me Pokelike . Obviously, my first thought was: "wait, I could build an harness around this and make bots play it." So that's what I did. My original idea was to build an environment for training reinforcement learning agents. That's already working, and I've built a few bots to test it out. They are still pretty terrible though”
“I recently bought a Shrike lite which has got 264KB of SRAM. I decided to train an image generation model that generates 32*32 pixel images. The microcontroller also has an FPGA onboard which I used to create two parallel INT8 MAC engines with 16 bit accumulation to speed up calculations, however the system soon hit a memory wall due to the high number of I/O operations, this meant that the system with parallel MAC engines ran slower than the MCU only model (~220 seconds per image vs ~70 seconds”
“Making things understandable is part of intelligence as much as producing the initial artifact is. Even if the proof checks out in Lean (or the code runs and passes QA) if it's a mess, it will be hard to use it to do anything further.This does not only matter doing cutting-edge mathematics. This, about the 'digested' version versus the original, should feel familiar to some folks here:> This formalization is more streamlined than the original formalization (it has about 15,000 lines of code, com”
“Free tier is very generous with its virtual environment. I guess if you are smart you can use it for something really cool. Anybody found cool use cases? Dataset: 70,000 real MNIST images Training: 60,000 images Evaluation: 10,000 independent test images Model: CNN with 421,642 parameters Best measured test accuracy: 98.79% Progress Loss Training accuracy Time 21% 0.0555 98.33% 8.4s 42% 0.0558 98.28% 15.3s 64% 0.0524 98.41% 21.9s 85% 0.0514 98.45% 28.5s 100% 0.0517 98.45% 33.3s submitted b”
“No need to quant the KV. The V vector is literally 0MB as it get absorbed by the latent K vector and restored during inference step. This is like pushing your car on a highway at 75 to save fuel.”
“🚀 The feature, motivation and pitch Sub issue for 25179 to track custom pytorch helion kernels for MultiHead Latent Attention + fp8 quantization. Helion guarantees to compile down to a single triton kernel, and with helion autotuner a optimal configuration can be found for the kernel. For complex and fused kernels such as MLA + dynamic fp8 quant, using pytorch native module of helion. kernel 1 : computes partial attention output over kv cache of different heads in different CTAs. mimics forward”
“🚀 The feature, motivation and pitch hi @hongxiayang currently CUDA vLLM has easy to understand PD disagg recipes for popular models. can u add this for ROCm vLLM too (blocker is acutally get ROCm vLLM disagg to work with vllm router https://github.com/vllm project/vllm/issues/38687 https://github.com/vllm project/vllm/issues/38692) https://recipes.vllm.ai/moonshotai/Kimi K2.5?strategy=pd cluster&variant=nvfp4 +viz @powderluv @chunfangamd @andyluo7 prefill command <img width="930" height="680" a”
“<html <head </head <body <h2 Your current environment</h2 <p Reproducible on L4 (NVIDIA, compute capability 8.9). The broader class of issue is expected to affect any SM<90 GPU (Ampere SM80, Ada Lovelace SM89, etc.) when running with batch invariance under <code torch.compile</code and/or CUDA graphs.</p <h2 Describe the bug</h2 <p On SM<90 GPUs, <code VLLM BATCH INVARIANT=1</code does not produce batch invariant outputs when combined with either <code torch.compile</code or CUDA graphs (a”
“🚀 The feature, motivation and pitch get conv copy spec in vllm/model executor/layers/mamba/mamba utils.py assumes the SD layout (num blocks, state len, dim) where slicing state[block, offset:] along state len yields a contiguous view because dim is the innermost (contiguous) axis. With the DS layout (num blocks, dim, state len) , slicing along the last dim ( state len ) when num accepted tokens 1 (i.e. offset 0 ) produces a non contiguous view because dim is strided by state len : MambaCopySpec”
“🚀 The feature, motivation and pitch Description Current LLM inference suffers from 15 30% compute waste (Parasitic Load) due to logical drift and physical reality violations. We propose integrating a monitoring layer support (SDAG Protocol) for Atomic Rewind & Correct (ARC). Motivation In our simulations, by utilizing KV Cache Rollback at the moment of violation detection (e.g., physical constraints failure), we achieved up to 85% compute cycles saved. Proposed Solution Implement/Expose low lat”
“🚀 The feature, motivation and pitch Context RFC 8913 proposes decoupling checkpoint formats from kernel integrations by standardizing how weights are represented at runtime and allowing kernel specific repacking/preprocessing as a separate concern. This RFC scopes that same idea specifically to quantized linear methods (e.g., GPTQ/AWQ/FP8/FP4 linear integrations) where kernel selection is currently constrained by checkpoint packing/layout assumptions. Problem Many quantized linear “methods” im”
“🚀 The feature, motivation and pitch To avoid redundant work in MoE models in the TP case, sequence parallelism was added to the Deepseek model definition in 24134 and expanded to other models in 24982. However, to avoid performing surgery on the linear layer, the current approach performs more communication than necessary. With a torch.compile custom pass, we can rewrite the graph to remove the redundant computation. More details Before the SP optimization, the ops in the model were: With seque”