Devs Hunting for Good Local STT/TTS Models
105 posts across 15 communities in the last 30 days; 35% from r/LocalLLaMA.
“Fair, the weights aren't NVIDIA-specific, the code is. But it's a TTS model, not a text LLM, so you can't just load it in llama.cpp or Ollama. You need the MoE backbone, the audio tower and the codec running, and that's the part I ported to ROCm. If that's not interesting to…”
“I submitted a ticket to OpenAI, hope that will help. I have found if you just let the text generate and then hit the three dots and select "Read Aloud" the voice you selected, in Standard Voice mode, will read back correctly. It's just broken for the actual two-way verbal…”
“This is hilarious, but an all-too-real problem with running continuous Whisper pipelines on open room mics. Whisper is trained on subtitle data, so when ambient noise/AC hum slips past naive energy VAD, the decoder's cross-attention forces it to transcribe something — which…”
Developers and language learners are struggling to find optimal, locally-run speech-to-text and text-to-speech models. They face challenges with model size, accuracy, features (like diarization), language support, and licensing restrictions, often requiring significant experimentation and fine-tuning. The desire for self-hosting and avoiding cloud APIs is a key driver.
SOURCES (60)
“Im tired of english models only. There are plenty of it, no need of another one. Bring multi language to small ASR models! [re: Tyger: open-weights streaming ASR + full-duplex turn-control in one model]”
“Looking for: creators, teachers, or anyone who speaks an Indian language (Hindi, Tamil, Bengali, Telugu, Marathi, Gujarati, Kannada, Malayalam, Punjabi and more). What it is: BhaShaVox dubs video/audio into 125+ languages, clones your own voice with consent, and translates documents and images. What I'm asking: try one dub (5 minutes), then tell me 1. How natural it sounds in your language 2. Where the flow confused you 3. What you'd use it for Free 14-day trial: https://www.bhashavox.co”
“I was looking for small TTS model which is steerable but couldn't find any, specially something I can adjust contextually by providing controls from LLM during a conversation. So I built one. What it is: 9.6M-param acoustic model (attention + bidirectional Mamba-3, FastPitch-style) + Vocos vocoder = 23M total 12 controls , each a z-score relative to the chosen voice’s own neutral speech: rate, loudness, median pitch, pitch range, final rise/fall, pauses, spectral tilt, breathiness-related me”
“Whenever we have an action using tts.cloud_say the speaker starts up but nothing is spoken. Anyone know what could be the issue or how to debug it? As far as I know there is no integration to restart for cloud tts?”
Feature request: Personal voice recognition
“It's an extremely lightweight speech recognition model. 949K parameters using a weight-shared Conformer-CTC architecture Just 1.06 MB for INT8 weights Around 436× real-time on a single CPU thread 11.43% WER on LibriSpeech test-clean, using greedy decoding with no external language model Apache 2.0 licensed Any feedback, question or criticism is welcome! submitted by /u/Tall_Abrocoma_3533 [link] [comments]”
“This model has good voice quality [re: Fish Audio Drama 3 says it needs a 48 GB NVIDIA card. My 16 GB Radeon disagrees.]”
“Fair, the weights aren't NVIDIA-specific, the code is. But it's a TTS model, not a text LLM, so you can't just load it in llama.cpp or Ollama. You need the MoE backbone, the audio tower and the codec running, and that's the part I ported to ROCm. If that's not interesting to you, that's fine too :3 [re: Fish Audio Drama 3 says it needs a 48 GB NVIDIA card. My 16 GB Radeon disagrees.]”
“https://preview.redd.it/w817kfat81wh1.png?width=906&format=png&auto=webp&s=05ae99d01ae7f8392dd4b02cfe4012e28f189e98 https://preview.redd.it/4gbgby9x81wh1.png?width=828&format=png&auto=webp&s=c76610fa24ed4f964ca3a004c999a2fbd18782fc Added two new things - Spanish premium voices and also made all premium voices available to everyone with no usage caps. Adding Spanish has been a goal of mine since I'm currently learning and wanted a way to get more consistent input with”
“Well, thanks for the clarification. I actually don't know how to identify a "direct speech to text" application, I Always discover that all local application are working with a delayed processing time, kind of. [re: Handy - a simple, open-source offline speech-to-text app written in Rust using whisper.cpp]”
“some girl on the openai forums just had the correct voices come back on ios [re: What’s happening with standard voice?]”
Why? [re: Client-side LLM Demo: PDF to AudioBook Generator]
“Hi r/LocalLLaMA , we're releasing Tyger, a small model that handles full-duplex turn-control for voice agents. What it is Base: NVIDIA Nemotron 3.5 ASR Streaming 0.6B + 14M causal GRU on top. Every 80 ms, with no lookahead, it picks one of three states: - answer : the user finished their turn - stop : the user cut in, so the agent stops talking - chime in : the agent says “mm-hm”, “oh, really” and keeps listening It hears both the mic and the agent's own playback, so the agent doesn'”
“I submitted a ticket to OpenAI, hope that will help. I have found if you just let the text generate and then hit the three dots and select "Read Aloud" the voice you selected, in Standard Voice mode, will read back correctly. It's just broken for the actual two-way verbal conversation [re: Voice update messed up anyone else's Voice mode, forcing the default instead of the chosen voice?]”
“I was expecting to hear audio in your recording. Is there any or something is wrong with my app? 👀 [re: Drama3 in audio.cpp: Up to 5× faster than real time at full speed, or 2× real-time speed with < 8 GB of VRAM on an RTX 5090!]”
“Yeah plus multi speaker diarization for over 4 people [re: New TTS king in town: Fish's Drama 3 35B3A!]”
“Any update? [re: Self-hosting Open-source AI Model for Accurate Pronunciation Scoring in Language Learning – Recommendations?]”
“Gpt Live [re: Pipeline vs speech to speech voice agent on an actual phone call (latency + cost)]”
“What model did you use for the speech to speech? [re: Pipeline vs speech to speech voice agent on an actual phone call (latency + cost)]”
“one is stt + llm + tts, the other is speech to speech. real phone call, i just let them talk for like 2 min the pipeline one costs around 3 cents a min vs almost 6 for the other speech to speech sounds way more human tho. mine kept telling the guy not to ride his bike lmao submitted by /u/Numerous-You3558 [link] [comments]”
“I built a benchmark around that failure. VoxEntity-EU scores whether a speech system captures a dictated name, phone number, email, date, postal address or reference code exactly, in German and French. One wrong character fails the slot. The two ASR systems are statistically indistinguishable (difference −2.9 pp [−7.1, +1.3]). It is published as a null result, because that is what it is. Dataset (audio included): https://huggingface.co/datasets/Gjusev/voxentity-eu Listen to the failures: https:/”
“Open-weights ASR + full-duplex turn control in one model. It hears when you finish, when you interrupt, and when you're just saying "mm-hm". • 250 ms median endpoint latency, 80 ms frames, no lookahead • Full voice agent tyger-s2s: TYGER + Gemma 4 E4B + VoxCPM2, one 32 GB GPU, up to 4 conversations • English only for now 🤗 https://huggingface.co/nineninesix/tyger 💻 https://github.com/nineninesix-ai/tyger-s2s submitted by /u/KokaOP [link] [comments]”
“Got the HF space to work finally, but the voice cloning is pretty bad. It's more like a cousin of the reference voice rather than an exact clone. [re: New TTS king in town: Fish's Drama 3 35B3A!]”
“Hello, Thanks for your effort to bring us this amazing project! I was wondering if you are planning to add hardware volume control to the DSPi firmware. I have been using a rotary encoder on a regular pi zero to do exact this for a while (DAC PCM5122/42), the code and documentation is basically there, see https://github.com/iqaudio/tools https://cdn.sparkfun.com/assets/e/6/5/a/9/20201209 IQaudIO v32.pdf (p.21 and p.23) The problem however is that the above implementation is based on wiring pi &”
“Even MOSS-TTS 1.5 beats both of those, IMHO. I know people like to talk about Qwen and Omnivoice and etc, but for pure voice cloning, MOSS-TTS is one of those engines everyone just completely ignored unless they actually tested them all side-by-side. The only part that sucks about it is small/short generations don't come out sounding good at all, but for like 95% of use cases, MOSS-TTS has been the best and everyone just kind of talks about all the others for some reason. I blame the audio b”
“Problem Currently, freellmapi relies entirely on the cloud delivered model catalog and only supports LLM (chat) models. The key management UI is also LLM centric. This creates two major limitations: Custom models cannot be used. If a provider offers a model that is not in the official cloud catalog (e.g., a self hosted embedding model, a custom image generation endpoint, or a niche audio model), there is no way to add it manually. The router simply ignores it. Multi modal providers are underutil”
“This is hilarious, but an all-too-real problem with running continuous Whisper pipelines on open room mics. Whisper is trained on subtitle data, so when ambient noise/AC hum slips past naive energy VAD, the decoder's cross-attention forces it to transcribe something — which usually ends up as hallucinated subtitle tags ( [Music] , [Applause] , Thank you for watching ) or repetitive ghost phrases that trigger the LLM. In J.A.R.V.I.S Plus (our open-source voice assistant: https://github.com/th”
“Sonia is now opensource https://github.com/grigio/sonia-assistant [re: Local Voice Assistant based on Qwen3.5 4B with skills/toolcalling. No dedicated GPU. Local STT+TTS]”
“there are STT models like whisper though ? they run fairly cheaply and have real time transcribing. there are samsung features for live translation on both the regular phone and the buds, i think even airpods have this feature on apple ecosystem, idk about other stuff tho. [re: Is there a way to use ChatGPT to live-translate spoken words?]”
“I'm hoping for some advice. I've been using ChatGPT to practive speaking French but about 1-2 weeks ago the voice transcription quality tanked and is essentially unusable. To test it I did a 5 minute French presentation I had already done using ChatGPT a month ago - previously it transcribed well, fixed my tenses, taught me new vocab when I switched to English, now it hallucinates garbage with little correlation to what I actually say. I feel the English transcription has slipped too but”
“需求描述 (Feature Description) 建议在插件中增加对阿里云百炼最新千问语音大模型 qwen audio 3.0 tts plus 与 qwen audio 3.0 tts flash 的配置与支持。 背景与优势 (Background & Motivation) 阿里云百炼现已推出新一代千问音频合成模型，相比旧版模型在表现力和性价比上均有明显提升： 音质与延迟表现 ： qwen audio 3.0 tts plus ：韵律与情感表达更加自然细腻，声音保真度高，适合对朗读品质要求高的场景。 qwen audio 3.0 tts flash ：响应速度更快、首包延迟更低，在划词翻译和实时朗读时体验更加跟手顺畅。 极高的性价比优势 ： 最新模型的调用成本与旧版本基本持平，官方定价 输出仅比之前模型多了约 0.2 元 / 万字符 。几乎以相同的成本就能换来明显升级的拟真度和延迟表现，对用户日常划词翻译朗读非常友好。 建议修改点 (Suggested Changes) 1. 模型配置项（Model List） ： 在模型下拉选择或配置中增加 qwen audio 3.0 tt”
Fair question! Plain transcription is already solved, so that's not what I'm competing on, and just to mention Speechy already has everything that other dictation tools have (filters removing pauses, snippets, dictionary,…
“Jokes aside, genuine question: between Apple's native macOS dictation and dozens of free/open-source Whisper tools that run locally, why should someone pick Speechy? What's your core differentiator? [re: My Mac app has 0 paying users, 0 revenue and 99.99% uptime after 20 months 🎉]”
“I just wanted to share a program I vibe coded. I couldn't find a free program that did exactly what I wanted so I vibe coded it. It uses the Whisper model to detect profanity, or really any words you configure in the word list, and mutes them and then produces a new copy of the movie that has been cleaned. The video stream is untouched and only the audio is edited. I am not a coder and it is coded 100% with Claude Code. I know how some feel about AI coding and I don't like AI coded slop”
“We’re building a phone voice agent for Algeria. Current models don’t handle Algerian Darija well, so we want to fine-tune our own. We have a lot of real call recordings, but we’ve never trained a speech model before. Looking for guidance on 4 things: Which model to pick? For speech-to-text and text-to-speech, with a commercial license. How to train it? Fine-tune vs LoRA, what hardware, any good guides or notebooks. What data do we need? How many hours, what format, do transcripts need to be perf”
“Is your feature request related to a problem? Please describe. mistralai =3.2.0 provides realtime transcription ( client.audio.realtime.transcribe stream , voxtral mini transcribe realtime 2602 ), but ag2.live cannot use it: STTConfig.transcribe(voice, context) str takes a finished clip, so there is no seam for incremental audio in and incremental text out. Describe the solution you'd like First a small design decision, then the implementation. 1. Protocol: widen STTConfig , or add a sibling str”
“Is your feature request related to a problem? Please describe. ag2.live ships speech to text configs for OpenAI and ElevenLabs, but not for Mistral. Mistral is already a first class chat provider ( mistral extra, mistralai =3.2.0 ), and the same SDK exposes an offline transcription API ( voxtral mini latest ), so a Mistral based voice agent currently has to bring in another vendor for STT. Describe the solution you'd like A MistralTranscriber implementing STTConfig , modelled on the ElevenLabs o”
“Hi guys, I’m back with a major update to my app, EasyWhisperUI 3.0! Whisper is an open-source AI speech recognition model developed by OpenAI that converts audio and video into text, but setting it up locally can be complicated for the average user. My app aims to be as close as a one click setup as possible. This update brings countless changes and improvements, including a completely revamped interface, easier setup, new features, and tons of bug fixes. EasyWhisperUI is completely local, free,”
“Yeah, its pretty amazing for its size. I run it as a custom android app on my old pixel phone in basic realtime. Quality is fine to dictate messages or todolists. [re: Whistle: speech to text in 16.9 MB]”
“It really depends. I can see such a model paired with cactus' own needle model for ultra light weight speech to tool calls, then something like kokoro 82M for TTS. Like your own personal "hey Google" that you can run on a smartwatch or any phone that can do many of the things google/Apple assistants do, but fully private and without needing internet [re: Whistle: speech to text in 16.9 MB]”
“STT: Voxtral TTS: Pocket Longer video: https://www.youtube.com/watch?v=1WmvMO-f-24 submitted by /u/Leading_Yogurt7025 [link] [comments]”
“Basically I am fed up of Google Transcribe (it's genuinely terrible). Is there a way to use Whisper Tiny instead? I know there is very little chance but I thought since Android is open source perhaps... submitted by /u/Potential_Top_4669 [link] [comments]”
“Not license wise [re: Fully local conversational AI: Whisper + Hermes 8B + Kokoro, zero cloud, running inside a plush toy]”
“I have been playing with a lot of $20 devices with no keyboards or screens. I wanted to use speech to interact with them, but without it having to run through a cloud service or use a fixed command vocabulary.I worked with Claude to implement high performance 4bit int kernels for the ESP32S3 and P4 and quantize some larger STT and TTS models. Surprisingly, it's quite usable on these tiny boards.Also includes a finetuned model that can better cope with my diesel drones, hacking coughs, and mumbli”
“Voice AI industry still uses transcripts to determine the emotions of the user.So when me and my friend were discussing about this we got the known limitations of transcriptions is that it doesn't capture the user's emotion from audio. Like, "I'm frustrated with your service" reads the same whether calm or angry.Then I thought, why can't the decision model be tested for this to get quicker decisions on the go, knowing the limitation that it can't accept the audio file directly? So I thought of c”
“> for STT, it's literally you saying it, with a model transcribing, and then another model correcting a little, and you also have control over editing it. I don't think any of the arguments on "etiquette re: sharing AI output" apply here.Yes, for doing extremely mild edits (like just removing uhh's etc.) this might be truebut I sometimes feel as if writing can allow me to shift paragraphs, so if I am writing para 1, para 2, I can shift back to para 1 and write another sentence in it and edit som”
“When you are running the model locally and trying to do other tasks at the same time, Whisper Large feels too clunky on normal hardware.I like Parakeet because it's good enough, relatively light, and fairly fast. I'm using this for things like meeting transcription and dictation. Since I'm sending most of the text to an LLM to clean up afterwards, it works well enough.I'm hoping for something the size of Parakeet (or smaller) but better quality. It feels like with all of the advances in makin”
“This is because, in the case of a restricted set of possibilities, voice recognition circa 2000 was actually very very good.If you can do something with an extremely limited vocab, voice recognition was fine using off the shelf microchips in the 70s, where you wired in a microphone connection and had discrete pins for output actions.LLMs are basically only useful for utterly free form transcription, but that doesn't actually help you turn that into tasks to perform and parameters for those tasks”
“It’s really domain dependent I think. If you are doing anything conversational interfacing with less AI-familiar users, latency matters a lot.If you’re feeding the results into a very smart LLM, it will figure out what you meant (but crucially ONLY if you warn it or tell it to do so, in some cases!). If you’re writing code directly or creating something for public consumption, you can’t tolerate mistakes. If you’re taking notes for yourself you just want it to work cheaply.If you are ok with the”
“I want to share a TTS server with an OpenAI-compatible API that generates speech really fast (about 0.3 seconds for a sentence on an RTX 3080) and can clone a voice from a short reference clip. I’ve optimized the server so generation starts quickly, and it processes long text in sequence, paragraph by paragraph. I use it to turn school books into audiobooks in my own voice, so I can listen to them while driving. Out of the box, it’s already tuned for the best settings, but you can change them ho”
“Which model would you like us to optimize next? GitHub: https://github.com/0xShug0/audio.cpp [re: [audio.cpp] Recent updates you might have missed: Higgs Audio TTS use 48% less VRAM (< 6GB), HTDemucs 2.2× faster, PocketTTS 2.2× faster on CPU, and WebUI generation history feature]”
“Hi all, a bunch of performance improvements have been landed in audio.cpp. The biggest highlight is Higgs Audio TTS, which now runs with around 6 GB VRAM , a 48% reduction in peak memory usage compared to the previous implementation. Thanks to https://github.com/mirek190 We also made some models significantly faster, especially HTDemucs on GPU and PocketTTS on CPU. No compromises in parity and correctness. Here's a summary of the improvements: Model Peak memory reduction Speedup Higgs Audio”
“Github: https://github.com/kadirnar/drifting-tts Demo: https://huggingface.co/spaces/Vyvo/drifting-tts-tr-demo submitted by /u/kadir_nar [link] [comments]”
“Before Submitting [x] I searched open and closed issues and discussions for an existing report. [x] I reproduced this bug on the latest release AND on the current dev branch, right before submitting this report. I did not just check an old version or rely on a check from days ago. [x] I understand that maintainers want a well written issue before any code pull request. [x] This is not a security vulnerability. Installation Method Git Clone Open WebUI Version dev at 0ecf244e1 Operating System Deb”