Self-hosted speech-to-text
Whisper provides transcription, translation, and language detection. faster-whisper documents CTranslate2 inference and workflow features such as VAD, timestamps, batching, and quantization. whisper.cpp provides local CPU and GPU backends. Choose between them by testing your own audio.
Updated July 11, 2026
The documented engine options
Accuracy, resource use, and throughput depend on the audio, model, hardware, and runtime configuration. Test representative recordings before selecting an engine.
| Engine | Pick it for | Documented capabilities |
|---|---|---|
faster-whisper Whisper on CTranslate2 | Transcription workflows using VAD, timestamps, batching, and quantization | CTranslate2-based inference |
whisper.cpp Local Whisper inference | Local CPU or GPU backends, including microphone examples | Quantized converted models |
OpenAI Whisper Reference implementation | Transcription, translation, and language detection | Model variants with FFmpeg input requirements |
Treat transcripts as output to review
The Whisper model card documents limitations and intended use. Review transcripts before using them in decisions or records, particularly when the recording is difficult to hear.
faster-whisper documents voice activity detection. Test any preprocessing and transcript-review workflow on the recording conditions you expect.
Plan speaker labels separately
If your application needs speaker labels, evaluate that requirement as a distinct part of the pipeline. Test the complete workflow with the number of speakers and overlap patterns you expect.
Start with the reference implementation
The Whisper repository is a useful place to review the reference implementation, available models, and input requirements before choosing a deployment runtime.
OpenAI's reference implementation documents transcription, translation, language detection, model variants, and FFmpeg requirements.
Compare hosted and self-hosted deliberately
Compare current hosted pricing and service terms with your expected workload, hardware, operational effort, and data-handling requirements. Self-hosting can be appropriate when local processing is a requirement; it also requires you to operate and evaluate the transcription stack. If transcripts feed a local model, use the 8 GB VRAM guide as part of planning the overall hardware budget.
Questions people actually ask
Do I need a GPU for self-hosted transcription?
No. whisper.cpp documents local CPU and GPU backends, and faster-whisper documents CTranslate2-based inference. Choose hardware after testing representative audio.
What is the best setup for meeting transcription?
faster-whisper documents voice activity detection, timestamps, batching, and quantization. Evaluate any additional diarization component separately for your recordings.
Why is my transcript worse than the demos?
The Whisper model card documents limitations and intended use. Test an engine on recordings that match your microphones, languages, and noise conditions.
How well does it work outside English?
Whisper documents transcription, translation, and language detection. Build a small evaluation set for the languages and audio conditions you need.
Sources
The projects above are rows in the Awesome Open Source AI registry for open-source AI projects.
