I need to transcribe audio frequently — voice memos, meeting recordings, podcast snippets. Sending audio to OpenAI Whisper API means paying per minute and trusting a third party with potentially sensitive content. So I built an MCP server that runs 100% locally using faster-whisper (CTranslate2) with NVIDIA GPU acceleration.
whisper-stt-mcp is an MCP server with 4 tools for audio transcription. It integrates seamlessly with Hermes Agent, Claude Desktop, Cursor, or any MCP-compatible client.
Architecture
Audio file (mp3/wav/m4a/ogg/flac/opus/webm)
│
ffmpeg conversion ← auto format handling
│
faster-whisper (GPU) ← CTranslate2 with INT8 quantization
│
VAD (silero-vad) ← filters silence, improves accuracy
│
JSON output ← text + word-level timestampsYour audio is processed entirely on your machine. No data ever leaves.
Key Features
4 MCP Tools
| Tool | Description |
|---|---|
transcribe | Transcribe a single audio file with full options |
list_models | List available and downloaded Whisper models |
download_model | Download a Whisper model (tiny/base/small/medium/large-v3) |
transcribe_batch | Transcribe an entire directory of audio files |
GPU Acceleration
NVIDIA CUDA with INT8 quantization means even large-v3 runs in ~2.5 GB VRAM:
| Model | VRAM | Speed (RTX 3050) | Quality |
|---|---|---|---|
| tiny | ~0.5 GB | 20x real-time | Low |
| base | ~0.8 GB | 15x real-time | Fair |
| small | ~1.5 GB | 10x real-time | Good |
| medium | ~2.0 GB | 6x real-time | Recommended |
| large-v3 (INT8) | ~2.5 GB | 4x real-time | Best |
Smart Features
- Voice Activity Detection — silero-vad filters silence before transcription
- Auto language detection — 99+ languages supported
- Word-level timestamps — exact timing for every word
- Auto format conversion — handles mp3, wav, m4a, ogg, flac, opus, webm via ffmpeg
- Batch processing — transcribe entire directories with one call
Quick Start
git clone https://github.com/fxckcode/whisper-stt-mcp.git
cd whisper-stt-mcp
uv venv
uv pip install -e .With Hermes Agent
Add to ~/.hermes/config.yaml:
mcp_servers:
whisper-stt:
command: "/path/to/whisper-stt-mcp/.venv/bin/python"
args: ["-m", "whisper_stt_mcp"]Tools appear as mcp_whisper_stt_transcribe, mcp_whisper_stt_list_models, etc.
With any MCP client
{
"mcpServers": {
"whisper-stt": {
"command": "/path/to/whisper-stt-mcp/.venv/bin/python",
"args": ["-m", "whisper_stt_mcp"]
}
}
}Project Structure
whisper-stt-mcp/
├── pyproject.toml
├── src/whisper_stt_mcp/
│ ├── __init__.py
│ ├── __main__.py # Entry point
│ ├── server.py # MCP tools
│ ├── transcriber.py # faster-whisper wrapper
│ ├── audio.py # ffmpeg conversion
│ ├── models.py # Model cache management
│ └── vad.py # Voice Activity Detection
├── tests/
│ └── test_smoke.py
└── hermes-skill/
└── SKILL.mdThe project is MIT licensed and open on GitHub.