Run your own dictation server

Talking is faster than typing for me, and it skips spelling altogether. The catch with every cloud dictation app is the same: your voice goes to someone else's computer, gets transcribed there, and you have to trust what happens to it next. On a work laptop that is often a hard no.

So I built my own. It turned out to be one command and a couple of things worth knowing. This is how I did it, why each step is there, and the traps I hit so you can walk around them. If a word here is new, the back-to-basics page explains the ideas in plain terms.

What you are building, in one line

A small program on a computer you own that listens for an audio clip and sends back the words as text. It runs on your machine and nothing you say leaves the building. In the jargon this is a self-hosted Whisper server; you do not need to care about those words to use it.

This is the exact setup behind the dictation button on this site. If you just want to try dictation without building anything, that button already works. Read on if you want your own.

Why these choices

• Whisper, because it is the free, well-proven model for turning speech into text, and it is accurate enough that I stopped fixing its mistakes by hand.
• Docker, because it lets you run the whole thing with one command instead of installing a pile of dependencies by hand. Think of it as a sealed box that already contains everything the program needs.
• A plain old computer, because this runs on almost anything. It is faster with a graphics card, and there is a separate guide for that, but you do not need one to start.

What you need

• A computer running Ubuntu (I used 22.04). Other systems work too; the commands differ slightly.
• About 3 GB of free disk space for the program and the speech model.
• Ten quiet minutes.

Everything below was run on a clean machine, then run again from scratch on a second clean machine to make sure the guide actually works start to finish. It gave the same result both times.

Step 1: install Docker

A fresh Ubuntu does not come with Docker. My first draft of this guide assumed it did, and the second machine immediately said "command not found". So, first:

sudo apt-get update
sudo apt-get install -y docker.io

Check it landed:

docker --version
sudo systemctl is-active docker

You want a version line and the word active. One thing that will trip you up: on a fresh install you have to put sudo in front of every docker command. Leave it off and you get permission denied while trying to connect to the docker API. That is not a real error, it is just the machine asking you to say please.

Step 2: make somewhere to keep the model

sudo mkdir -p /srv/whisper/data

The speech model is a biggish download. This folder is where it lives so it only downloads once.

Step 3: make a key so only you can use it

This key is what stops anyone else on your network sending audio to your server. Generate one and save it to a file:

openssl rand -hex 24 | tee ~/whisper-api-key.txt

The long random string it prints is your key. It is now saved in ~/whisper-api-key.txt.

Step 4: start the server

One command does it:

sudo docker run -d --name whisper \
  -p 9000:9000 \
  -v /srv/whisper/data:/var/lib/whisper \
  -e WHISPER_MODEL=small \
  -e WHISPER_DEVICE=cpu \
  -e WHISPER_API_KEY=$(cat ~/whisper-api-key.txt) \
  --restart unless-stopped \
  hwdsl2/whisper-server:latest

You do not need to memorise that. In plain terms it says: run the Whisper program in the background, let it be reached on port 9000, keep the model in the folder from Step 2, use the medium-small model on the processor, protect it with the key from Step 3, and start it again automatically if the machine reboots.

The first time it starts it downloads the model, so give it a minute. Watch it get ready:

sudo docker logs whisper

Wait for the line Whisper speech-to-text server is ready. I rebooted the test machine on purpose to check the "start again automatically" part. It came back on its own.

One thing to ignore: the startup log prints a suggested address that happens to be your public internet address. Do not use it and do not open this up to the internet. On your own network, localhost or your machine's local address is all you need.

Step 5: test it with your voice

Put a short audio clip in your home folder. A phone voice memo is fine. Then:

curl -s http://localhost:9000/v1/audio/transcriptions \
  -H "Authorization: Bearer $(cat ~/whisper-api-key.txt)" \
  -F [email protected] \
  -F model=whisper-1

It sends the clip and prints your words back as text. That is the whole thing working. A dictation app can now point at http://your-machine's-address:9000/v1 with the same key, and you have private dictation.

How fast is it, honestly

I will not pretend here. On a modest four-core machine with no graphics card, a five-second clip took about twelve seconds to come back. That is fine for firing off a voice note or a short paragraph, and it is not fast enough to feel like live dictation where the words appear as you speak.

If you want that, a graphics card changes everything: the same clip comes back in under a second. That upgrade is its own short guide, and it is the step that turned this from "a neat thing I built" into "the thing I actually use every day".

When something goes wrong

• permission denied while trying to connect to the docker API means you left off sudo.
• An error about a missing authorisation header means you forgot the key. Annoying in the moment, but it also proves the lock works.
• Lost your key? Read it back with cat ~/whisper-api-key.txt.

If you want it gone

sudo docker rm -f whisper
sudo docker rmi hwdsl2/whisper-server:latest
sudo rm -rf /srv/whisper

That removes the program, the download and the model, and your machine is exactly as it was before. No leftovers, nothing phoning home. That, really, is the whole point.