On-Device vs Cloud Speech to Text: Privacy, Speed, and Accuracy Compared

The privacy argument isn't theoretical

People dictate passwords into login forms. Doctors dictate patient notes between appointments. Lawyers dictate case strategy on their commute. A freelance writer dictates a draft that includes client confidential data.

With cloud speech-to-text, every word of that audio travels over a network to a server, gets transcribed, and the text travels back. Even if the provider deletes audio immediately (and most say they do), the data existed on their infrastructure for the duration of processing. A subpoena, a breach, or a rogue employee during that window exposes it.

With on-device transcription, the audio goes from your microphone to a model running on your CPU or Neural Engine. It never touches a network interface. There is no window of exposure because the data never leaves the machine.

Real latency numbers

On an M1 MacBook Air running WhisperJot:

• Parakeet (on-device): ~200ms from end-of-speech to first text appearing. This is the CTC engine optimized for short utterances under 15 seconds.
• WhisperKit (on-device): ~400-600ms for a 30-second utterance. Longer audio takes proportionally longer because Whisper processes in 30-second chunks.
• Jot v1 (cloud): ~300ms including network round-trip on a fast connection. On coffee-shop Wi-Fi with 80ms ping, expect ~500ms.

The surprise: on-device is faster than cloud for short dictation on Apple Silicon. The Neural Engine processes the Parakeet model without the overhead of serializing audio, transmitting it, waiting for server-side inference, and deserializing the result. Cloud wins on older hardware where the local model runs slowly on CPU — an Intel MacBook Air takes 2-3 seconds for the same utterance that takes 200ms on M1.

Accuracy: closer than you'd expect

OpenAI's Whisper large-v3 (1.5 billion parameters, cloud-hosted) achieves roughly 3% word error rate on standard English benchmarks. WhisperKit running the same model on-device via CoreML achieves within 0.5% of that — the model weights are identical, the inference framework just differs.

Parakeet-CTC (600 million parameters, on-device) hits about 4% WER on the same benchmarks but runs 5x faster for utterances under 15 seconds. For dictation — where most utterances are a sentence or two — that 1% WER difference is inaudible in practice. You might see it on rare proper nouns or heavy accents, where the larger model's bigger vocabulary wins.

The real accuracy gap shows up on non-English languages. Cloud models trained on massive multilingual datasets handle accented English, code-switching, and minority languages better than smaller on-device models. If you regularly dictate in Tagalog-English or Swiss German, cloud still has an edge.

When cloud is the right call

On-device isn't always better. Cloud transcription makes sense when:

• Your hardware can't run the model. A 2.3GB Whisper model needs ~4GB of RAM overhead. Machines with 8GB total and a browser open don't have that headroom. Jot v1 needs no local model at all.
• You need maximum accuracy on rare terminology. Cloud providers can update models faster and run larger ensembles than fit on a laptop.
• You're in a live meeting and latency matters. Cloud can start streaming partial results before you finish speaking — some on-device models process only complete utterances.

When on-device is the right default

For daily dictation into text fields — which is what most people do most of the time — on-device wins on every axis except one (hardware requirements):

The strongest argument for defaulting to on-device: you probably don't know what you're about to say before you say it. Setting up cloud for "non-sensitive" dictation and switching to local for "sensitive" content requires you to predict sensitivity before speaking — which nobody actually does. Default to the mode that handles the worst case.