On-Device vs Cloud Speech to Text: Privacy, Speed, and Accuracy Compared

The privacy argument isn't theoretical

Everyday writing can contain private information. Consider where audio is processed before choosing a dictation workflow. WhisperJot is not a medical-records product; do not dictate protected health information into it.

With cloud speech-to-text, every word of that audio travels over a network to a server, gets transcribed, and the text travels back. Even if the provider deletes audio immediately (and most say they do), the data existed on their infrastructure for the duration of processing. A subpoena, a breach, or a rogue employee during that window exposes it.

With local engines, audio is not uploaded for transcription. Jot Cloud, our optional cloud engine, is opt-in, runs only while selected, and audio is never stored on our servers. Optional Identify Speakers uploads retained meeting audio for speaker labels; audio imports can upload files for cloud processing when you choose that workflow. Transcript sync, usage stats, and crash reports have independent settings.

What affects responsiveness

Responsiveness depends on hardware, utterance length, and network conditions:

• Parakeet (on-device): ~200ms from end-of-speech to first text appearing. This is the CTC engine optimized for short utterances under 15 seconds.
• WhisperKit (on-device): ~400-600ms for a 30-second utterance. Longer audio takes proportionally longer because Whisper processes in 30-second chunks.
• Jot v1 (cloud): ~300ms including network round-trip on a fast connection. On coffee-shop Wi-Fi with 80ms ping, expect ~500ms.

The surprise: on-device is faster than cloud for short dictation on Apple Silicon. The Neural Engine processes the Parakeet model without the overhead of serializing audio, transmitting it, waiting for server-side inference, and deserializing the result. Cloud wins on older hardware where the local model runs slowly on CPU — an Intel MacBook Air takes 2-3 seconds for the same utterance that takes 200ms on M1.

Accuracy: closer than you'd expect

OpenAI's Whisper large-v3 (1.5 billion parameters, cloud-hosted) achieves roughly 3% word error rate on standard English benchmarks. WhisperKit running the same model on-device via CoreML achieves within 0.5% of that — the model weights are identical, the inference framework just differs.

Parakeet-CTC (600 million parameters, on-device) hits about 4% WER on the same benchmarks but runs 5x faster for utterances under 15 seconds. For dictation — where most utterances are a sentence or two — that 1% WER difference is inaudible in practice. You might see it on rare proper nouns or heavy accents, where the larger model's bigger vocabulary wins.

The real accuracy gap shows up on non-English languages. Cloud models trained on massive multilingual datasets handle accented English, code-switching, and minority languages better than smaller on-device models. If you regularly dictate in Tagalog-English or Swiss German, cloud still has an edge.

When cloud is the right call

On-device isn't always better. Cloud transcription makes sense when:

• Your hardware can't run the model. A 2.3GB Whisper model needs ~4GB of RAM overhead. Machines with 8GB total and a browser open don't have that headroom. Jot v1 needs no local model at all.
• You need maximum accuracy on rare terminology. Cloud providers can update models faster and run larger ensembles than fit on a laptop.
• You're in a live meeting and latency matters. Cloud can start streaming partial results before you finish speaking — some on-device models process only complete utterances.

When on-device is the right default

Local engines provide offline transcription without an audio upload for that processing step. Choose the engine based on your hardware, privacy preferences, and results with your own recordings:

The strongest argument for defaulting to on-device: you probably don't know what you're about to say before you say it. Setting up cloud for "non-sensitive" dictation and switching to local for "sensitive" content requires you to predict sensitivity before speaking — which nobody actually does. Default to the mode that handles the worst case.