Skip to main content
Transcription

The Speech-to-Text Accuracy Guide: How to Read the Numbers and Improve Yours

What word error rate actually measures, why 99% accuracy claims are usually meaningless, and the changes that measurably improve your own transcription accuracy.

The myvoicenotes.ai Team7 min read
Share

Every transcription vendor claims accuracy in the high nineties. They are not necessarily lying, and the number is still close to useless, because it was measured on audio that has almost nothing in common with your Tuesday afternoon meeting.

Understanding what the number measures — and what actually drives it in your own recordings — is the difference between blaming your tool and fixing your setup.

What word error rate actually measures

The industry standard is word error rate, or WER. Align the transcript against what was really said, count three kinds of mistake, divide by the number of words spoken.

  • Substitutions: a wrong word replaces a correct one. "Fifteen" becomes "fifty".
  • Deletions: a spoken word is missing entirely.
  • Insertions: a word appears that nobody said, often from a cough or crosstalk.

A 5% WER on a 1,000-word recording means about 50 word-level errors. "Accuracy" in marketing copy is normally just 100% minus WER.

The metric's weakness is that it weights every word equally. Losing "um" and losing a contract value both count as one error. So two transcripts can score identically while one is fine and the other is a liability. Always ask which words were wrong.

Why benchmark numbers do not transfer

Published figures come from research datasets: single speakers, close microphones, quiet rooms, careful or scripted speech, common vocabulary. Under those conditions, current models genuinely do reach two to five percent WER, occasionally better than human transcribers.

Your recordings differ on every axis at once, and the factors compound rather than add.

Distance and reverb. Signal quality falls off with distance while reverberation and noise stay constant, so the ratio of useful speech to everything else collapses. A microphone in the middle of a table is metres from some speakers.

Overlapping speech. Two people talking at once is the single hardest condition. Error rates in overlapped regions can be several times the baseline, and it also corrupts speaker labels for the surrounding sentences.

Background noise. Steady broadband noise — air conditioning, traffic, a laptop fan — masks exactly the high-frequency consonant detail that distinguishes similar words.

Domain vocabulary. Product names, drug names, internal acronyms and surnames are guessed phonetically unless the model has seen them. This is where the errors that matter concentrate.

Speaking style. Fast speech, trailing sentences, mumbling and heavy filler all raise WER. Enthusiastic conversation is harder than a monologue.

Accent and language mixing. Modern multilingual models handle accents far better than five years ago, but accuracy still varies with representation in training data, and code-switching mid-sentence remains difficult.

A vendor benchmark at 3% can easily be 15% in a real meeting room without anything being broken. That is expected behaviour, not a defect.

The improvements ranked by actual payoff

We measured these on the same conversations, re-recorded under different conditions.

1. Get the microphone closer. The largest single win available. Moving from a laptop's built-in array to a wired headset or lapel microphone roughly halved word errors in our tests. No vendor change comes close to that, which is why comparing tools before fixing audio is backwards.

2. Record separate streams for remote calls. When each participant's audio is captured independently, overlap stops being a mixing problem and speaker labels become nearly perfect. If your setup allows per-participant tracks, use them.

3. Reduce reverb. Soft surfaces, smaller rooms, occupied rooms. Glass-walled meeting rooms are the worst common environment and there is nothing software can do about a room that smears syllables.

4. Eliminate steady noise. Turn off the fan, close the window, move away from the coffee machine. Thirty seconds of preparation, measurable improvement.

5. Feed the model your vocabulary. If custom vocabulary or keyword hints are available, load your product names, client names and acronyms. This targets precisely the errors that cause real damage.

6. Speak slightly more deliberately at the important moments. Not artificially slowly for the whole recording, just clearly when saying a number, a name or a date. The two seconds cost nothing.

7. Spell unusual names once. "Northwind, N-O-R-T-H-W-I-N-D." It appears in the transcript and gives you something searchable even if later mentions are mangled.

Notice that six of the seven are free and none of them involve changing software.

Why speaker labels fail separately from words

Diarisation — deciding who said which words — is a different task from recognition, and it fails in its own characteristic way.

The reliable case is distinct voices, taking turns, on decent microphones. Under those conditions modern diarisation is genuinely good. The failures cluster in four places.

Turn boundaries. The last few words of one speaker frequently get attributed to the next. This is the single most common diarisation error and it matters because those trailing words are often the conclusion of a thought.

Overlapping speech. When two people talk at once, the model has to assign one label to a mixed signal. Whatever it picks is wrong for somebody.

Similar voices. Two speakers in the same pitch range with similar speaking styles get merged, sometimes for a whole recording.

Short interjections. A two-word "yeah, agreed" often gets absorbed into the surrounding speaker's turn and disappears as a separate contribution.

The structural fix is to record each participant on a separate track, which turns attribution from an inference problem into a bookkeeping one. Most remote meeting setups can do this, and it is the biggest accuracy improvement available to anyone whose work depends on knowing who said what.

If separate tracks are impossible and attribution matters legally or contractually, plan on human verification. No current tool removes that obligation, and a transcript that confidently attributes a statement to the wrong person is worse than one that says nothing.

How to measure your own accuracy in twenty minutes

Vendor numbers are irrelevant to you. Your own measurement is not hard.

  1. Record five minutes of your most typical audio — the real thing, not a test read.
  2. Transcribe it with whatever tool you are evaluating.
  3. Listen back and mark every error, noting the category: filler, common word, name, number, technical term.
  4. Compute the crude WER, then separately count errors in the categories that matter to you.

That second count is the number you actually care about. Six errors in filler words and none in figures is a good result. Two errors, both in prices, is a bad one, even though the WER is lower.

Repeat with your headset instead of your laptop microphone. The difference is usually large enough to end the conversation about which vendor to buy.

Reading transcripts safely

Transcription errors have an unusual property: the output always reads fluently. Models produce plausible words, so a wrong word arrives in a grammatical sentence and does not announce itself the way a garbled handwritten note does. Nothing in the text signals low confidence.

Practical defences:

  • Verify anything you will act on. Numbers, dates, names, commitments. Jump to the timestamp and listen. Ten seconds.
  • Treat unfamiliar terms as suspects. A phrase that seems slightly off in an otherwise coherent paragraph is usually a substitution, not an unusual turn of phrase.
  • Never quote publicly without listening. Not for legal reasons alone — the risk of quoting a model's guess as somebody's words is reason enough.
  • Distrust the transcript at overlap boundaries. Where speaker labels change, error rates are highest. If attribution matters, verify it.

Multilingual audio and the code-switching problem

Two related situations get conflated and behave very differently.

A whole recording in one non-English language is generally handled well by current multilingual models, with accuracy varying by how much data existed for that language. Major European and East Asian languages perform close to English; lower-resource languages lag, sometimes considerably.

Code-switching within a single conversation is much harder. When a speaker moves between languages mid-sentence — extremely common in multilingual workplaces — models that commit to one language at the start will transcribe the other one phonetically into nonsense. The output looks like a recognition failure but is really a language-identification failure.

Practical mitigations, in order of effectiveness: set the language explicitly rather than relying on auto-detection if the recording is predominantly one language; split recordings at language boundaries where you can; and expect to correct the switched passages manually. If your meetings are genuinely bilingual throughout, test this specific case before committing to a tool, because performance varies far more between vendors here than it does on clean monolingual English.

Accented speech in English is a separate axis again. The gap between accents has narrowed substantially but has not closed, and the only number that matters is the one you measure with your own voice.

Where accuracy stops being the right thing to optimise

Beyond a certain point, transcript quality stops determining usefulness. If your goal is recall rather than a verbatim record, a 92% transcript and a 97% transcript perform almost identically: you ask what was decided about pricing, and both surface the right moment, because meaning survives a handful of wrong words.

That is why we would rather have complete coverage at 93% than perfect transcripts of the third of conversations that happen to occur inside scheduled calls. The recording that does not exist has an error rate of 100%.

So chase accuracy up to the point where the transcript is reliably usable — which mostly means fixing your microphone — and then spend your remaining effort on capture coverage and retrieval. That is where the returns are.

Share

Frequently asked questions

What is word error rate?

Word error rate, or WER, is the number of substitutions, deletions and insertions the transcript contains, divided by the number of words actually spoken. A WER of 5% on a 1,000-word recording means 50 word-level errors. It is the standard measure because it is simple and comparable, but it treats every word as equally important, which is why a transcript with a low WER can still get the one number in the sentence wrong.

Is 95% transcription accuracy good?

It depends on what the errors are. A 5% error rate spread across filler words and articles produces a transcript that reads as clean and is perfectly usable. The same 5% concentrated in names, prices and dates produces a transcript that is actively dangerous to rely on. Always judge accuracy by which words were wrong, not just how many.

Why is my transcription accuracy worse than advertised?

Benchmark numbers come from clean, single-speaker audio recorded on good microphones in quiet rooms with scripted or careful speech. Your real recordings have room reverb, overlapping speakers, a laptop microphone two feet away, background noise and domain jargon. Each of those factors independently increases error rate, and they compound, so a 3% benchmark can become 15% in a real meeting room without anything being broken.

What single change improves transcription accuracy the most?

Reducing the distance between mouth and microphone. Signal quality falls off sharply with distance while room reverb and background noise stay constant, so moving from a laptop microphone at arm's length to a headset or lapel microphone typically halves word errors. It beats every software setting and every vendor switch available to you.

Do accents reduce speech-to-text accuracy?

Historically, considerably. Modern multilingual models have narrowed the gap substantially but not closed it, and accuracy still varies by accent depending on how well represented it was in training data. If you or your colleagues have accents that fall outside the dominant training distribution, test with your own voice on your own audio rather than trusting published averages.

How can I fix recurring errors on names and jargon?

Use a custom vocabulary or keyword hint feature if the tool offers one, which biases the model towards terms it would otherwise guess phonetically. Where that is unavailable, spell unusual names out loud the first time they appear in a recording, and expect to correct product names and acronyms in any transcript you intend to publish or quote.

Try the AI note taker this blog is about

myvoicenotes.ai records anything in one tap, transcribes it automatically, and lets you ask questions across everything you have ever recorded. Join the waitlist for early access.