Caption Accuracy: How It Is Measured, and What a 99% Score Actually Means

How is caption accuracy measured?
Two ways, and they disagree. One divides correct words by total words — the method behind the familiar 99% figure. The other is word error rate, which adds substitutions, deletions and insertions over the reference word count. The two scores differ by exactly the insertion rate, so the same file can honestly be reported as 95% or 90%.
Every caption contract, delivery specification and vendor comparison eventually arrives at a percentage. Someone promises 99%. Someone else measures 94% and asks for a credit. Both parties produce a spreadsheet, and the spreadsheets do not agree.
They usually do not disagree because anybody lied. They disagree because “accuracy” is not one measurement. It is at least three, with three different denominators, and the industry uses the same word and the same percent sign for all of them.
It matters most at the point an automatic transcript is handed over as a draft and somebody has to say how good it is. This guide sets out what each method actually counts, proves why two of them cannot return the same number, and shows what to write down so a score means something to the person who receives it.
Two Definitions, One Number
Start with the thing that causes the most expensive arguments: the two definitions in daily use are not variations on a theme. They are different arithmetic.
| Metric | What it computes | Denominator | Range |
|---|---|---|---|
| Correct-word accuracy | Correct words ÷ total words | Words in the programme | 0% to 100% |
| Word error rate (WER) | (Substitutions + deletions + insertions) ÷ reference words | Words in the reference transcript | 0% upwards, no ceiling |
| NER | (Subtitle words − edition errors − recognition errors) ÷ subtitle words | Words in the delivered subtitles | 0% to 100% |
Read the denominator column again. Correct-word accuracy and word error rate both measure against the source material. NER measures against what was delivered. A file that drops a third of the audio can still score well on NER if what it did deliver was faithful, because the words it never produced are not in its denominator.
That is not a flaw. It is a deliberate choice, made for live subtitling, for reasons covered in the NER section below. But it means the first question to ask about any accuracy figure is not “how high is it” but “what is on the bottom of the fraction.”
Where 99% Comes From
The 99% figure that appears in almost every captioning contract has a traceable origin, and it is not the FCC caption quality rule. It sits in a different paragraph doing a different job.
The caption quality standards live at 47 CFR § 79.1(j)(2) and are entirely descriptive: captioning must be accurate, synchronous, complete and appropriately placed. No number appears there.
Further down the same section, in the best practices for real-time captioning vendors at § 79.1(k)(2)(iv), accuracy is given an arithmetic definition — described there as a measurement of the percentage of correct words out of total words in the programme. The paragraph then illustrates it: 7,000 total words in a programme, less 70 errors, equals 99% accuracy.
What that means in practice. 7,000 − 70 = 6,930, and 6,930 ÷ 7,000 = 99%. The arithmetic is internally consistent and the definition is clear. What it is not is a threshold. It appears in a list of best practices directed at real-time vendors, as an illustration of a method. Whether the FCC rules impose any numerical accuracy requirement is a separate question, answered in our guide to the FCC caption rules. This article is about the measurement, not the obligation.
Two things follow from the definition itself, and both matter more than the 99%.
It counts correct words, not errors. The quantity on top of the fraction is the words that came out right. That sounds like a distinction without a difference until you meet an inserted word — a word in the captions with no counterpart in the audio at all. It is unmistakably an error, and it does not reduce the number of correct words by one. It is not in the numerator and it is not in the denominator.
It is bounded at both ends. Correct words can never exceed total words, so the score cannot exceed 100%, and it cannot fall below zero. That boundedness is the property most people assume all accuracy scores have. One of them does not.
What Word Error Rate Counts
Word error rate came from speech recognition research, and it is the number every automatic speech recognition vendor publishes. It is a length-normalised edit distance: the smallest number of single-word changes needed to turn the output into the reference, divided by the length of the reference.
| Term | What it is | Example against “the meeting starts at nine” |
|---|---|---|
| Substitution (S) | A reference word replaced by a different word | “the meeting starts at wine” |
| Deletion (D) | A reference word missing from the output | “the meeting starts nine” |
| Insertion (I) | A word in the output with no counterpart in the reference | “the meeting starts at about nine” |
| Correct (C) | A reference word reproduced exactly | — |
The formula is WER = (S + D + I) ÷ N, where N is the number of words in the reference. That last point is worth stating plainly because getting it wrong is the single commonest error in home-made accuracy spreadsheets: the denominator is the transcript you are measuring against, never the output you are measuring.
And because every reference word is either correct, substituted or deleted, N = S + D + C. Insertions are absent from that identity — which is precisely where the two definitions part company.
The Gap Is the Insertion Rate
Many vendors and most QC spreadsheets convert word error rate into an accuracy figure by subtracting it from 100%. The two numbers look like the same quantity expressed two ways. They are not, and the discrepancy is not approximate — it is exact and it is nameable.
Substituting N = S + D + C into the subtraction:
- 1 − WER = (N − S − D − I) ÷ N
- and since N − S − D = C, that is (C − I) ÷ N
- while correct-word accuracy is C ÷ N
Subtract one from the other and everything cancels except the insertions:
Correct-word accuracy − (1 − WER) = I ÷ N. The difference between the two most widely used caption accuracy figures is exactly the insertion rate. Not roughly, not usually — identically, for every file, always.
Put numbers on it. A file measured against a 100-word reference, with 2 substitutions, 3 deletions and 5 insertions:
| Quantity | Value | Working |
|---|---|---|
| Reference words (N) | 100 | Given |
| Correct words (C) | 95 | 100 − 2 − 3 |
| Word error rate | 10% | (2 + 3 + 5) ÷ 100 |
| Accuracy by subtraction | 90% | 100% − 10% |
| Correct-word accuracy | 95% | 95 ÷ 100 |
| Spread | 5 points | 5 insertions ÷ 100 words |
Ten errors in a hundred words, and a five-point spread between two defensible scores for the same file. On a delivery with a 95% floor, one method passes and the other fails, and no one has misread anything.
Insertions are not an exotic failure mode either. Expanding a contraction produces one: “I'm” transcribed as “I am” adds a word. So do filler words a recogniser catches and a human transcriber would not have typed, repeated words at a stutter, and hesitation markers. Any ASR output measured against a clean human transcript carries insertions, which means the two scores always differ on machine-generated captions; it is only the size of the gap that varies.
The Score That Goes Below Zero
Boundedness is where the asymmetry gets genuinely awkward. Correct-word accuracy cannot leave the 0–100% range. Word error rate can exceed 100%, because insertions add to the numerator without adding to the denominator. Once insertions outnumber the correct words, the rate passes 1.0.
This is not a pathological hypothetical. A recogniser that loses its place and loops, a transcript accidentally concatenated twice, a burned-in translation captured alongside the dialogue — each produces far more output words than reference words.
| Quantity | Value |
|---|---|
| Reference words | 100 |
| Substitutions / deletions | 0 / 0 |
| Insertions | 120 |
| Word error rate | 120% |
| Accuracy by subtraction | −20% |
| Correct-word accuracy | 100% |
Every reference word was reproduced correctly, so the correct-word method returns a perfect score for a file no viewer could use. The subtraction method returns a negative percentage, which is arithmetically honest and will still break any report template that validates a 0–100 range.
Neither number describes the file. That is the real lesson: a single accuracy percentage cannot distinguish a caption file that says too little from one that says too much, and the two failure modes need opposite remedies. Report the error counts alongside the score and the ambiguity disappears.
The Step That Decides the Answer
Before any of these metrics can run, both texts have to be reduced to comparable tokens. Normalisation is the step nobody mentions in a results email, and it moves the score more than the choice of metric does.
| Step | Effect on the score |
|---|---|
| Lower-casing | Removes every capitalisation error from the count |
| Stripping punctuation | Removes every punctuation error from the count |
| Expanding contractions | Changes the token count on both sides, so it changes N |
| Converting digits to words | Turns “2026” into words, which can be one token or four |
| Standardising spellings | Stops a regional variant counting as a substitution |
| Removing non-speech tags | Drops sound-effect and music cues from scoring entirely |
The magnitude is easy to underestimate. Speechmatics publishes a worked example in its own benchmarking documentation in which the same spoken content scores roughly 41% raw and about 8% after normalisation — and the reference word count moves from 22 to 24 in the process.
That second detail is the one to hold on to. Normalisation changes the denominator as well as the error count, which means two scores computed with different normalisers are not comparable even in principle. They are not two measurements of one quantity; they are measurements of two different quantities.
The practical rule. An accuracy figure without its normalisation rules is not a result, it is a rumour. If a vendor quotes a word error rate and cannot tell you whether punctuation was scored, the number cannot be compared with anyone else's — including your own.
What the Number Cannot See
Every metric on this page scores one property: whether the right words are present in the right order. A caption file has several other properties that decide whether it is usable, and no accuracy percentage reports on any of them.

| Property | Why the metric misses it | Where it is judged |
|---|---|---|
| Punctuation | Normally stripped before scoring | Named in the FCC accuracy standard |
| Capitalisation | Normally lower-cased before scoring | Named in the FCC accuracy standard |
| Speaker identification | Labels are not reference words | Speaker ID conventions |
| Non-speech information | Sound and music cues are usually removed as tags | Non-speech cues |
| Synchronisation | No timing information enters the calculation | FCC synchronicity standard |
| Reading speed | A perfectly accurate caption can be unreadably fast | Reading speed limits |
| Placement | Position is not text | Caption placement |
| Completeness | A sample cannot show whether captions run to the end | FCC completeness standard |
Line those omissions up against regulation and the result is uncomfortable. The FCC's accuracy standard explicitly asks for correct spelling, correct punctuation and capitalisation, and the inclusion of speaker identity and non-speech information. Standard normalisation removes the first three from the count and the fourth never enters it.
The honest summary. A word error rate score addresses part of one of the four FCC quality characteristics. It is a useful number. It is not a compliance result, and a delivery specification that asks only for an accuracy percentage has specified roughly an eighth of what it cares about.
This is also why automated scoring and a human review pass are not substitutes for one another. The properties a metric cannot see are mostly the ones that need the picture, which is the same boundary that governs which caption errors can be corrected automatically: an issue with a target value can be measured and fixed by a rule, and an issue without one cannot.
Editing Is Not Error
Here is the problem that breaks word error rate for a large share of real captioning work: not every difference from the audio is a mistake.
Captions for education and for younger audiences are frequently edited for readability. The Described and Captioned Media Program's Captioning Key states that it supports editing based on research results, and describes editing as often essential so that viewers have time to read — while requiring that quoted speakers, poems and song lyrics be captioned verbatim. The FCC's own accuracy standard forbids paraphrasing except where time constraints force it.
So editing is sanctioned, sometimes required. And a plain word error rate counts every deliberate omission as a deletion. The same applies to the extra information an SDH file carries and a plain subtitle file does not.
The perverse consequence. Score a well-edited caption file and a verbatim one against the same verbatim reference, and the edited file loses. The metric penalises the editing that the guidance asks for. A tool cannot tell a considered cut from a dropped word, because on the page they are the same event.
There are only two honest ways out, and choosing between them is a decision to make before measuring, not after.
- Score against an edited reference. Build the reference to the same editorial standard as the deliverable, so the editing decisions are baked into the thing you are comparing with. This works, and it means the reference is now an editorial artefact that someone has to approve.
- Use a metric that classifies the error. Separate the omissions that were chosen from the ones that were failures. That is exactly what the live subtitling world built, and it is the subject of the next section.
What does not work is scoring an edited file against a verbatim reference and treating the result as a quality signal. It measures how heavily the file was edited.
How Live Subtitling Scores Instead
Live subtitling faced the editing problem earlier and more acutely than anyone else, because a respeaker working in real time has no option but to condense. The response was the NER model, developed in respeaking research and used for public broadcast assessment in several European countries.
The model takes the number of words in the subtitles, subtracts edition errors and recognition errors, and divides by that same word count. Two features of that design do the work:
| Error class | What it captures | Why it is separated |
|---|---|---|
| Edition errors (E) | Decisions made by the person producing the subtitles about what to condense or cut | Editing is expected in live work; only editing that loses meaning should be penalised |
| Recognition errors (R) | The speech engine or the respeaker producing the wrong words | A straightforward failure, judged differently from a choice |
Second, errors are not counted equally. Each is graded into severity bands — minor, standard and serious — and the grading is driven by the impact an error has on the viewer. An error that does not affect comprehension is treated differently from one that creates a different and credible meaning, which is the most dangerous kind: the viewer has no way to detect it.
The published quality threshold for the model is 98%, and the research is blunt about what the band below it feels like — a 96% score is described as equating to virtually unreadable subtitles. A two-point drop separating “acceptable” from “unreadable” tells you how compressed the useful range of any accuracy scale is.
The cost of the design. Deciding whether an omission was a reasonable edit, and how serious an error is, cannot be automated — the research is explicit that this has to be done manually by a trained evaluator. NER buys the right question at the price of a human. Word error rate is cheap, instant and answers a narrower one. Neither is the better metric in general; they are priced differently.
When a Regulator Measured
The largest published exercise in caption quality measurement remains Ofcom's live subtitling programme in the UK. The first report appeared on 30 April 2014, the first of four produced over a two-year period, sampling programmes from the BBC, ITV, Channel 4, Channel 5 and Sky and scoring them with the NER model.
The scale is the part worth knowing: the exercise covered roughly 78,000 subtitles from around 300 programmes over two years, assessed by about 30 trained evaluators, with average disagreement between assessors reported at a fraction of a percentage point.
Two findings from that programme have aged well.
Latency was the headline problem, not word accuracy. Ofcom reported median latency at close to twice the recommended maximum delay of three seconds. The subtitles were largely right; they arrived too late to be any use against the picture. No accuracy metric on this page would have caught that, because none of them look at time.
Measuring consistently is itself hard work. Thirty trained evaluators and a formal certification process were needed to get assessor agreement to that level. Any internal QC programme that scores accuracy with whoever is free that afternoon should expect its own numbers to disagree with each other.
Counting Characters Instead
Word error rate assumes you can identify words, which assumes the writing system puts spaces between them. For a significant share of the world's content, it does not.
Character error rate applies the same edit-distance arithmetic at the character level: substituted, deleted and inserted characters over the reference character count. Two situations call for it.
- Languages written without word spacing. In Chinese, Japanese and Thai, tokenising into words is an analytical decision, and different tokenisers will hand you different scores for identical text. Characters remove that variable.
- Work where spelling carries the risk. Proper nouns, drug names, product names and part numbers. A single wrong character in a surname is one whole substitution to word error rate, identical in weight to any other wrong word — character error rate reflects that most of the name was right, and that the remaining fix is small.
Character error rate does not replace word error rate; it answers a different question, and for CJK delivery it is usually the only one of the two that can be computed reproducibly. If you are commissioning subtitles across scripts, specify which metric applies to which language rather than quoting one figure for the whole package — the same care that translating subtitles for broadcast needs on text expansion.
What the Standards Require
Given how freely 99% circulates as “the industry standard,” it is worth stating what the major accessibility authorities actually publish on numerical accuracy.
| Authority | On accuracy | Numerical threshold |
|---|---|---|
| FCC, 47 CFR § 79.1(j)(2) | Descriptive four-part standard, with a de minimis allowance | None |
| FCC, § 79.1(k)(2)(iv) | Defines the correct-words method for real-time vendors | 99%, as an illustration in a best practice |
| DCMP Captioning Key | States that errorless captions are the goal for each production | None |
| WCAG 2.2, SC 1.2.2 | Requires that captions be provided; a presence requirement | None |
| NER model | Published quality threshold for live subtitling assessment | 98% |
So the position is this: of the authorities most often cited in caption contracts, the one numerical figure in wide circulation is an illustration inside a voluntary best practice, and the one genuine published threshold — 98% — belongs to a model that requires a trained human assessor and is scoped to live subtitling.
That does not make 99% useless. It makes it a service level: a commercial commitment that both parties can measure, which is a perfectly good thing for a contract to contain. It is simply not a legal standard, and the distinction matters when a client asks whether a 97% file is compliant. European obligations under the EAA follow the same descriptive pattern.
Their Benchmark Is Not Your Score
Speech recognition vendors publish word error rates, and those figures are usually honest. They are also close to useless for predicting what you will get, because the number is a property of the audio at least as much as of the engine.
The variables that move a transcription result are the ones a benchmark corpus is chosen to minimise: audio quality, speaking clarity, background noise, overlapping speakers and specialist vocabulary. Clear dialogue with limited background noise produces the best results, which is another way of saying that a benchmark is a measurement of clean audio.
| Your content has | Effect on the score | What actually helps |
|---|---|---|
| Overlapping speakers | Substitutions and deletions through the crosstalk | A provider with speaker diarisation, and a manual pass on the overlaps |
| Proper nouns and brand names | Consistent substitutions, repeated every time the term occurs | A terminology guide supplied with the job |
| Accents and dialect | Substitutions concentrated in particular speakers | Locale selection where the provider offers it |
| Music and effects beds | Insertions, and deletions under the loud passages | Score from a clean dialogue stem if one exists |
| Code-switching mid-file | Whole passages substituted or dropped | Auto-detect language modes rather than a fixed source language |
| Low-bitrate or phone audio | Raises every error type at once | Nothing in post; fix the capture |
Closed Caption Creator submits transcription jobs to a choice of providers — AssemblyAI, Gladia, Deepgram, Rev.ai, Google Speech-to-Text, Speechmatics, VoiceGain, ElevenLabs Scribe, GPT Transcribe and Gemini 3.5 Transcribe — which differ in language coverage and in features such as diarisation and word-level timing. The transcription tools expose those options per job. Those differences matter more to your score than the gap between their published benchmarks, because they decide which of the rows above you can do something about.
The only benchmark worth acting on is yours. Take twenty minutes of representative audio from the content you actually caption, run it through two or three providers, and score the results with one normalisation rule. The ranking will often differ from the published one, and it is the ranking that applies to your work.
Moving the Number Before You Measure
Most accuracy effort goes into measuring and then correcting. There is one intervention that acts earlier, on the recognition step itself, and it targets the errors that cost the most per occurrence.
Proper nouns, specialist terminology, brand names and uncommon phrases are what general speech models get wrong, and they get them wrong consistently: the same mishearing, every time the term appears. A recurring character name in a series is not one error, it is one error per episode multiplied by every mention.
Supplying known vocabulary ahead of the job changes the recogniser's behaviour at the point where it is choosing between competing phonetic interpretations. In Closed Caption Creator this is a KNP (Key Names and Phrases) guide, attached to a transcription job by team accounts, and passed to the provider as custom vocabulary. The documentation is direct about the effect: it measurably reduces word error rate for specialist terminology, proper nouns and uncommon phrases.
The same guide can be attached to an automatic translation job, so approved renderings of names carry across target languages instead of being re-guessed per language. It is the same discipline that glossaries and do-not-translate lists apply on the translation side. For episodic work this is the highest-leverage accuracy work available, because the guide is built once and applies to every episode after it — and it solves the consistency problem among human captioners at the same time.
Order of operations. Build the terminology guide, then transcribe, then measure. Measuring first and correcting afterwards fixes one episode. Supplying the vocabulary first fixes the series — and it moves the score you are about to report rather than the file you are about to deliver.
Building a Defensible Reference
Every metric here is a comparison against a reference transcript, which makes the reference the most consequential artefact in the whole exercise and the one most often produced carelessly. Any error in it becomes an error attributed to the file under test.
A reference that will survive a dispute needs decisions recorded, not just text:
- Who made it, and from what. Transcribed from the finished programme audio, not from a script or an as-recorded transcript — what was said is frequently not what was written.
- Verbatim or edited. Decided explicitly, in writing, before transcription starts. This single choice can move the score by several points.
- How disfluencies are treated. False starts, repetitions and filler words either appear or they do not, and whichever way you go, both sides must match.
- How numbers, dates and currency are written. “Twenty twenty-six” and “2026” are different token counts for the same sound.
- How names are spelled. From the production's own terminology list where one exists, so the reference and the deliverable agree on what correct looks like.
- What is out of scope. Non-speech cues, speaker labels, on-screen text — either scored or excluded, stated either way.
- Who checked it. A second pass by someone other than the transcriber, because an unchecked reference is an opinion.
Those seven decisions are the measurement protocol. Two labs with the same files, the same metric and different answers to any one of them will produce different scores and both will be right.
How Much to Measure
Scoring a whole programme against a hand-made reference is expensive, so almost everyone samples. There is no published standard for how much of a caption file to sample, which means the choice is yours to make and yours to defend.
Three properties make a sample worth trusting.
| Choice | Why it matters |
|---|---|
| Several short windows, not one long one | Error density varies enormously by scene. One ten-minute block measures one acoustic environment |
| Windows chosen before anyone looks | A sample picked after reading the file measures the picker, not the file |
| Hard passages deliberately included | Crowd scenes, crosstalk and specialist dialogue are where delivery risk lives |
Ofcom's exercise is the useful reference point for scale: roughly 78,000 subtitles across about 300 programmes, which is a sustained programme of work rather than a QC step. If your own sampling is a few minutes per deliverable, treat the result as an indicator that triggers a closer look, not as a measurement of the programme — and say so when you report it.
A clean complement to sampling is to run the deterministic checks across the whole file rather than a window. Reading speed, line length, duration, gaps and overlaps can all be validated on every event at no extra cost in a caption QC pass. Sample what needs a human; check everything that does not — the same split between content and technical review that broadcast QC already makes.
Writing Accuracy Into a Contract
An accuracy clause that reads “captions shall be 99% accurate” is unenforceable in both directions. It does not say what is measured, against what, or by whom, so neither party can prove compliance or breach. Six lines fix it.
- The metric, named. Correct-word accuracy or word error rate — and if the latter, whether the reported figure is the rate or 100% minus the rate.
- The normalisation rules. Whether punctuation and capitalisation are scored, how numbers are written, what is excluded.
- Who provides the reference, and to which editorial standard. Verbatim or edited, stated.
- The sample. How much, chosen how, by whom, and before or after delivery.
- What else is in scope. Reading speed, placement and synchronisation need their own numbers; an accuracy figure does not cover them.
- The remedy. Re-work against a re-score, with a stated turnaround, rather than an unspecified credit.
The single most valuable line. State whether the figure is correct-word accuracy or a word error rate subtraction. That one sentence is worth the whole insertion rate — which, on machine-generated captions, is never zero.
Reporting a Reproducible Score
A score someone else can reproduce needs the counts, not the conclusion. Report these together and no one has to ask a follow-up question:
- Reference word count (N), and substitutions, deletions and insertions as separate figures
- The metric used, and the formula as applied
- The normalisation rules, as a list
- Which timecode ranges were sampled, and how they were chosen
- Who produced the reference, and whether it is verbatim or edited
- The deterministic checks run across the whole file, and their results
Splitting the three error types out is the detail that earns its place. Substitutions point at vocabulary and audio quality, and a terminology guide addresses them. Deletions point at passages the engine never attempted, which is usually noise or crosstalk. Insertions point at hallucinated or duplicated output and need a different fix entirely. One percentage tells you how bad; three counts tell you what to do.
Keep the exported QC report alongside the score as the record that the file was checked. An accuracy figure plus a QC report describes a delivery; an accuracy figure alone describes a fraction of one.
What Goes Wrong
| Symptom | Cause | Fix |
|---|---|---|
| Vendor and client scores differ by a few points on the same file | One counted correct words, the other subtracted word error rate | Compare the insertion counts — the gap will equal the insertion rate |
| Accuracy reported above 100% or below 0% | A word error rate above 100%, from insertions, pushed through a subtraction | Report the rate and the raw counts; treat a rate over 100% as a duplicated or looping output |
| A carefully edited file scores worse than a rough verbatim one | Deliberate omissions counted as deletions against a verbatim reference | Score against an edited reference, or use a model that classifies edition errors |
| Scores from two tools disagree on identical input | Different normalisers, which change the denominator as well as the error count | Fix one normalisation rule and re-run both; never compare across rules |
| A high score on a file the client rejected | The failure was placement, reading speed or sync, none of which accuracy measures | Specify and check those separately; accuracy covers part of one quality characteristic |
| The same name misspelled throughout the series | No terminology guide supplied to the recogniser | Build a KNP guide once and attach it to every job on the title |
| A provider's published benchmark not reproduced on your content | Benchmarks are measured on clean audio with general vocabulary | Benchmark two or three providers on your own representative audio |
| Two reviewers score the same sample differently | Severity and edit judgements are not reproducible without training | Use a deterministic metric for routine QC; reserve judgement-based scoring for trained assessors |
| A passing sample on a failing programme | One contiguous window measured one acoustic environment | Several short windows, selected before anyone reads the file |
Measurement Checklist
- Decide which metric applies, and write it down before any measuring starts.
- State whether a reported accuracy figure is correct words or 100% minus word error rate.
- Fix the normalisation rules in writing: punctuation, capitalisation, numbers, non-speech tags.
- Decide verbatim or edited, and build the reference to that standard.
- Transcribe the reference from the finished programme audio, not from the script.
- Have a second person check the reference before it is used to judge anyone.
- Supply a terminology guide to the recogniser before transcription, not after.
- Choose the sample windows before reading the file, and include the hard passages.
- Record substitutions, deletions and insertions separately, never just the total.
- Treat a word error rate above 100% as a signal about the output, not a score.
- Run the deterministic checks — reading speed, duration, gaps, overlaps — across the whole file.
- Check placement, speaker labels and synchronisation separately; no accuracy metric sees them.
- Benchmark providers on your own audio before committing to a number in a contract.
- Report the counts, the formula, the normalisation rules and the sampled ranges together.
Frequently Asked Questions
How is caption accuracy measured?
Two methods are in common use. The first divides correct words by total words in the programme, which is the method behind the familiar 99% figure. The second is word error rate, which adds substitutions, deletions and insertions and divides by the number of words in the reference transcript. Both need a reference transcript, a normalisation rule and a stated sample.
Where does the 99% caption accuracy figure come from?
It appears as a worked example in the FCC's best practices for real-time captioning vendors at 47 CFR § 79.1(k)(2)(iv), which defines accuracy as the percentage of correct words out of total words and illustrates it with 7,000 words less 70 errors. It is a best practice for vendors and an illustration, not a numerical requirement in the caption quality rules.
Is word error rate the same as caption accuracy?
No. Subtracting word error rate from 100% gives a different number from the percentage of correct words, and the gap between them is exactly the insertion rate. A file with 2 substitutions, 3 deletions and 5 insertions in 100 reference words is 95% correct words and 90% by the subtraction, with the 5-point difference being the 5 insertions.
Can word error rate be higher than 100%?
Yes. The denominator is the reference word count, but insertions add words that have no counterpart in the reference, so the numerator can exceed the denominator. A recogniser that loops and emits 120 inserted words against 100 reference words scores a word error rate of 120%, and an accuracy derived by subtraction would read as −20%.
What is the NER model?
A scoring model for live subtitling that subtracts edition errors and recognition errors from the number of words in the subtitles, then divides by that same word count. Separating the two error types lets deliberate editing be judged differently from misrecognition, and each error is graded in severity bands weighted by its impact on comprehension. It requires a trained human assessor.
Does the FCC require a minimum caption accuracy percentage?
No. The caption quality rule at 47 CFR § 79.1(j)(2) sets a descriptive four-part standard covering accuracy, synchronicity, completeness and placement, judged with a de minimis allowance rather than a score. The only percentage in the section is the illustrative 99% inside the separate best practices for real-time vendors. See our FCC requirements guide for the regulatory detail.
What does word error rate not measure?
Punctuation and capitalisation are usually stripped before scoring, so they are not counted. Speaker identification, caption placement, reading speed and synchronisation are outside the metric entirely. Three of those omissions — punctuation, capitalisation and speaker identification — are named in the FCC accuracy standard, so a word error rate score addresses only part of one of the four quality characteristics.
How do you measure the accuracy of edited captions?
Not with a plain word error rate, because every deliberate omission registers as a deletion, so a well-edited file scores worse than a verbatim one. Either score against an edited reference that encodes the editing decisions, or use a model such as NER that classifies edition errors separately from recognition errors.
When should you use character error rate instead of word error rate?
For languages written without spaces between words, where word tokenisation is itself a decision that changes the score, and for work where spelling matters more than word choice, such as proper nouns and product names. Character error rate measures the same edit distance at character level instead of word level.
How accurate is automatic transcription?
It depends on the audio rather than the engine alone. Audio quality, speaking clarity, background noise, overlapping speakers and specialist vocabulary all move the result, which is why a provider's published benchmark rarely predicts your score on your content. Treat an automatic transcript as a first pass to be reviewed, and measure it on your own material before committing to a number.
A percentage is only as good as the protocol behind it. Want help setting a measurement standard your clients will accept? Talk to our team or start a free trial.