Whisper

Whisper is OpenAI's open-weight family of encoder-decoder speech models and the name of its MIT-licensed Python reference implementation. It transcribes speech in the source language, detects spoken language and—when a compatible multilingual checkpoint is used—can translate speech into English. The repository is a developer component, not a hosted transcription website: installation, compute, file handling, security, batching and transcript review remain the operator's responsibility.

Three similarly named things are often confused. openai-whisper is the Python package that runs downloadable checkpoints. whisper-1 is a separately operated model in OpenAI's hosted Audio API. Applications such as whisper.cpp, faster-whisper, WhisperX and browser demos are independent implementations or pipelines built around Whisper-family weights. A fair review must specify which layer it is evaluating.

The decision in one sentence

Choose upstream Whisper when you need a transparent baseline, local control and broad ecosystem compatibility—and can own the compute and quality-assurance loop. Choose a managed API when operations matter more than keeping inference beside the audio. Choose a specialized runtime when throughput, CPU deployment, word alignment or diarization is the main constraint.

The project's durable advantage is deployment optionality, not a guarantee that it tops every modern speech benchmark. The same model family can run in a research notebook, an offline workstation, a GPU batch worker or a third-party edge runtime. That makes Whisper a useful common baseline for evaluating a speech pipeline, but it also means performance claims made by one wrapper or hardware stack should not be attributed automatically to upstream Whisper.

What is included—and what is not

Whisper is not a speaker-diarization system, meeting editor, consent manager or compliance product. It also does not supply a production real-time service out of the box. Those capabilities can be assembled around it, but they should not be represented as native features.

Model and deployment choices

*OpenAI's README reports relative speed for English transcription on an A100 versus the large model. Real speed changes with language, audio, hardware and runtime; the figures are not universal latency promises. The README values above were checked 2026-08-20.

The four smaller sizes have both multilingual and .en checkpoints. OpenAI notes that English-only variants tend to help most at tiny and base, with a smaller difference at small and medium. Start with turbo for general transcription on a capable GPU, small for a more economical multilingual baseline, or an .en checkpoint when the corpus is strictly English. Use medium or large for speech-to-English translation because the optimized turbo checkpoint returns the original language even when a translate task is requested.

“Supports 98 languages” describes tokenizer/model coverage, not equal accuracy. OpenAI's model card says results vary with language, accent, dialect and training-data availability. Treat language detection as a prediction: for known monolingual jobs, supplying the language can remove one source of error. For mixed-language recordings, evaluate code-switching explicitly rather than assuming a single detected label is sufficient.

Local installation and a reproducible first run

The upstream route is intentionally plain: install the openai-whisper package, make the ffmpeg command available, download a checkpoint and run the CLI or Python API. The current project metadata requires Python 3.8 or newer and lists classifiers through Python 3.13, while the README's compatibility text is more conservative. Pin the exact package commit and model name in production instead of silently following the repository head.

python -m venv .venv
source .venv/bin/activate
pip install -U openai-whisper

# ffmpeg must also be installed by the operating system
whisper interview.wav --model turbo --language English   --output_format all --output_dir transcripts

The CLI can write TXT, VTT, SRT, TSV and JSON. The Python transcribe() method processes audio through a sliding 30-second window and returns combined text plus segments and detected language. First-run time includes model download; measure that separately from warm inference. CPU execution is possible but can be slow, especially for larger checkpoints, and upstream GPU memory estimates are only planning figures.

A production-minded batch and subtitle workflow

1. Define the decision. Decide whether the output is searchable notes, draft subtitles, a publishable transcript or evidence used in a high-impact process. The review burden changes accordingly.
2. Build a private test set. Include clean speech, noise, silence, music, accents, code-switching, proper nouns, numbers and the longest expected recording. Produce human-verified references.
3. Separate audio preparation from recognition. Preserve an original copy. Decode consistently, avoid arbitrary enhancement that changes speech, and introduce voice activity detection only after measuring its effect on clipped words and hallucinations.
4. Run controlled model trials. Hold hardware, runtime, language hint and decoding settings constant while comparing two or three model sizes. Record model revision and real-time factor.
5. Inspect failure classes. Score names and numbers separately; flag repeated phrases, empty/silent sections with text, language switches and timestamp drift. Average WER alone can hide the errors that matter.
6. Create the delivery format. Generate SRT/VTT for captions or JSON for downstream processing, then enforce line length, reading speed, punctuation and speaker-label rules in a separate step.
7. Require human review where consequence is high. Medical, legal, employment, research quotations and accessibility publication need source-audio verification, not confidence in a model name.

audio + consent
      ↓
decode / optional VAD ── preserve original
      ↓
model + language + recorded settings
      ↓
segments / timestamps / confidence signals
      ↓
silence · repetition · names · numbers · drift checks
      ↓
human review ── export ── retention/deletion

Accuracy, hallucination and evaluation

Whisper's paper established strong zero-shot robustness from 680,000 hours of weakly supervised multilingual and multitask audio. That training strategy is also central to its limitations. The official model card warns that Whisper may produce text that was not spoken, repeat text, perform unevenly across languages and show demographic differences across accents and dialects. Hallucination is particularly important around silence, music, noise and uncertain speech because the decoder can generate linguistically plausible continuations.

We do not publish a single “Whisper WER” here because it would be misleading without checkpoint, language, dataset, preprocessing and decoding configuration. The official language chart above and appendices to the paper are the right sources for reported benchmarks; a production decision should be based on a versioned private evaluation.

Prompts can bias spelling toward domain terms and proper nouns, but they are not a guarantee and may themselves be echoed or influence unrelated text. Treat prompts as one tested configuration. The upstream code exposes thresholds, temperature fallback, previous-text conditioning, word timestamps and a hallucination-silence option; changing them without a fixed test set simply trades one error pattern for another.

Privacy, safety and operational limits

Whisper versus adjacent choices

OpenAI's current API guide recommends gpt-transcribe for ordinary recorded-speech transcription and retains whisper-1 for needs such as word/segment timestamps, subtitle formats or speech-to-English translation. This does not replace the open-source repository; it is a separate procurement and architecture decision. As checked 2026-08-20, the official API pricing table listed estimated transcription costs of $0.0045/minute for gpt-transcribe, $0.006/minute for gpt-4o-transcribe and $0.003/minute for gpt-4o-mini-transcribe. Prices and model availability are dynamic—recheck the official page before budgeting. The API guide also states a 25 MB file limit for transcription uploads.

FAQ

Is Whisper free?

The upstream code and released model files use the MIT license, so there is no per-minute license fee for running them yourself. Self-hosting still costs compute, storage, engineering and review. OpenAI's hosted transcription API is a separate paid service.

Does Whisper work offline?

Yes, after the package, dependencies and selected model files are present locally, inference can run without sending audio to an API. Verify the complete application, model download path, telemetry and storage policy before claiming an air-gapped workflow.

Which Whisper model should I use?

Start with turbo for fast GPU transcription, small for a balanced multilingual baseline or an .en model for English-only audio. Compare at least two sizes on your own recordings. Use medium or large—not turbo—when translating speech into English.

Does Whisper identify speakers?

No. It transcribes speech but does not natively diarize speakers. WhisperX and other pipelines can combine Whisper with separate alignment and diarization models, which require their own evaluation and permissions.

Can Whisper transcribe in real time?

Not with the upstream package as a ready-made streaming service. Third-party runtimes can process chunks near real time, but microphone capture, voice activity detection, context handling, latency and correction behavior belong to those systems.

Why does Whisper invent or repeat text?

The official model card identifies hallucinated and repetitive text as known sequence-to-sequence failure modes, especially where audio evidence is weak. Include silence and music in tests, inspect decoding signals and require human review instead of assuming fluent output is faithful.

Sources reviewed

• OpenAI Whisper official repository
• Official README: setup, models, languages and CLI
• Official model card: intended use, training data and limitations
• MIT license
• Whisper research paper
• OpenAI Audio API transcription guide
• OpenAI API pricing
• faster-whisper repository
• whisper.cpp repository
• WhisperX repository and paper links

Independent technical review: 2026-08-20. Model revisions, repository instructions, API models and prices can change. Verify the linked primary sources and run a private evaluation before deployment.