VoxCPM

VoxCPM2 is the current major release of OpenBMB's tokenizer-free speech-generation stack. The 2-billion-parameter model is described as trained on more than two million hours of multilingual speech and supports 30 languages, nine listed Chinese dialect groups, text-only voice design, reference-based voice cloning, reference-plus-transcript continuation, streaming generation, SFT/LoRA adaptation and native 48kHz output. Code and weights are released under Apache-2.0.

Those capabilities make it relevant to self-hosted narration, multilingual assistants, game characters, accessibility prototypes and controlled branded voices. They also make identity governance essential. A technically open model is not permission to copy a person's voice, mislead listeners or use recordings outside their consent.

Four generation modes that solve different jobs

Choose a mode by the product requirement, not by which demo sounds most impressive. A fictional character may be safer and more controllable with voice design. A contracted actor may need cloning for continuity, but only inside the agreed languages, scripts, media, regions and term. Reference-plus-transcript continuation is useful when similarity is important and the transcript can be verified exactly.

Architecture and what “tokenizer-free” means

VoxCPM2 generates continuous speech representations rather than first converting audio into a sequence of discrete codec tokens. The official material describes a diffusion-autoregressive system operating in AudioVAE V2 latent space through LocEnc, TSLM, RALM and LocDiT stages, using MiniCPM-4 as its backbone. The language-model token rate is listed as 6.25 Hz.

Tokenizer-free does not mean text processing disappears, nor that pronunciation is automatically correct. Text normalization, numbers, abbreviations, names, punctuation, code-switching and language-specific reading conventions still shape output. The model can infer prosody from context, but production scripts should make pronunciation and pauses explicit where mistakes matter.

Current model facts and their limits

Quick start

pip install voxcpm

from voxcpm import VoxCPM
import soundfile as sf

model = VoxCPM.from_pretrained(
    "openbmb/VoxCPM2",
    load_denoiser=False,
)
wav = model.generate(
    text="A short, reviewed production test.",
    cfg_value=2.0,
    inference_timesteps=10,
    seed=42,
)
sf.write("test.wav", wav, model.tts_model.sample_rate)

The repository lists Python 3.10–3.12, PyTorch 2.5 or newer and CUDA 12 or newer for the primary path. It also documents CPU, MPS and CUDA choices for the demo, Nano-vLLM for throughput, vLLM-Omni for multi-tenant serving and an independent llama.cpp-omni GGUF path for CPU, Metal, CUDA or Vulkan. Treat each runtime as a separate product configuration with its own numerical output, latency, supported flags and maintenance status.

Reference-audio checklist

• Obtain written consent that names the speaker, permitted use, languages, media, audience, geography, duration, model training/cloning and revocation path.
• Capture clean, single-speaker audio without music, room echo, effects, compression pumping or another voice.
• Preserve natural variation but avoid a clip whose emotion conflicts with the desired neutral identity baseline.
• For continuation cloning, transcribe every spoken word, filler and relevant punctuation exactly; do not “clean up” the transcript.
• Hash the source file, encrypt it, restrict access and keep consent metadata next to—not embedded ambiguously inside—the asset workflow.
• Test whether the reference contains sensitive statements, private background speech or metadata that should be removed.

How to evaluate speech quality

The repository publishes Seed-TTS-eval, multilingual and instruction-control results. Those tables are useful for forming hypotheses, not replacing a private evaluation. Some results are internal and use automated transcription or similarity models; public benchmark averages can hide accents, numbers and domains that matter to your product.

A production evaluation protocol

1. Define the voice contract. Specify identity, allowable styles, pronunciation guide, disclosure and prohibited content.
2. Build a multilingual test set. Include native reviewers, code-switching, names, dates, currencies, acronyms, emotional shifts and difficult punctuation.
3. Run fixed and varied seeds. A fixed seed supports regression tests; varied seeds reveal generation variance.
4. Compare modes. Test voice design, basic cloning and transcript-conditioned continuation only where each is legally permitted.
5. Segment long scripts. Split at semantic boundaries, maintain a style state and listen across joins.
6. Load test the chosen runtime. Capture GPU memory, first-chunk latency, RTF, throughput, queueing and failure recovery.
7. Red-team identity abuse. Try public-figure clips, mismatched consent, prohibited scripts and attempts to remove disclosure.
8. Archive evidence. Store model/revision, runtime, prompt, seed, settings, reference hash, reviewer, consent and output hash.

Deployment choices

Consent, disclosure and incident response

Apache-2.0 governs the released software and weights. It does not grant publicity rights, copyright in scripts or recordings, labor rights, biometric consent, trademark permission or the right to deceive. Requirements differ by jurisdiction and context, so obtain qualified legal review for real-person cloning and high-impact uses.

• Block cloning until consent scope and speaker identity are verified.
• Label synthetic audio in the user interface and content where listeners could reasonably be misled.
• Use provenance metadata or watermarking where available, while recognizing it can be stripped.
• Prevent voice assets from being exported or reused outside authorized projects.
• Create a rapid takedown, revocation and incident process with searchable output provenance.
• Never deploy cloned speech for authentication, emergency instructions, political persuasion, financial authorization or impersonation without specialized controls and legal basis.

Alternatives

Frequently asked questions

Is VoxCPM2 free for commercial use?

The official repository and model card use Apache-2.0. Commercial deployment still needs rights to text, reference recordings, speaker identity and intended use.

Does 48kHz guarantee studio quality?

No. Sample rate describes output bandwidth, not pronunciation, performance, recording cleanliness or absence of artifacts. Listen and measure the complete signal.

How much GPU memory is required?

The official comparison lists about 8 GB for VoxCPM2. Runtime, concurrency, input length and serving engine affect actual peak memory; test with headroom.

Can it run on Apple Silicon?

The demo documents MPS selection, and llama.cpp-omni documents Metal/GGUF execution. The repository reports an illustrative RTF around 1.76 for Q8_0 on an Apple M4 Pro; reproduce on your machine.

How much reference audio is needed?

The model supports cloning from a short clip, while fine-tuning guidance says 5–10 minutes can adapt a speaker/domain. Clean, representative, consented audio matters more than a universal duration.

Are all 30 languages equally good?

No. The official limitations explicitly say performance varies with training-data availability. Use native reviewers and domain-specific scripts for each locale.

Primary sources

• Official VoxCPM repository, setup, benchmarks and limitations
• Official VoxCPM2 model card
• Official VoxCPM documentation
• VoxCPM technical report
• Official audio samples and project page
• Apache-2.0 license file
• vLLM-Omni serving project
• llama.cpp-omni edge inference project

Last reviewed July 25, 2026. Hardware, benchmark and training claims are attributed to the official project. Verify the current revision, dependencies, laws and consent contract before production use.