Skip to main content
Back

IBM Research's CAST Benchmark Reveals TTS Systems Can't Stress the Right Words

A new benchmark called CAST shows that state-of-the-art text-to-speech models fail to place emphasis on the right word based on conversational context.

  • CAST benchmark tests whether TTS systems can infer which word to stress from conversational context alone.
  • 113 contrastive pairs where identical sentences require different stress based on preceding discourse.
  • All evaluated TTS systems (CosyVoice3, Qwen3-TTS, GPT-4o-mini-tts, Higgs Audio V2, Chatterbox, Kokoro) score near zero on strict Pair-Correct metric.
  • Text-only Claude Haiku hits 88.1% Contrast and 76.1% Correct on the same items, proving the signal is recoverable from text.
  • Explicit stress markup improves results (CosyVoice reaches 40.3 Pair-Contrast), but context alone does not.
  • Team releases benchmark, evaluation code, generation pipeline, and a 10k synthetic corpus: arxiv.org/abs/2604.10580.

Say the sentence She published the essay in the winter issue out loud. Now say it again, but imagine you are correcting someone who thought the essay ran in spring. The words are identical, but the stressed word shifts, and so does the meaning. A new benchmark from The Hebrew University of Jerusalem and IBM Research argues that modern text-to-speech systems, despite sounding remarkably natural, mostly cannot make that shift on their own.

The paper, Knowing What to Stress, introduces CAST (Context-Aware Stress TTS), a benchmark that isolates one specific question: given only a conversational context and a sentence, can a TTS model figure out which word to emphasize, and actually emphasize it in the audio? Text-only language models reliably recover the intended emphasis from context, yet TTS systems frequently fail to realize it in speech.

The gap between saying and stressing

Prosody, the melody and rhythm of speech, does real semantic work. Sentence stress refers to emphasis placed on particular words or phrases and can dramatically change meaning for the same written form. In many conversational settings, the appropriate prosody depends on discourse context rather than the sentence itself.

Prior benchmarks have looked at pieces of this problem, like detecting stress from recorded speech, or realizing it when a target word is explicitly marked with control tokens. What none of them measured is whether a TTS system can infer the stress target purely from surrounding dialogue and then produce it acoustically. That is the hole CAST fills.