Skip to main content
Back

Researchers Catch LLMs Looping Into the Same Sentences After 50 Rewrites

A new paper models repeated LLM rephrasing and translation as Markov chains, revealing when outputs collapse into loops versus keep drifting.

Paper, Full Access
  • EMNLP 2026 paper models iterative LLM rephrasing and translation as Markovian generation chains over sentences.
  • Greedy decoding traps chains in fixed points or 2-cycles after just a few steps across all tested models.
  • Sampling decoding (temp 0.7, top-p 0.9) produces much longer transients and many chains never repeat in 50 iterations.
  • Seed sentence length correlates with diversity; GPT-4o-mini on BookSum hits Pearson r of 0.64.
  • Prompted LLM translation drifts far more than Google Translate v3, which stays near-deterministic on fixed inputs.
  • Distinct from training-time model collapse: inference-time behavior can increase diversity, not just reduce it.

Repeatedly feeding an LLM’s output back into the same prompt can produce either a short cycle or dozens of new sentences, depending largely on the decoding method. An EMNLP 2026 Findings paper formalizes this workflow as a Markovian generation chain and measures how quickly repeated paraphrasing and translation begin to recur.

Each run is deliberately memoryless: the model receives a fixed prompt template and the previous output, without chat history. The resulting process forms a time-homogeneous Markov kernel over sentence strings, meaning the next output depends only on the current sentence and a transition rule that stays fixed across iterations.

Why Repeated Inference Needs Its Own Model

Model collapse describes a training-time effect in which repeated use of synthetic data narrows the learned distribution. The behavior studied here occurs during inference, while model parameters remain fixed. Some inputs even gain sentence diversity over successive calls, so training-data collapse does not adequately describe the observed dynamics.

Markov-chain concepts give developers a more precise vocabulary for these pipelines. Transient states are outputs the process leaves behind, recurrent classes are sets of outputs it revisits, and KL contraction measures whether different output distributions become more alike after applying the same transition. These concepts help characterize repeated calls in agent systems, rewriting tools, and translation workflows.

Inside the 50-Step Test

The authors begin with 150 opening sentences drawn from BookSum, ScriptBase-alpha, and BBC News2024. Each sentence passes through 50 rounds of paraphrasing or round-trip translation under the following configurations: