Skip to main content
Back

KiToke Keeps 89% Video AI Accuracy While Ditching 99% of Tokens

A training-free token compression method keeps Video LLMs accurate even when 99% of visual tokens are thrown away, with a 12.5x prefill speedup.

  • KiToke is a training-free token compressor for Video LLMs that works at retention ratios as low as 1%.
  • It uses kernel density estimation to score tokens by global diversity across the entire video.
  • Content-adaptive temporal intervals prevent merging tokens across scene cuts or semantic shifts.
  • Hits 89.2% of uncompressed accuracy at 1% retention on LLaVA-OneVision, beating prior SOTA by 7 points.
  • Delivers 12.5x prefill speedup on an A100 with the lowest compression overhead among tested methods.
  • Works across LLaVA-OneVision, LLaVA-Video, and Qwen3-VL from 0.5B to 72B parameters.

Long video inputs can produce tens of thousands of visual tokens. Transformer self-attention compares token pairs, so its compute and memory costs rise roughly with the square of the sequence length. A new training-free method called KiToke selects and merges projected visual tokens before language-model prefill, reducing the sequence without changing model weights. Tests at 1% retention show smaller performance losses than competing token-compression methods.

Why 1% retention exposes weak selectors

A 1% retention budget discards 99 of every 100 tokens, leaving little room for duplicate information. Near-identical views of a static background can consume much of the remaining budget and displace a brief action, small object, or unusual frame. Temporal mistakes add another failure mode: merging tokens across a scene cut combines evidence from unrelated shots.

Existing training-free methods address parts of the problem, but their selection rules become less effective as the budget shrinks:

  • PruneVID treats temporal and spatial redundancy separately, which can miss duplicates spanning both dimensions and perform poorly on dynamic motion.
  • VidCom2 measures token uniqueness against an averaged prototype. Dynamic footage can make that prototype unstable, and the score does not directly control redundancy among retained tokens.
  • FastVID uses density-based ranking within temporal intervals. Rare, informative tokens can receive low priority, while similar tokens still consume the limited budget.

Global density finds scarce evidence

KiToke treats selection as a global diversity problem across the full video. For each visual-token embedding, it sums Gaussian-kernel similarity against every other token. Repeated frames, static backgrounds, and slowly changing regions produce high-density scores because many nearby embeddings look alike. Unusual tokens have fewer close neighbors and receive lower density scores.