Covert channels in language models: from the tokens they pick to what they mean

Jerry Buaba · working draft, October 2026 · code and data

Abstract

A language model can carry a hidden message in an ordinary answer by letting the message decide which of several plausible words it writes next. We build an open harness that plants such messages, reads them back out of the plain text, and records what happens inside the model while it writes. On a 0.5B-parameter instruction model, two textbook channels carry 16 bits in a 72-token answer with exact recovery, at a perplexity cost of roughly a third to a half, and the model’s residual stream separates from the clean answer at the first word that carries a bit. Everything on this page is live: one timeline, controlled from the bar at the foot of the page, drives every figure.