01 · Language modelling Real bets from a trained model

It bets on the
next letter.

Underneath everything — every chatbot, every essay it writes — a language model does just one thing: guess the next letter. Not look it up. Bet on it, with odds. Press the button and watch it place its bets, then commit. Here it always takes its top bet — because the odds are all a model ever produces, and how you pick from them is a separate dial, handed to you at a later station. This is the whole idea; the other stations just make the guess smarter.

Its bet for the next letter

That's a language model — the entire trick. Feed it text, it learns which letter tends to follow which, and to write it just keeps betting on the next one and appending its pick. Everything ahead — tokens, attention, training, thinking — is in service of making this one guess better. Watch it long enough and you will catch it repeating itself: that is the price of always taking the top bet, and later you get the dial that fixes it. Next: what a "letter" even is to the model.

uv run python 04_pretrain/typeahead_extract.py # the model's real per-letter bets, always taking its top one

The trained model's own odds · greedy — it always takes the top bet, so nothing here depends on luck and you see exactly what we saw · nothing faked. Built by AgentSure.