Reading the answer straight from the logits

When a chat model answers a multiple-choice question, it writes text, and you hope the text contains one of your options. We take a shorter path.

How it works

A shared renderer turns the state and question into a prompt and labels each option with a letter, A through Z. The model runs once. At the position where an answer would start, we look only at the scores for the valid letters, divide by the fitted temperature, and apply a softmax. That’s the answer: a probability for each option, mapped back to your keys.

Training uses the same renderer, so what the model sees in training matches what it sees in production. Options are shuffled every epoch so it can’t learn that “A” is usually right.

What it buys

• Every answer is valid. The model can only choose among the options you gave it.
• You get a full distribution, not a single guess.
• There’s nothing to parse, so there’s nothing to break.

What it doesn’t buy, yet

Avoiding generation doesn’t by itself make the model fast or cheap. Each question still processes the whole state. Reusing that computation across questions, and a dedicated pointer head, are on our research list.