A 0.6B model decodes a 31B model's hidden state

Give a photograph to gemma-4-31B. At layer 47 of 60, during the prefill and before a single token has been generated, take one 5376-dimensional vector, pooled over the image token positions.

Qwen3-0.6B receives that vector and nothing else. It is a text-only model, and those 5376 numbers are everything it gets about your photograph. It produces a description of what is in the frame. The same vector passes through one linear map and retrieves from a gallery of 1000 photographs. The host is frozen and was trained for neither job.

Then the control: shuffle that vector's 5376 dimensions and pass the reader identical values with an identical norm. The description collapses. Arrangement is what carries the content.