Geniuses speak in full sentences


One thing one notices in descriptions of geniuses—and parenthetically, I suppose one should put quotations around both "thing" and "geniuses"—is that they are often described as speaking in complete paragraphs, as opposed to you and I, who merely speak in complete sentences.

When I am composing actual writing, or dictating something knowing that I know will turn into writing, I hardly ever achieve that. I have very noticeable pauses in between clauses, not even at the level of complete phrases (...this being a good example, as during dictation there was a lengthy pause just then). Even though in casual speech I generally do not pause for a great deal of time between phrases, clauses, or sentences, I still do not generate them in parallel.

At the other extreme, at the atomic level, one thinks of the atom of communication being the word (or, I suppose, the token). Aside from those moments when a word is at the tip of your tongue or you do not know it, the word generally appears unbidden in your mind.

I have been thinking not about the process, but about the characteristics of LLM generation. Should we be thinking of them as strictly serial because the model generates only a token at a time? Or should we be thinking about the parallel aspects associated with the attention vector? Although the vector describes a single point in this high-dimensional concept space, that point is attached to the next point, and the next point, and the next point—which is to say, the next token, and the next token, and the next token.

The LLM conversation is actually a trajectory in this high-dimensional space. It does have an arrow in time—not strictly monotonically, but an arrow in time nonetheless. One token always produces the next token, produces the next token, produces the next token. If we were to graph it over t, we would have a function and not a statement about geometry.