Language-model dynamics

LLM dynamics

I think the frontier models have a lot of extra human-specific stuff encoded through post-training and the more interesting unit of analysis is the base LLM. What does it really mean to accurately predict language across the output of many humans? We have to analyze what language is used for in its full generality: to convey information, but what type of information and how?

I think the fact that we are mapping bit strings to a high-dimensional redundant space is interesting. What does the emptiness correspond to in this continuous embedding space?

Sample from a base language model, say ten pages, perhaps a thousand times, and see what that looks like. Then try to check the trajectory in embedding space, projecting to two or three dimensions.

Dynamical analysis brings in notions of velocity, acceleration and energy, i.e. farther points in space require more energy. What’s interesting is that it’s scale invariant, i.e. what matters are relative distances.

The cool thing about representing language as such is that it becomes potentially much richer, and not necessarily limited to a finite amount of discrete tokens. When sampling vectors we don’t necessarily need to commit to a token, we can just provide the continuous embedding of probabilities. In terms of motion, this is like at each step changing the force (probability) that attracts the object to each token in the 3D space.