The model starts out knowing absolutely nothing. Training is the process of making it progressively less wrong until it's useful. Here's how that actually works.
Before training begins, a language model is initialized with random values — essentially noise. If you asked it to complete a sentence at this stage, it would produce incoherent garbage. Every word would be equally likely. It has no concept of language, meaning, or structure.
Training is the process of replacing that noise with structure.
Training works by playing a game, millions of times over. The model is shown a piece of text with the last word hidden, and it has to guess what the hidden word is.
For example: "The coffee was hot and tasted ___"
The model produces a guess. Early in training, the guess is essentially random — maybe it says "elephant." The training process then reveals the correct answer — say, "bitter" — and penalizes the model for being wrong. The penalty is proportional to how wrong it was.
Then the model's internal values are adjusted — just slightly — to make it a little more likely to produce "bitter" (or similar words) in that context next time. This adjustment is tiny. But multiply it across billions of examples, and the model gradually learns the statistical fabric of language.
No one tells the model what grammar is. No one explains what a verb is, or what a sentence means. The model infers all of it indirectly — as a side effect of getting better at guessing the next word across trillions of examples. Language structure emerges from the statistics.
Inside the model are billions of numbers — called weights. You can think of them as dials, each controlling how much influence one piece of information has on the model's output.
Training turns those dials. Every wrong guess triggers a calculation that determines which dials contributed to the mistake and by how much. Each of those dials is adjusted — slightly toward being more right next time.
After enough examples, the dials have been adjusted so many times that they've settled into values that produce good predictions across the full breadth of human language. The model hasn't memorized the training data — it has absorbed the patterns in it.
A large language model might train on trillions of words. The training run might last weeks, on thousands of specialized chips running in parallel. The compute cost for a frontier model can run into tens or hundreds of millions of dollars. This is why only a handful of organizations in the world can train frontier models from scratch.
At some point, training stops. The dials are locked in place. The model's weights are frozen. From that moment on, the model cannot learn anything new from conversations, from new events in the world, or from feedback you give it during a chat session.
When you interact with an AI and it says something wrong, and you correct it — it appears to update within that conversation. But that's just the conversation history being fed back into the same frozen model. The correction lives in the chat window. It does not change the model's weights. The next person who talks to the model starts from the same frozen baseline.
After the main training run, a model is often fine-tuned — trained again on a much smaller, more curated dataset to shape its behavior. This is how a raw text-prediction engine becomes a helpful assistant: human trainers rate model responses, and additional training nudges the model toward responses that humans rated as better.
Fine-tuning doesn't rebuild the model. It takes all those carefully calibrated dials and turns a subset of them a little further in a specific direction. The vast majority of what the model knows comes from the original training run. Fine-tuning is the last mile — shaping personality, tone, and safety behavior.
If training is just "guess the next word, get penalized for being wrong, adjust slightly" — repeated billions of times — what happens if the training data contains a lot of incorrect information? What does "correct" even mean when the model has no external reference for truth, only the patterns in the data it was given?