Everything we've covered — attention, training, inference — was explained using text. But the transformer architecture has no opinion about text. It only requires one thing from your data. And that one requirement opens the door to everything.
A transformer model has exactly one requirement for its input data: it must be representable as a sequence of tokens.
That's it. The architecture doesn't know or care what those tokens represent. It doesn't know that words mean things, that pixels form images, or that frequencies make sound. It sees numbers. It learns relationships between those numbers by predicting what comes next. It generalizes those relationships into something useful.
The field of AI spent years assuming language models were specifically about language. Then researchers started asking: what if we tokenize something else?
Any data that can be broken into an ordered sequence of chunks can be fed into a transformer. Images, audio, DNA, protein structures, network packets, sensor readings, musical notes, molecular graphs — all of these have been tokenized and trained on using the same fundamental architecture that powers language models.
In the text world, tokenization means breaking a sentence into words or word-fragments. Each fragment gets a number. The model sees the numbers.
But tokenization is really just the answer to one question: what is the atomic unit of your data?
For text, the atom is a word or subword. For images, it could be a small patch of pixels. For audio, it could be a short slice of a spectrogram. For DNA, it could be a single nucleotide — A, T, G, or C. For network traffic, it could be a single packet header. For music, it could be a single note event.
Once you've decided what your atom is and assigned it a number, the rest of the pipeline is identical to what you'd use for language.
When a language model trains on text, it learns the statistical structure of language — which words follow which, how ideas connect, what a coherent sentence looks like. It learns this without anyone explicitly teaching it grammar.
The same thing happens with any other data type. Train a transformer on images, and it learns the statistical structure of visual scenes — which patches tend to appear near which other patches, what patterns characterize a dog versus a chair. Train it on DNA sequences, and it learns the statistical structure of genomes — which nucleotide combinations are common, which patterns appear near gene boundaries.
In every case, the model learns the grammar of the data domain — not because anyone defined that grammar, but because it emerges from predicting what comes next across enough examples.
Fields that have never had useful AI tools — genomics, proteomics, network security, materials science — are suddenly seeing breakthroughs, not because someone invented new AI for those fields, but because researchers recognized that their data was just another sequence, and applied the same transformer architecture that was already working for text.
We'll walk through five data types that illustrate how far this generalizes:
The goal is not to survey every application. It's to internalize a single idea: the transformer is a general-purpose sequence learner, and the decision of what constitutes a "token" is the only domain-specific choice you have to make.
Before moving on: what data do you work with that has a sequential, ordered structure? Log files? Packet captures? Call records? Sensor readings? By the end of this track, you'll have a framework for thinking about whether that data could be treated as a language — and what you might learn from a model trained on it.