Where Else This Goes — The Full Picture

DNA and Genomic Sequences

A DNA strand is a sequence of four characters: A, T, G, and C — the four nucleotide bases. The human genome is approximately 3 billion of these characters long. If that sounds like a sequence that a transformer could learn from, you're right.

Models like DNABERT and the Nucleotide Transformer (trained by InstaDeep and NVIDIA) treat DNA exactly like text — each nucleotide or short k-mer (overlapping chunk of 3-6 bases) is a token, and the model is trained to predict masked or missing positions from their context. What emerges is a model that has learned the statistical grammar of genomes: which sequences appear near gene boundaries, which patterns are conserved across species, which mutations are likely to disrupt protein function.

This has practical consequences. A model trained on millions of genomic sequences can score a novel mutation and estimate how disruptive it is likely to be — not because anyone programmed in the biology, but because the statistical patterns across enough genomes encode that biological knowledge implicitly.

Protein Sequences and Structure

Proteins are chains of amino acids — 20 distinct types, typically hundreds to thousands long. The sequence of amino acids determines the protein's 3D folded shape, and the shape determines its function. Predicting shape from sequence is one of the hardest problems in biology.

Meta's ESM (Evolutionary Scale Modeling) family of protein language models treats amino acid sequences exactly like text — each amino acid is a token, and the model is trained on hundreds of millions of known protein sequences. AlphaFold2, which achieved a historic breakthrough in structure prediction, uses a similar attention-based architecture internally.

What these models learned, without being given any structural data during pre-training, is the coevolutionary patterns in protein sequences — which positions tend to mutate together, which amino acid combinations are compatible, which sequences produce stable folds. That knowledge encodes structural information. The grammar of protein sequences is the grammar of how proteins fold.

Time Series: Sensors, Markets, and Physical Systems

Any instrument that produces readings over time generates a time series: a temperature sensor, a stock price feed, an ECG machine, a power grid load monitor, a seismograph, a vehicle's OBD-II port. Each reading is a token. The sequence of readings over time is the "text."

Transformer models for time series — models like PatchTST, iTransformer, and Temporal Fusion Transformer — have demonstrated strong performance on forecasting tasks across radically different domains. The same model architecture that predicts the next word can predict next hour's electricity demand, next week's server load, the next few seconds of a patient's heart rhythm, or the next day's asset price — with different training data and different tokenization strategies, but the same underlying mechanism.

What makes this work is that physical and economic systems have temporal grammar, just like language. Markets have momentum and mean reversion patterns. Power grids have daily and weekly cycles. Engines have characteristic failure precursors. A model trained on enough examples learns these patterns and can extrapolate them forward.

System Logs and Operational Data

A system log is structured text, but it's more useful to think of it as a behavioral sequence — each log entry is an event, and the sequence of events is the story of what a system is doing. Log lines have implicit grammar: a service starts, then listens, then receives connections, then processes requests, then occasionally throws errors in characteristic patterns before a crash.

Log-based transformers are being actively developed for root cause analysis (what event sequence precedes this class of failure?), anomaly detection (what log sequence is statistically unusual for this service?), and automated incident correlation (which events across which services belong to the same incident thread?).

The appeal over regex-based log parsing and rule-based alerting is the same as for network traffic: the model can detect patterns no one wrote rules for, and can correlate across long time windows and multiple sources simultaneously through its attention mechanism.

Robotics and Physical Action Sequences

A robot arm moving through space generates a sequence of joint angles over time. A self-driving car generates a sequence of control inputs. A human performing a task generates a sequence of body poses. All of these can be tokenized and fed to a transformer.

Models like Google's RT-2 and DeepMind's Gato treat robot actions as tokens in a sequence, mixing them with visual tokens (images from the robot's camera) and language tokens (the instruction being given). A single model can accept a text command, process a camera feed, and output a sequence of motor control tokens — because all three data types have been reduced to the same representation.

This is where the architecture's generality becomes most striking: a model architecture designed to predict the next word in a sentence, extended to predict the next motor command given a visual scene and a language instruction. No fundamental new machinery. Just a broader definition of what a token can be.

What This Means for Fields That Generate Data

The practical implication is that almost any field that generates ordered data — medicine, security, infrastructure, manufacturing, environmental science, finance — is sitting on sequences that a transformer could learn from. The bottleneck is rarely the architecture. It's:

• Data quality and quantity — transformers need substantial, clean, consistently formatted data to learn reliable patterns
• Tokenization design — defining the right atomic unit requires domain expertise; bad tokenization produces bad models regardless of architecture quality
• Supervision signal — the model needs to know what "correct" looks like; in text that's "the next word"; in other domains defining the training objective requires careful thought
• Evaluation — knowing whether the model learned something useful is harder in novel domains than in text, where human judgment provides a natural quality check

None of these are architectural problems. They're data and domain problems. The transformer is ready. The question for any new application is whether the data is ready too.