The dangerous zone isn't where text models obviously fail — it's where they appear to succeed. A confident, fluent, plausible-sounding wrong answer is worse than an obvious error. Here's where that happens most often.
When you ask ChatGPT or Claude to "generate an image," the text model is not generating the image. It can't. A text model produces tokens. Tokens are not pixels.
What's actually happening: the text model interprets your request, refines or restructures it into a prompt optimized for an image generation system — typically a diffusion model like DALL-E, Stable Diffusion, or Imagen — and that separate system produces the actual image. The text model is a translator and intermediary. The image comes from a completely different architecture trained on completely different data using completely different methods.
This matters because the limitations you encounter in image generation are not text model limitations — they're diffusion model limitations. Difficulty rendering text accurately in images, issues with hands and fingers, challenges with precise spatial relationships — these are properties of how image generation models work, not the text model that prompted them. Asking the text model to "fix" these issues by rephrasing the prompt is the right approach. Expecting the text model to directly control the pixel-level output is a misunderstanding of the architecture.
Where text models genuinely add value in image workflows: prompt engineering (translating your vague creative intent into the specific, structured language that image models respond to well), analyzing an existing image and describing it in text, and iteratively refining prompts based on output feedback. The text model is the translator. The image model is the painter. They're not the same system.
As we covered in the Beyond Text track, network traffic can be treated as a language — but only if you build a model specifically trained on network data, using network events as tokens, with network behavioral patterns as the grammar it learns.
Feeding raw packet captures, log files, or network event data to a general text language model and asking it to "analyze this for anomalies" is a category error with three specific failure modes:
Anomaly detection requires knowing what normal looks like for your specific environment. A text model has no knowledge of your network's normal behavior. It can describe what packet data looks like in general terms — because packet capture formats appear in documentation and blog posts in its training data — but it cannot identify that this particular traffic pattern is anomalous for your particular environment at 2 AM on a Tuesday. That requires a model trained on your data.
When a text model "reads" a packet capture, it's reading it the way it reads everything else — as a sequence of tokens representing characters. The numerical values, the bit flags, the protocol fields — these aren't being parsed as network protocol structures. They're being tokenized as text. The model may generate plausible-sounding analysis because packet capture documentation is in its training data. That analysis is linguistic pattern matching, not network protocol analysis.
This is the critical one. A specialized network anomaly detection model returns a score or a flag. A text model returns a paragraph that sounds authoritative whether it is or isn't. It may identify "suspicious" traffic based on surface-level text similarity to security documentation it has seen — flagging things that are benign and missing things that matter. Confident fluency is not accuracy.
Using a text model for security analysis is not just ineffective — it can create false confidence. "The AI reviewed the logs and didn't flag anything" is not a meaningful security assurance if the AI was doing linguistic pattern matching rather than behavioral analysis against a trained baseline. The absence of a flag means the text model didn't recognize anything it had words for. It doesn't mean nothing was wrong.
You've probably seen this: ask an AI to do multi-step arithmetic and it sometimes gets it wrong. Not because language models are bad at math in general — they can handle a surprising range of mathematical reasoning when it's expressed in language. But when computation requires precise, deterministic arithmetic across many steps, the probabilistic token-prediction mechanism breaks down.
The model doesn't compute 7,493 × 8,271 by doing multiplication. It generates the tokens that statistically tend to follow that expression — which is usually close to right for simple cases and increasingly unreliable for complex ones. There's no internal calculator. There's no carry operation. There's next-token prediction, which happens to work well for common arithmetic patterns and fails on edge cases.
The right tool for arithmetic is a calculator or a code interpreter. The right tool for querying a structured database is SQL. The right tool for transforming structured data is a programming language. A text model can write the SQL or the code — that's language generation, its strong suit — but the execution should happen in systems designed for precision, not in the model's probabilistic token space.
A text model processes a fixed context window and generates a response. It has no concept of a data stream, no ability to monitor continuously, and no way to trigger on events. Asking a text model to "watch this feed and alert me when something changes" is architecturally impossible — the model doesn't run continuously. It runs once per prompt and stops.
Real-time monitoring, streaming analytics, and event-driven alerting are jobs for systems designed around continuous execution: stream processors, monitoring platforms, rule engines, dedicated ML models running inference at the edge. A text model can be part of the response workflow — "when the monitoring system fires an alert, have the text model draft a plain-English summary of the incident" — but it cannot be the monitoring system itself.
Every mismatch on this page has the same shape: the task requires determinism, precision, real-time data, or domain-specific training that the general text model was never given — and the model doesn't flag this. It produces output anyway. The output sounds coherent. The output may be wrong. The model cannot reliably distinguish between the two. That's not a bug. It's the natural consequence of using a probabilistic language pattern matcher for tasks that require something else.