System prompts. Chain of thought. RAG. These are not exotic features — they're all extensions of the same idea. More context, better shaped context, context that does work before your question even arrives.
Every commercial AI application — every customer service bot, every coding assistant, every document analyzer — has a system prompt. It's a block of context that gets prepended to every conversation, before you type a single word. You usually can't see it. It's always there.
The system prompt is where application builders define the model's persona, constraints, scope, and behavior. "You are a customer support agent for Acme Corp. You only answer questions about our products. You never discuss competitors. You always offer to escalate to a human if the customer is frustrated." None of that is baked into the model. All of it is context — loaded before the user speaks.
This is why the same underlying model can feel like completely different products depending on where you access it. The weights are the same. The system prompts are radically different, steering the same probability distribution toward completely different behavioral neighborhoods.
When you build anything with an AI API — a tool, an automation, a workflow — the system prompt is your configuration file. It's where you invest the most design effort. A well-crafted system prompt does the prompting work for every user interaction that follows, without requiring users to understand prompting at all.
A language model generates tokens sequentially, left to right. Each token is produced based on all the tokens before it — including the tokens it has already generated in the current response. This means you can use the model's own output as additional context for its subsequent reasoning.
Chain of thought prompting exploits this directly. Instead of asking for an answer, you ask the model to reason through the problem step by step first, and then give the answer.
── WITHOUT CHAIN OF THOUGHT ─────────────────────────────── Q: A server rack has 42U of space. Each 1U server draws 200W. A 2U server draws 450W. We need 20 servers total: 15 of the 1U type and 5 of the 2U type. What's our total draw? A: 4,250W ── WITH CHAIN OF THOUGHT ────────────────────────────────── Q: [same question] Think through this step by step before giving your final answer. A: First, calculate draw from 1U servers: 15 × 200W = 3,000W. Next, calculate draw from 2U servers: 5 × 450W = 2,250W. Total: 3,000 + 2,250 = 5,250W. Final answer: 5,250W.
The first response is wrong. The second is right. The only difference is the instruction to reason explicitly. Because the model generates the reasoning tokens before it generates the answer token, those reasoning tokens become additional context that improves the answer's accuracy. The model isn't smarter — the context is doing more work.
Multi-step problems are hard to solve in one leap, for the same reason humans find them hard. Showing the intermediate work keeps each step grounded in correct prior steps. Errors in the reasoning become visible and correctable — by the model itself, or by you. "Think step by step" is one of the highest-leverage phrases in prompting.
RAG — Retrieval Augmented Generation — sounds like a complex system, and the engineering to implement it can be. The concept is simple: when the model lacks the knowledge needed to answer a question, retrieve the relevant information from somewhere that has it, and inject it into the context window.
The flow looks like this:
RAG is how AI applications stay current without retraining, how they answer questions about private company data without that data being in the training set, and how they cite specific sources rather than generating plausible-sounding information from memory.
From the model's perspective, RAG is just a longer context window. It has no awareness that the context was retrieved — it simply reads the document it's given and uses what's there. The intelligence is in what gets retrieved and how it's formatted before being injected. Context, again, doing the work.
A model answering from training data alone will sometimes generate plausible-sounding but incorrect information — because there's nothing to anchor it. A model given the relevant source document in its context will answer from that document. The statistical pull toward the grounded content in the context window is strong. RAG doesn't eliminate hallucination, but it substantially reduces it by giving the model something accurate to attend to.
When you need machine-readable output — JSON, CSV, a specific schema — context that establishes the exact format is critical. The most reliable approach is to show the model a complete example of the output structure, then ask it to fill it in.
── INSTEAD OF ────────────────────────────────────────────── Extract the key information from this incident report as JSON. ── USE ───────────────────────────────────────────────────── Extract the key information from this incident report. Return ONLY valid JSON in exactly this structure, no other text: { "incident_id": "string", "start_time": "ISO 8601 datetime", "duration_minutes": number, "affected_services": ["string"], "root_cause": "string", "customer_impact": "string", "resolved": boolean }
The schema example in the prompt is context. It shifts the token probability distribution heavily toward producing tokens that match that schema. "Return ONLY valid JSON" is a negative constraint — it reduces the probability of preamble text, explanations, or markdown code fences that would break downstream parsing.
System prompts. Chain of thought. RAG. Structured output. Few-shot examples. Role assignment. Negative constraints. Every prompting technique, no matter how it's labeled, is a different strategy for getting the right information into the context window in the right shape.
There's no magic. There are no prompt spells. The model is a frozen probability machine. The context window is the only input. Every technique is just a way of making that input richer, clearer, and more precisely targeted at the output you need.
The practitioners who get the most out of these systems are not the ones who know the most tricks. They're the ones who have internalized this model deeply enough that constructing good context becomes intuitive — who understand, before they type a word, that the model knows nothing about their situation except what they're about to tell it.
The quality of the output is bounded by the quality of the context. A better model raises the ceiling. Better context determines how close you get to it. Both matter — but you control one of them completely, and it's the one most people underinvest in.