Module 5, Lecture 5.2 | Section 3: Prompt and Context Engineering
Lecture 5.1 covered prompting principles that apply to any LLM interaction. This lecture narrows the focus to the system prompt specifically — the portion of the context that agent engineers write, own, and are responsible for. The question is not just what to include, but how much, in what order, and why each of those decisions directly affects agent behavior.
The system prompt is not documentation. It is not an ad-hoc configuration. It is an engineering artifact that the agent reads at the beginning of every single turn — a primary mechanism for defining what the agent is, what it can do, and how it must behave.
Unlike user messages, which change turn by turn, the system prompt is authored by the developer and recurs in every API call for the lifetime of the agent. Its cost is paid on every request. Its quality determines the behavioral baseline of the system. It deserves the same engineering discipline as any other critical component: deliberate design, version control, and iterative testing.
A production agent system prompt has four functional sections, ordered deliberately:
The identity section establishes who the agent is and what it is for. This is not a name or a personality — it is a behavioral frame. Telling the model "You are a coding assistant" primes every subsequent token to be generated in the context of how coding assistants respond. This framing is statistically meaningful: the model has seen enormous amounts of text generated by people in specific roles, and a clear role statement activates those patterns.
The identity section should be concise and confident. One or two sentences. No hedging, no qualifications, no motivational language.
Tool documentation is typically the most token-expensive part of the system prompt. It must explain which tools exist, what each one does, when to call it, and what it returns. This section grows as the agent evolves and new capabilities are added.
Because tool descriptions recur on every API call, every token here carries a recurring cost. The tradeoff is real: more detailed descriptions improve compliance; longer descriptions consume more of the shared context budget.
Behavioral guidelines define how the agent should act: communication style, error handling approach, when to ask a clarifying question versus acting directly, how to report results. This section is where the counter-instructions from Lecture 5.1 live — the explicit overrides for verbosity, sycophancy, and over-engineering.
Guidelines express preferences. They can flex under task pressure. When a guideline conflicts with an unusual situation, the model can reasonably deviate. This is different from constraints.
Constraints are non-negotiable. They are the hard limits that cannot be overridden regardless of what the user asks or what the task seems to require. "Never delete files." "Do not execute code." "Only access files within the project directory."
The distinction between a guideline and a constraint matters both semantically and positionally — constraints are placed at the end of the system prompt for a specific structural reason.
The "lost in the middle" phenomenon from Lecture 3.4 applies within the system prompt itself. In a long prompt:
This has direct implications for how to order the four sections:
Identity and critical behavioral rules go first. They establish the frame for everything that follows and receive maximum attention.
Tool documentation goes in the middle. Tools are necessary but tolerant of imperfect compliance. If a tool call is malformed, the API returns an error; the model can adapt on the next turn. Occasional tool call failures are recoverable. Missed identity or constraint instructions are not.
Constraints go last. Placing hard limits at the end of the system prompt primes the model immediately before it begins reading the user's message — the point where it is about to make decisions. This positioning reinforces that constraints are active and non-negotiable.
This ordering is not arbitrary convention. It is a response to empirical attention patterns in transformer-based models.
The context window is a shared resource. Every token in the system prompt is unavailable for conversation history, tool results, and the model's response.
| Component | Typical token range |
|---|---|
| System prompt | 500 – 2,000 |
| Conversation history | 5,000 – 20,000 |
| Tool results (per call) | 100 – 2,000 |
| Reserved for response | 1,000 – 4,000 |
A system prompt that grows to 5,000 tokens because every tool has extensive inline documentation is directly competing with conversation history. As the system prompt expands, something else must shrink — and that something is usually the history the agent needs to reason about what has already happened.
The goal is the minimum system prompt that produces the behavior you need. Not the most thorough, most comprehensive, most exhaustive one. Every token in the system prompt is overhead paid on every turn, in every session, for the life of the agent. Scrutinizing every word is not perfectionism — it is engineering discipline.
A static system prompt contains the same content on every turn. For many agents, this is appropriate: the identity, core tools, and behavioral rules do not change.
A dynamic system prompt injects content programmatically based on context — the current workflow phase, the active user, the tools relevant to the current task. Dynamic injection allows agents to carry a lean static core while loading specialized instructions only when needed.
The principle for now: only put in the static system prompt what the agent needs on every single turn.
Tool documentation for tools the agent rarely uses does not belong in the static prompt — it wastes tokens on every call where that tool is irrelevant. Instructions specific to one phase of a multi-step workflow do not belong in the static prompt. Identity, core behavioral constraints, and always-needed tools do.
Dynamic loading — including the skills-based architecture for on-demand tool injection — is covered in later modules. The static/dynamic distinction here is the conceptual foundation for that work.
When an agent wants to call a tool, it must communicate three things: which tool, with what arguments, and that this is a tool call rather than a response to display to the user.
The naive approach is to instruct the model to emit tool calls as formatted text:
TOOL: read_file
ARGS: {"filename": "main.py"}
Then parse this format in the agent code with string matching or regex. This is brittle. Models do not always follow the format. Partial tool calls appear. Extra text gets mixed in. The parsing code grows into a fragile state machine that breaks in subtle ways.
The Anthropic API — and most modern LLM APIs — handles tool calls as structured objects, not strings. You define tools as JSON schemas:
tools = [{
"name": "read_file",
"description": "Read the contents of a file by path.",
"input_schema": {
"type": "object",
"properties": {
"filename": {"type": "string", "description": "Path to the file"}
},
"required": ["filename"]
}
}]
When the model decides to call read_file, the API response contains a tool_use content block with structured name and input fields — not a string to parse. The agent code inspects the content block directly.
The advantages are significant: no regex, no string matching, schema validation built in, multiple tool calls in a single response natively supported, and — importantly — the model was trained on this format and is more reliable when using it. The SDK is not just a convenience layer; it is a behavioral improvement.
The description field in a tool schema is not documentation for the developer. It is a prompt to the model. The model reads it to decide whether to call this tool, when to call it, and how to form its arguments.
A good tool description answers three questions:
Compare:
| Quality | Example |
|---|---|
| Weak | "Does file stuff." |
| Strong | "Search a file for lines matching a query. Returns matching line numbers and snippets. Use this before read_lines to identify which lines to retrieve." |
The weak description gives the model nothing to work with. The strong description answers all three questions and even specifies the intended calling sequence.
Every word in a tool description is re-sent on every API call. Keep descriptions concise — but complete enough for the model to use the tool correctly. This is the same tension as the system prompt as a whole: more detail improves compliance; more tokens increase overhead.
Patterns from production agent system prompts worth adopting:
Confident role framing. "You are a coding assistant." Not "You are an AI assistant that can help with coding tasks." The first is specific and activates a clear behavioral frame. The second is vague and leaves the output space wide open.
Specific, testable behavioral rules. "When editing files, always read the file first. Never edit a file you haven't read in the current session." This is measurable — you can observe whether the agent follows it. Vague rules ("be careful with files") cannot be evaluated.
Constraints visually distinct from guidelines. Hard limits should be clearly separated — a dedicated ## Constraints section makes them impossible to overlook and signals to the model that these are categorically different from preferences.
No tutorials. A system prompt states what the agent is and what it must do. It does not explain why tool calling works, how the API processes responses, or provide background context the model already has. Tutorial content wastes tokens and dilutes the instructions that matter.
Motivational language. "You are an incredibly helpful assistant who always gives 110%." The model does not need motivation — it needs constraints. Filler tokens crowd out useful instructions without improving behavior.
Contradiction. "Be concise. Provide thorough explanations with complete context." The model resolves this by averaging, producing responses that are neither concise nor thorough.
Vague constraints. "Be careful with user data." Careful how? "Do not store, log, or repeat user-provided data outside the current session" is actionable. The original is not.
Missing error behavior. No instructions for what to do when a tool call fails, a file does not exist, or the user's request is outside the agent's scope. Without explicit guidance, the model will improvise — and improvisation in error conditions is where agent systems produce their most spectacular failures. Explicit error handling instructions — "if a tool call fails, report the error and ask how to proceed" — are some of the highest-value tokens in a system prompt.
Should identity always come first? In most cases, yes. The identity frames everything that follows and should receive maximum attention. Some agents lead with a brief one-line identity and then immediately state critical constraints — the underlying principle is the same: the most important instructions should not be buried.
How much do tool descriptions actually affect reliability? Significantly. The model's decision about which tool to call and how to form arguments is based almost entirely on the description field. Improving tool descriptions — even slightly — often has more impact on agent reliability than changes to the surrounding prompt.
What is the difference between a guideline and a constraint? Guidelines are preferences: "prefer concise responses," "use the simplest approach." The model may deviate from them when the task warrants it. Constraints are hard rules: "never delete files without user confirmation," "only read files within the project directory." They are non-negotiable and should be placed and labeled accordingly.
Should few-shot examples go in the system prompt? Selectively. A few-shot example of the response format the agent should use on every turn belongs in the system prompt — it provides a concrete behavioral template. Examples relevant to only some tasks should be loaded dynamically. Lecture 5.3 demonstrates this with a few-shot example in the coding agent's system prompt.