This lecture closes Module 6 by connecting everything we have built — the agent loop, real tools, the tool registry — to the subject of the next section: context management. Once an agent can call tools, its real work happens in long, multi-step cycles of tool calls. Understanding how that behavior emerges, how it fails, and how to measure its cost is what prepares us to manage context in Module 7.
Consider a task that no single tool can satisfy: "Find all the Python files in this directory and add a one-line docstring to any function that doesn't have one." Nothing in the system prompt tells the agent how to do this. It describes three tools and a rule or two. Yet the agent works through it: it lists the directory to discover the files, reads each Python file, identifies the functions that lack docstrings, and edits them one by one before moving to the next file.
It is tempting to say the model "made a plan." It did not. The essential thing to understand about an LLM in an agent loop is that it does not generate a plan for itself unless you ask it to — it generates the next tool call. At each turn of the inner loop the model sees the full conversation so far (the user's request, every previous tool call, every tool result) and produces the single most likely next action given that context. The loop executes that action, appends the result, and the model predicts again from the updated context.
What makes the multi-step behavior possible is that tool results re-enter the context. When the agent calls list_files, the directory listing becomes a new input. On the next turn the model reads the original request and that listing, and only now can it decide which file to read first. One tool call at a time, each result feeding the next decision, the agent walks through the task.
The effect is an emergent chain of tool calls. It looks as though the model designed an elaborate plan up front, but that is an illusion. There is no persistent plan object and no task queue. The "plan" exists only in the pattern of tool calls the model produces, step by step, from its context.
This emergent quality is a strength as much as a property to be aware of. Because the model decides each step fresh, it can improvise: when a tool returns something unexpected, the agent adapts and chooses a different next call rather than blindly executing a pre-committed script. A program that planned everything in advance could not respond that way.
It also has a limitation hiding inside it. Because the plan is implicit rather than stored, the model can lose it. On a long task, earlier reasoning scrolls toward the edges of the context window, attention spreads thin, and the agent can forget what it was doing. The plan was never written down anywhere durable — so when the context degrades, the plan degrades with it.
The chain is not strictly one-call-at-a-time in every case. A single API response can contain several tool_use blocks when the model judges the calls to be independent — for example, reading two unrelated files. The dispatch code from the previous lectures already handles this, because it iterates over every content block in the response and executes each tool call it finds. What remains sequential is the API calls themselves: you cannot send the next request until you have processed the current response and appended its results.
The moment an agent has an inner loop that calls tools, three characteristic failure modes appear. Naming them gives you a diagnostic vocabulary for the problems you will hit immediately in Lab 4.
The model calls the same tool, with the same arguments, over and over. It may be stuck on an error it does not know how to recover from, or it may be waiting for a result that signals "done" that never arrives. Whatever the cause, the loop does not terminate on its own.
This matters because you pay for every API call. A single tool call might cost a fraction of a cent, but an infinite loop runs that cost without bound. For this reason, nearly every production agent puts a cap on the inner loop:
MAX_TOOL_ITERATIONS = 25
iteration = 0
while True:
iteration += 1
if iteration > MAX_TOOL_ITERATIONS:
print("\n[Agent hit iteration limit — forcing stop]")
messages.append({
"role": "user",
"content": "You have reached the maximum number of tool calls. "
"Please summarize what you've accomplished and stop."
})
response = client.messages.create(...) # one final summary call
break
response = client.messages.create(...)
# ... rest of inner loop
The iteration limit is a safety valve, not a design constraint. Good tool descriptions, clear instructions, and informative tool results all reduce how often the agent gets confused — but because the model is genuinely making it up as it goes, you almost always want the cap as a backstop. If your agent regularly hits the limit, that is a signal the task is too large for one user message or the tools are returning poor information.
The opposite problem: the model produces end_turn and replies with text before the work is actually finished. It edited one file and declared success without checking the others; or the task was too vague — "clean up this code" has no completion criteria — so the model decided arbitrarily that it was done.
The fix lives in the system prompt. You cannot rely on users to phrase tasks well, but you can give the agent good completion criteria and behavioral guidance, such as "When completing multi-file tasks, verify you've handled all files before reporting completion." What you should not do is impose an artificial floor like "you must call at least five tools" — there is no correct minimum number of turns. The goal is to help the agent judge, from what is in its context, whether it has genuinely satisfied the request.
This is the most common failure, and the one that bridges into the next module. The context window — whether 200,000 tokens or a million — fills up, and tool calls make it fill faster. Every file the agent reads, every tool call and result, stays in the context for the remainder of the session. As it grows:
We will spend the next section implementing real strategies to manage this — sliding window, selective preservation, compaction — in Module 7. But before writing any of that code, you have to be able to see the problem. That is what instrumentation is for.
The three failure modes share a root cause: the plan is implicit, and implicit plans drift. A lightweight countermeasure is to make the plan explicit — and this is exactly the chain-of-thought technique from Lecture 5.1, now applied inside the agent loop.
Recall that the model only reasons in tokens that actually exist in its context. If you instruct it, in the system prompt, to outline its steps before calling any tools and to output that outline as part of its response, the plan becomes real text in the conversation. From that point on, every subsequent step is conditioned on the written plan — the model is reading its own roadmap as it works.
This is, in a sense, "tricking" the model into committing its strategy to the context where it can serve as an anchor. It directly counters the failure modes:
It is worth being clear about what this is and is not. Explicit planning is not required for emergent tool chaining to work — the agent will chain tools without it. It is an optional reliability tool, and it carries the same tradeoff described in Lecture 5.1: planning tokens cost context, and on a simple one-step task they add noise without benefit. Use explicit planning for genuinely multi-step work; skip it when a single tool call will do.
This also answers a natural question — is there a way to give the model a plan up front? There are two ways. You can hand the model a plan in the user message ("first list files, then read each .py file, then edit the functions without docstrings"), or you can ask the model to produce the plan itself before acting, as described above. Both put the plan in the context. Handing the plan over constrains the model more — predictable, but rigid — while having the model plan first preserves more of its ability to improvise. The choice is the explicit-planning/emergent-planning tradeoff again.
Counting tokens is not the model's job; it is your job as the developer of the agent harness. Fortunately the data is already there. Every response from the Anthropic API carries a usage object:
response = client.messages.create(...)
print(f" Input tokens: {response.usage.input_tokens}")
print(f" Output tokens: {response.usage.output_tokens}")
input_tokens is the size of everything you sent: the system prompt, the tool schemas, and the entire messages array. output_tokens is what the model generated this turn. Because every inner-loop iteration carries the full history, input_tokens grows on each call — and tracking it over time draws the context-growth curve.
A small class is enough to start. It records the usage from each response and reports a summary:
class TokenTracker:
"""Track token usage across the agent session."""
def __init__(self):
self.calls = [] # (input_tokens, output_tokens) per API call
self.turn_boundaries = [] # indices where new user messages start
def record(self, response):
self.calls.append((
response.usage.input_tokens,
response.usage.output_tokens,
))
def mark_turn(self):
self.turn_boundaries.append(len(self.calls))
def report(self):
total_input = sum(c[0] for c in self.calls)
total_output = sum(c[1] for c in self.calls)
print(f"\n--- Token Usage ---")
print(f"API calls: {len(self.calls)}")
print(f"Total input: {total_input:,} tokens")
print(f"Total output: {total_output:,} tokens")
print(f"Peak input: {max(c[0] for c in self.calls):,} tokens")
You will extend this class through the rest of the course. A more sophisticated version takes an initial context budget and exposes methods like "am I at 85% of my limit?" — a threshold that can trigger context-management actions.
There is a particularly interesting move available here: expose the token tracker to the model as a tool. Just as list_files or read_file are tools, you can register a tool that returns the current token count. The model can then request its own context usage and decide what to do with that information — for instance, summarizing and wrapping up as it approaches the limit. This turns instrumentation from something the harness merely observes into something the agent can reason about. We will return to ideas like this when we implement context management.
Run the docstring task and watch the input token count climb on each call:
Call 1: 1,240 in / 45 out ← list_files
Call 2: 1,340 in / 520 out ← read_file (file contents enter context)
Call 3: 2,100 in / 85 out ← edit_file
Call 4: 2,240 in / 78 out ← edit_file (second function)
Call 5: 2,380 in / 490 out ← read_file (second file)
Call 6: 3,100 in / 92 out ← edit_file
Call 7: 3,250 in / 65 out ← final text reply
The pattern is unmistakable: input tokens grow monotonically, because every call carries the full history. The largest jumps come on read_file calls — a file's contents enter the context and remain for every subsequent call. This is the abstract "context growth" problem from Module 4 made concrete, and it shows exactly why token-efficient tools matter. In Lab 4 you will measure the dramatic difference when read_file is replaced by search_file + read_lines.
This is also where instrumentation answers a practical question — how do I tell whether my agent is stuck or just doing a lot of work? Print each tool call as it happens (the dispatch logging from Lecture 6.3 does this). If you see the same call repeated with identical arguments, it is stuck; if you see different calls progressing through the task, it is working.
The cost is real and worth keeping in front of you. At the Sonnet pricing in effect when this was recorded — roughly $3 per million input tokens and $15 per million output tokens — the seven-call example above costs only about $0.07. But a realistic session of twenty user messages over larger files reaches $2–5 easily, and an organization running a hundred such sessions a day is spending several hundred dollars a day. In that light, context management is not an optimization you do if you have time; it is an economic necessity.
By the end of Module 6 you should have a solid mental model of what happens inside an agent's inner loop and how it differs from the outer loop where the human participates.
Context grows monotonically, tool calls accelerate that growth, and eventually it breaks the agent. Seeing the problem clearly is the whole point of this lecture — and managing it is the subject of Module 7.
Streaming responses, which improve the user experience during long tool-calling sequences, were introduced at the API level in Lecture 3.1. Wiring streaming into the agent loop changes only the user experience, not the loop's logic, so it appears as an optional implementation task in Lab 4 rather than as a separate topic here.