Chapter 1. What an Agent Actually Is

A few months ago a friend asked me to look at a system he was building. He said: "My agent keeps forgetting what it's doing."

He showed me the code — a while loop, a call to messages.create(), a list of three tools. The agent was calling the right tool most of the time, but after about forty turns it would get confused, start over, re-derive facts it had established an hour ago, apologize for being slow. He wanted to know how to give it better memory.

The answer wasn't memory. His while loop was not a harness — it was a prompt running on repeat, and the forgetting he was chasing was just the most visible symptom of everything else the loop wasn't doing.

That gap is the problem this book exists to solve. The model — Claude, GPT, whatever — is the easy part: you call an API, you get a response. What's hard is everything around the model: the loop that decides when to stop, the protocol that shapes each turn, the context that shrinks faster than your agent grows smarter, the tools that have to work ten thousand times without corrupting state, the observability that tells you why last night's run went wrong. That "everything around the model" has a name — the harness — and building a good one is its own distinct craft, with its own failure modes, its own literature, and its own accumulated body of operational wisdom that is rarely written down in one place.

This chapter is about getting the vocabulary right before we write a single line. If the right name for your system is "workflow" when you've been calling it an "agent," you will solve the wrong problems for six months. If your harness is a bare while loop, you will ship something that doesn't know how to bound itself — and the failure mode, when it arrives, will look like a mystery rather than an oversight. So we start here, with definitions sharp enough that the rest of the book has stable ground to stand on.

By the end of this chapter, you'll have a repo skeleton, an understanding of the design space we're navigating, and a preview of the twenty-one chapters that follow. No agent yet; the next chapter builds one in forty lines and then breaks it in five different ways, and those five breakages become the itinerary for everything after.

1.1 Models, Agents, and the Category Error

Three definitions worth getting straight, because conflating them is the root cause of roughly every bad architecture I've seen in the past year.

A model is a function: you hand it a context, and it returns a probability distribution over next tokens. That is the whole contract. A model has no memory, no goals, and no capacity to act in the world — it responds to what it is given, and then it is done. This is true whether the model is a frontier system with hundreds of billions of parameters or a locally-hosted open model a fraction the size; the protocol is tokens → tokens, and everything else lives outside. Russell and Norvig's Artificial Intelligence: A Modern Approach draws the same line in classical terms: the model is the function, and an agent is the system that uses the function to perceive and act.

An agent is a loop around a model — plus bounded state, a set of tools, and a policy for managing context across turns. Where a model is stateless, an agent has memory, imperfect because context is finite, but persistent across turns through the transcript the loop keeps feeding back in. Where a model has no goals, an agent pursues goals expressed through the context it is given and re-given each turn. And where a model cannot act, an agent acts through tools that the loop dispatches on the model's behalf. The classical agent literature — Franklin and Graesser's 1996 "Is it an agent, or just a program? A Taxonomy for Autonomous Agents" is the most-cited piece — tends to require four properties: autonomy (it makes its own decisions), reactivity (it responds to its environment), proactivity (it pursues goals rather than only reacting), and situatedness (it is embedded in a world where its actions have consequences). Production LLM agents satisfy all four, though the "world" in question is usually a bounded environment defined by the tools you chose to give them.

A harness is the engineering that surrounds the model and turns it into an agent. The word has useful history: a test harness runs test code inside a controlled environment that provides setup, isolation, and teardown; an operating system is, in a practical sense, a harness around the processes it runs, giving them addresses, scheduling, I/O, and crash recovery while the processes themselves do only compute. An LLM harness is the same kind of thing for a model. Because a model's contract is narrow — tokens in, tokens out — everything else required to make the model useful in a production setting has to live outside the model. That "everything else" is the harness, and its scope is broader than most first attempts reckon with:

• A loop that decides when to invoke the model, when to call a tool, when to retry a failed call, and when to stop and return a final answer.
• A turn protocol that structures each call's input (system prompt, history, tool schemas, available state) and parses each call's output (text, tool calls, reasoning traces, stop reasons).
• Context management — the policy that decides what the model sees turn after turn as the session grows past the window, through compaction, retrieval, scratchpad offload, or observation masking.
• Tool orchestration that registers the agent's available actions, dispatches them safely, validates arguments against schemas, and routes results back into context without corrupting anything upstream.
• Error handling that keeps a single bad tool call, malformed response, or transient provider failure from poisoning the rest of the session.
• Observability that tells you, after the fact, why a run went the way it did — which tool took twelve seconds, which compaction dropped the fact the final answer needed, which sub-agent burned the tokens.
• Persistence that lets a session survive a process crash or a deliberate pause, with side-effecting tool calls de-duplicated on resume so nothing runs twice.
• Permission and budget controls that prevent the agent from taking actions you didn't authorize or spending money you can't afford.

None of these is optional for a production system. They are either present by design, because you built them, or present by accident, because you didn't — and accidental harnesses are where most of the public post-mortems come from. The 2025 $47K agent-loop incident, in which two agents ping-ponged requests for eleven days while token-budget alerts fired but no enforcement existed to stop them, is one recent example; the MAST study of multi-agent failure modes (Cemri et al., 2025) traces 36.9% of observed failures to coordination breakdowns the harness was supposed to mediate. The pattern these share is that a harness is what decides whether a model's capability becomes a system's capability. The book's argument, elaborated across twenty-two chapters, is that a harness deserves the same engineering discipline you'd give a database schema or a service mesh — because, for this new class of system, that is what it is.

The category error — and it is everywhere — is the claim that an agent simply is a model, or that "building an agent" means "picking a good model." It sneaks in through framing: you ask a vendor for an agent and they sell you a model; you type a prompt into a chat interface, watch the interface use a tool, and the whole exchange gets called "agentic." Sometimes the usage is just loose. Sometimes it is load-bearing, and that is where it hurts — if your mental model says "the model does the agent things," you will not build the parts that actually need building, and your users will feel it in reliability, latency, cost, and safety, usually in that order.

A useful test. If the model gets twice as good tomorrow — OpenAI ships GPT-6, Anthropic ships Claude 5 — does your agent get twice as good? If the answer is yes, you have built well: your harness is a thin, honest conductor for the model's capability, and a rising tide lifts it cleanly. If the answer is no — and in practice it is almost always no — then something else is the bottleneck, and that something lives in the harness. Most systems that call themselves agents fall into the second category, and so does every system the book teaches you to build.

Claude Code is a harness. LangGraph is a harness. The OpenAI Agents SDK is a harness. Claude itself is a model. The shape of the harness determines almost everything about how the assembled system behaves — its failure modes, its operating costs, its observability story, its user experience — and the rest of this book is about how to shape yours well.

1.2 The Four-Axis Design Space

Every agent harness can be located on four axes. I'll return to these throughout the book; they are the coordinate system we'll navigate. They are a pragmatic taxonomy rather than a formal one — if you prefer Russell and Norvig's classical simple reflex / model-based / goal-based / utility-based / learning hierarchy, our axes span roughly the same design space, but cut it along lines that matter for LLM-powered systems specifically.

Autonomy. How much decision-making the harness delegates to the model. At one extreme is a deterministic workflow with LLM calls inserted at fixed points — no loop, no model-initiated choice, the control flow is yours and the model is an ingredient. At the other is a bare loop that hands the model every decision, including when to stop. Most production systems sit somewhere in between, and pretending to be more autonomous than you actually are — "our pipeline is fully agentic!" — misleads both you and your users, usually in ways that only become visible under load.

State. What the harness remembers across turns. Three tiers appear often enough to be worth naming. Context-only state lives entirely in the model's window and is lost on compaction. Session state is persisted structurally outside the window — a scratchpad, a plan object, a running tally — and survives within a single run. Durable state survives process crashes and deployments, stored in a database row or a serialized checkpoint on disk. Most naive harnesses sit at tier one and don't realize they've chosen that tier; most production harnesses need all three, and this book will build all three across the course of its chapters.

Tools. What the agent can do in the world. Zero tools and you have a chatbot. A handful of well-designed tools — shell, file read and edit, search, retrieval — and you have a capable assistant. Fifty tools and you have a scalability problem most teams discover too late, long after the tool list has calcified into something nobody wants to touch. Chapter 12 is devoted to the "tool cliff," the non-linear performance collapse that happens somewhere north of twenty tools for most models, and the two standard fixes (BM25-based selection and a pinned discovery tool) that keep capacity growing after the cliff.

Context. How the harness manages what the model sees, turn after turn. The options form a rough progression: pure append (the naive default, which fails around turn forty in most setups), sliding window, summarization, compaction, retrieval, scratchpad offload. Each has its own cost, failure mode, and appropriate home. Context engineering — the discipline of doing this well — has, in the last year, become the central skill of production agent work, to the point that several large vendors now treat it as a distinct specialty within applied AI. Chapters 7 through 11 live there.

An honest mapping of your own system onto these four axes will tell you what to build next. A good harness does not maximize all four; it picks the point that fits the problem and then engineers that point hard. The harness we build in this book lands at medium autonomy, all three state tiers, about a dozen carefully-designed tools, and aggressive context engineering — a specific choice, not a universal answer. You should feel free to locate your own projects elsewhere, and part of what this chapter's taxonomy is for is giving you the vocabulary to defend that choice when you do.

1.3 Workflow vs. Agent — Anthropic's Useful Distinction

In December 2024 Anthropic published a post titled "Building Effective Agents" that drew a line I'll lean on throughout the book. In paraphrase: a workflow is a predefined code path where LLM calls happen at specific, pre-decided steps — the control flow is yours, and the model is an ingredient you drop into slots you've chosen. An agent, by contrast, is a system where the LLM directs its own control flow — deciding which tools to call, in what order, and when to stop — without the code above it dictating the next step.

Both are legitimate architectures, and most production systems are in fact mixtures of the two. The Anthropic post's most-skipped observation, and the one most worth internalizing early, is that workflows win more often than builders expect. If your problem has predictable structure, a workflow will be faster, cheaper, more reliable, and easier to debug than an agent doing the same work; agents earn their complexity only when the problem genuinely requires dynamic tool selection, iterative refinement, or open-ended exploration.

The foundational paper behind the agent-as-a-loop view is Yao et al.'s 2022 "ReAct: Synergizing Reasoning and Acting in Language Models," which established that interleaving reasoning steps with tool calls — think, act, observe, think, act, observe — outperforms either pure reasoning or pure tool-use in isolation. Nearly every LLM agent built since has been a variation on that loop. Simon Willison has pushed Yao's framing further into the vernacular, arguing that the word "agent" has become so overloaded in marketing copy that it's almost unusable, and that the operational definition practitioners have converged on is simply tools in a loop. That is a deliberately modest framing — it names what the software actually does without promising autonomy it doesn't have — and it is the framing this book will adopt throughout.

Both positions matter here. We are going to build a loop around a model — a real, production-capable loop — but we are not going to pretend the loop has wisdom it lacks. Every design decision we make will be about giving the model enough structure to do useful work, without enough rope to hang our users.

Here is a concrete diagnostic, worth running the next time someone on your team says "let's make this agentic":

1. Can the problem be solved by a pipeline of N fixed LLM calls? If yes, build that, and don't build an agent.
2. Does the problem require the model to decide, at runtime, which of several tools to invoke and in what order? If yes, you have the beginning of a case for an agent.
3. Does the problem require the agent to iterate — call a tool, read the result, decide on a next tool based on what it learned? If yes, you need a real loop.
4. Does the problem require the agent to plan, fail, and re-plan over a long horizon? If yes, you need a full harness of the sort this book builds, and you need evaluations alongside it, because the failure modes at that horizon are not obvious and will not surface until you measure them.

Three out of four production "agents" I've seen should have been workflows. That is not a criticism — workflows are often the right answer, and a well-built workflow with three carefully-placed LLM calls will frequently outperform a loosely-wired agent on the same task. The problem is that calling a workflow an agent hides the real engineering that keeps it working, and that hiding is what lets systems drift into the accidental-harness territory §1.1 warned about.

1.4 What Makes a Harness Hard

Three things, roughly in order of how often they'll bite you.

Context is a moving target. Models have fixed context windows — typically 200K tokens at the time of writing, sometimes 1M on the premium tier — but tool outputs are unbounded, and tool outputs go into context. Do the arithmetic: your agent reads a 60,000-token file and half its window is gone; read two such files and the session is effectively over, because there is no room left for the actual reasoning the model is supposed to do. The hard limit is only half the story. Liu et al.'s 2023 "Lost in the Middle: How Language Models Use Long Contexts" showed that models attend disproportionately to information at the beginning and end of the window and systematically miss content in the middle, and the Chroma research team's July 2025 study extended that finding into what they named context rot: performance on retrieval-style tasks degrades continuously as context fills, well before any hard limit is reached. The window, in practice, is both smaller and more hostile than the spec sheet suggests. Every production harness has some answer to this, and the quality of the answer is more or less the quality of the harness itself.

Tools lie, and the model believes them. A tool that returns truncated output without saying so will make the model confidently wrong downstream. A tool whose description says "sends a message" but actually sends five will get abused in ways the description gave no warning about. A tool that fails silently, or returns a null value that could plausibly mean either "not found" or "permission denied," will send the agent into a loop while it tries to disambiguate a situation the tool's author already forgot about. Tools are not a programming interface for humans; they are an interface for a non-human consumer with very specific cognitive constraints, and designing them is its own discipline. It's also the discipline most under-invested in practice, which is why Chapter 11 spends an entire chapter on getting this right and why the tool-cliff problem in Chapter 12 exists in the first place.

Failure compounds. An agent with 95% per-step accuracy has roughly a 60% chance of completing a ten-step task cleanly; with 85% per-step accuracy — which is not a low number in absolute terms — the end-to-end success rate drops to about 20%. Long-horizon agents fail not because any single step is catastrophic, but because every step is an independent coin flip and the flips multiply. The MAST study (Cemri et al., 2025) measured this empirically across real multi-agent systems and found that most failures come from accumulated small errors — misinterpretations, specification misunderstandings, coordination slips — rather than from any single dramatic mistake. A harness earns its keep by turning coin flips into decisions: validating each step, recovering from errors, preventing one small mistake from poisoning the next ten. This is why most of the engineering in this book is defensive in character — not a disappointment, it's the job.

Every chapter of the book addresses one of these three forces. You will notice as we go that the interesting design questions almost never concern the model itself. They concern the protocol around it.

1.5 What This Book Builds

By the end of Chapter 22 your repo will contain a harness that:

• Runs against any provider through a thin adapter layer — Anthropic, OpenAI, a locally-hosted open-source model — without changing agent logic. Provider-agnostic from Chapter 2 on; we never hard-code a vendor in the core.
• Manages its own context window through compaction, observation masking, and external scratchpad storage.
• Supports bounded sub-agent delegation with per-agent cost attribution and a spawning budget.
• Sandboxes tool execution with filesystem allowlists and an explicit permission gate.
• Consumes external tools via the Model Context Protocol, inside the same permission model as built-ins.
• Streams responses, handles interruption, and resumes durably after a crash.
• Emits OpenTelemetry traces with per-session, per-task, per-agent attribution.
• Runs a golden-trajectory regression suite before any model upgrade.

That list looks long because it is, and the book builds it piece by piece — every chapter takes one subsystem from naive to correct, and the chapter after that uses it, so the harness grows in a way that stays runnable at every stage rather than requiring a big-bang reveal at the end.

The harness will not be a clone of Claude Code, LangGraph, or the OpenAI Agents SDK. It steals shamelessly from all three, and from SWE-agent, smolagents, and every public post-mortem I could find while writing. What it will be is yours — a working system you understand end-to-end, small enough to read in an afternoon, capable enough to do real work.

One constraint is deliberate and worth flagging early: the harness is provider-agnostic from the start. The very first loop we write, in Chapter 2, runs against a mock provider — not because mocks are pedagogically cute, but because the moment you hard-code one vendor's SDK into your harness you have built a thing that's harder to migrate than it needs to be, and harder to test than it needs to be, and more tightly coupled to a pricing model you don't control than is comfortable. The adapter seam is the first piece of real architecture we lay down, and every subsequent chapter respects it. By Chapter 22 you will run the same agent code against Anthropic, OpenAI, and a local open-source model side-by-side — swapping providers is a configuration change, not a rewrite, which is the concrete payoff of an investment made in the opening chapters.

1.6 How to Read This Book

Two modes are supported, and most readers will use a bit of both.

Linear. Open Chapter 2, type every line, run the tests, and commit. Read the theory halves alongside the code — they aren't decoration; they are the argument for why the code is shaped the way it is. By the end of each chapter, git log in your repo should tell the story of the chapter in five to fifteen commits. If something stops working, the companion repo has one annotated tag per chapter — git checkout ch03-transcript will put you on known-good ground to resync.

Reference. You already have a harness and you want to understand compaction, or sub-agents, or evals. Each chapter stands alone well enough to skim: the opening frames the problem in concrete terms, and the "what you now understand" close tells you what you should have gained so you can check yourself. Every concept has exactly one canonical home in the book, and other chapters point back rather than re-explaining.

Both modes assume Python 3.11+, comfort with type hints and async/await, and prior exposure to at least one LLM API. You don't need to have built an agent before — if you have, some of the early chapters will go fast, which is fine, because the vocabulary we establish here pays off in Chapters 7 through 11.

A word on the code. Every code block in the book runs when assembled in order. There are no # ... ellipses hiding the load-bearing parts; when I simplify for teaching purposes, I say so, and the companion repo has the fuller version. The code is not decorative — it is the argument made concrete.

A word on opinions. This book is opinionated. Where a defensible alternative exists I'll name it and explain when to take it; where I'm picking one path because the book needs a through-line, I'll say that too. My goal is not to convince you that my way is the only way. It is to give you enough understanding of the tradeoffs that you can make your own decisions, and, when a new framework or new model generation lands in six months, evaluate it on substance rather than marketing.

1.7 Setting Up the Repo

Enough framing. Let's make a place to put the code.

We need Python 3.11 or newer and uv. If you don't have uv: curl -LsSf https://astral.sh/uv/install.sh | sh (or brew install uv). Everything in this book runs under uv — it manages the Python toolchain, the venv, and dependencies in one binary, and it's ~10× faster than pip for the install loops we do across chapters.

Create the project directory and initialize:

mkdir agent-harness && cd agent-harness
uv init --package --python 3.11

uv init --package scaffolds a pyproject.toml with a src/ layout, which matches the layout we want. Replace the generated pyproject.toml with this — it declares the package, the Python version floor, and a few light dependencies:

# pyproject.toml
[project]
name = "harness"
version = "0.1.0"
requires-python = ">=3.11"
dependencies = [
    "httpx>=0.27",
]

[project.optional-dependencies]
anthropic = ["anthropic>=0.40"]
openai = ["openai>=1.40"]

[dependency-groups]
dev = ["pytest>=8.0", "pytest-asyncio>=0.23"]

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.hatch.build.targets.wheel]
packages = ["src/harness"]

Notice that anthropic and openai are optional extras. That's deliberate. The core harness should install without either. We'll exercise this in Chapter 3, when the mock provider does all the work while the real SDKs stay uninstalled. dev lives under [dependency-groups] — uv's native way to express dev-only dependencies without polluting the published extras.

Sync the environment. uv creates .venv/ automatically on first run:

uv sync

That's the whole install. You don't need to source .venv/bin/activate — we use uv run for every command, and uv picks up the project's venv automatically.

The project layout we will grow into. Create it now; most of the files are empty. We'll populate them as we go.

agent-harness/
├── pyproject.toml
├── README.md
├── src/
│   └── harness/
│       ├── __init__.py
│       ├── agent.py          # the loop (Chapter 2)
│       ├── messages.py       # typed transcripts (Chapter 3)
│       ├── providers/        # provider adapters (Chapter 3)
│       │   ├── __init__.py
│       │   ├── base.py       # the Provider protocol
│       │   └── mock.py       # in-memory fake
│       ├── tools/            # tool protocol + registry (Chapters 4-5)
│       │   └── __init__.py
│       └── context/          # accounting + compaction (Chapters 7-11)
│           └── __init__.py
├── tests/
│   └── __init__.py
└── examples/

One file worth writing this early. A smoke test that proves the package imports and the Python version is what we need.

# tests/test_smoke.py
import sys

import harness


def test_python_version() -> None:
    assert sys.version_info >= (3, 11), "This book assumes Python 3.11+"


def test_package_imports() -> None:
    assert harness is not None

Run it:

uv run pytest tests/test_smoke.py -q

You should see two passing tests. If you don't, fix the import path before going further. Every chapter in this book assumes uv run pytest runs clean at the start. Broken tests accumulate badly.

Commit:

git init
git add .
git commit -m "ch01: project skeleton"
git tag ch01-skeleton

That tag will matter in Chapter 2, when we want to show you exactly what changed. Every chapter ends with one tag of this shape.

1.8 Try It Yourself

1. Map three systems. Pick three agent-shaped systems you've used — ChatGPT, Claude Code, Cursor, a customer support chatbot, whatever. For each, place it on the four axes from Section 1.2. Where is it high-autonomy? Where is its state? How many tools? What's its context strategy, as far as you can tell from the outside? Write it down. This is the last time in the book I'll ask you to think without writing code, but it's worth doing before Chapter 2.
2. Workflow or agent? Take a task you or your team is about to automate. Apply the four-question diagnostic from Section 1.3. Where does the task land? If the honest answer is "workflow, actually," notice that and consider whether you still want to read this book. (You probably do — harness patterns show up in workflow design too — but the goal framing changes.)
3. Read one harness's core loop. Open the source of one of the harnesses we surveyed — smolagents/agents.py is a good starting point at ~1,000 lines, and mini-swe-agent/src/minisweagent/agent.py is even shorter at ~100 lines. Read the main loop. Find the three moments where the loop decides: to call a tool, to stop, to produce final output. Don't try to understand everything; just find those three moments. You will write your own versions in the next few chapters, and it helps to have seen real examples first.

Chapter 2. The Minimum Viable Loop

Previously: we set up a repo skeleton and agreed on vocabulary. Model is a function; agent is a loop; harness is the engineering around the loop. No code yet.

A while loop is the smallest thing that separates a chat interface from an agent, and it's what turns one API call into many — the model reads the output of its previous turn and decides what to do next, a pattern Yao et al. formalized in 2022 as ReAct (reason, act, observe) and that nearly every modern LLM agent is some variation on. That decision, made once per iteration, is the whole point: a model that cannot observe its own last action cannot debug, recover, or finish, while a model that can — even badly — is the start of an agent.

We are going to write that loop now. Forty lines in one file, no frameworks, no abstractions we haven't earned — and then we are going to break it in five specific ways, on purpose, and watch each break ripple through the design. Those five breaks will become the itinerary for the rest of the book, and most of the engineering in subsequent chapters is traceable back to one of them.

By the end of this chapter, your harness can answer a question by calling a calculator tool in a loop, against a mock provider rather than a real API. The mock provider is not a placeholder we'll throw away; it is the first piece of real architecture we lay down, the seam that makes your harness provider-agnostic from day one, and the reason every subsequent chapter can add capability without ever hard-coding a vendor's SDK into the core.

2.1 What the Loop Has to Do

Three decisions happen on every iteration of the loop. They map cleanly onto the think-act-observe cycle of ReAct and onto the Planning → Tools → Memory → Action decomposition that Lilian Weng's widely-read 2023 post "LLM Powered Autonomous Agents" offered as a reference model for the field:

1. Ask the model what to do next. Send the current transcript, get a response.
2. Decide whether the response is a tool call or a final answer. If it's a tool call, execute the tool and append the result to the transcript; if it's a final answer, stop and return it.
3. Bound the loop. If we somehow hit N iterations without a final answer, stop anyway — a loop without a bound is a bug, not a feature, and in production it's the bug that usually shows up in the cost report before it shows up anywhere else.

That is the whole shape of the thing. Everything else in this book is accretion on top of those three decisions: compaction, sub-agents, streaming, evals — they all live inside, around, or between steps 1 and 2, while step 3 is where the cost-runaway failure modes get caught and bounded.

Two subtle points are worth naming before we write any code.

The transcript is the state. The loop has no other memory of what happened turn to turn; if a fact needs to persist across turns, it either lives in the transcript (and costs tokens forever) or it doesn't survive at all. Later chapters introduce external state — scratchpads, checkpointers, retrieval — but every one of them exists precisely because the transcript is too narrow a container for durable memory.

The provider is a dependency, not the protagonist. The loop doesn't care whether the response came from Anthropic, OpenAI, a locally-hosted Llama, or a mock; it cares only that something returns a response in a shape it can interpret. Designing that shape is the work of Chapter 3, and for this chapter a mock is all we need — strictly better than a real API for our purposes here, because it runs offline, deterministically, and costs nothing.

2.2 The Provider Protocol, Introduced Early

Most tutorials start by calling anthropic.Anthropic() or OpenAI() directly in the loop — the right thing to do when you're exploring, and exactly the wrong thing when you're building something you expect to last. The moment a vendor SDK is imported from your core loop, you have taken on the vendor's quirks as part of your design: response envelope shape, streaming protocol, token-counting method, error taxonomy, all of it. Refactoring later means touching every file that ever touched the loop, and by then there are usually many.

Instead, we'll define a Provider protocol — a small, stable interface — and write a mock implementation of it. Chapter 3 writes real Anthropic and OpenAI adapters to the same protocol, and every subsequent chapter depends only on the protocol, never on a specific vendor's API surface.

# src/harness/providers/base.py
from __future__ import annotations

from dataclasses import dataclass
from typing import Protocol


@dataclass(frozen=True)
class ProviderResponse:
    """What a provider gives us back: either text, or a tool call."""
    kind: str  # "text" or "tool_call"
    text: str | None = None
    tool_name: str | None = None
    tool_args: dict | None = None
    tool_call_id: str | None = None


class Provider(Protocol):
    def complete(self, transcript: list[dict], tools: list[dict]) -> ProviderResponse:
        """Given a transcript and available tools, produce one response."""
        ...

Two notes on this protocol are worth pausing on before we use it.

The transcript and tools are plain list[dict] for now, which is a deliberate simplification; Chapter 3 promotes both of them to typed dataclasses with proper block structure and a Transcript wrapper. Using dicts here keeps the mock trivially easy to write, and the protocol stays small enough to read in about five seconds — an honest test of whether an abstraction is paying its way.

ProviderResponse collapses two cases into one shape. Real provider responses are richer than this — they carry token counts, finish reasons, multiple content blocks, streaming chunks, reasoning traces — but none of that matters for the loop at this stage. The loop wants to know one thing: did the model ask me to call a tool, or did it give me an answer? Everything else is someone else's problem until we need it, and dragging it in now would be premature.

Now the mock. It implements a tiny scripted scenario: ask about 2 + 2, the mock calls a calculator tool, reads the result, and produces the answer.

# src/harness/providers/mock.py
from __future__ import annotations

from .base import Provider, ProviderResponse


class MockProvider(Provider):
    """A scripted provider for teaching and testing.

    Walks through a fixed list of responses, one per call.
    """

    def __init__(self, responses: list[ProviderResponse]) -> None:
        self._responses = list(responses)
        self._index = 0

    def complete(self, transcript: list[dict], tools: list[dict]) -> ProviderResponse:
        if self._index >= len(self._responses):
            raise RuntimeError("mock ran out of responses")
        response = self._responses[self._index]
        self._index += 1
        return response

A note on the MockProvider(Provider) line. In Python, a Protocol is satisfied structurally — any class with matching methods counts, no inheritance required. So why inherit? Two reasons. It documents intent: a reader sees "this class implements the Provider contract" without having to diff method signatures. And it turns a silent mismatch into a type-checker error at class definition time: forget to add complete, or change its signature, and mypy/pyright flag the class instead of letting the bug surface later in the loop. The real-provider adapters in Chapter 3 do the same thing.

That's the whole provider abstraction for this chapter. Thirty lines of code and we have a seam we will keep for the entire book.

2.3 The Loop

Here is the naive loop. I am calling it naive because it's about to break in five ways we already know about. It's still a useful starting point — everything it doesn't do will be motivated by a specific failure.

# src/harness/agent.py
from __future__ import annotations

from typing import Callable

from .providers.base import Provider, ProviderResponse


MAX_ITERATIONS = 20


def run(
    provider: Provider,
    tools: dict[str, Callable[..., str]],
    tool_schemas: list[dict],
    user_message: str,
) -> str:
    transcript: list[dict] = [{"role": "user", "content": user_message}]

    for _ in range(MAX_ITERATIONS):
        response = provider.complete(transcript, tool_schemas)

        if response.kind == "text":
            transcript.append({"role": "assistant", "content": response.text})
            return response.text or ""

        if response.kind == "tool_call":
            if response.tool_name is None:
                raise RuntimeError("tool_call response is missing tool_name")
            if response.tool_name not in tools:
                raise RuntimeError(f"unknown tool: {response.tool_name!r}")

            tool_fn = tools[response.tool_name]
            result = tool_fn(**(response.tool_args or {}))

            transcript.append({
                "role": "assistant",
                "content": [{"type": "tool_use", "name": response.tool_name,
                             "id": response.tool_call_id, "input": response.tool_args}]
            })
            transcript.append({
                "role": "user",
                "content": [{"type": "tool_result", "tool_use_id": response.tool_call_id,
                             "content": result}]
            })
            continue

        raise RuntimeError(f"unexpected response kind: {response.kind!r}")

    raise RuntimeError(f"agent did not finish in {MAX_ITERATIONS} iterations")

Notice the three explicit guard clauses inside the tool-call branch: tool_name is None, tool_name not in tools, and a final else that raises on an unexpected response.kind. The comment-based assumption # response.kind == "tool_call" that a lot of tutorial code relies on is, in practice, a silent None-deref or an opaque KeyError waiting for its turn. The rule for this book is simple: if the type system doesn't narrow the case for you, narrow it yourself and raise a descriptive error. Later chapters replace these raises with structured ToolResults that the model can read and recover from, but even then every branch is still enumerated — defensive enumeration of cases is the engineering discipline the harness rests on.

Read that loop twice. Everything after it in the book is about one of the implicit choices you can see right here:

• The transcript is a list of dicts — no type safety, no validation, no block-level structure.
• Tools are a dict[str, Callable] — no schema check, no side-effect declaration, no permission gate.
• The tool is called with **response.tool_args and we trust it implicitly.
• result is a string. If the tool returned 50,000 characters of JSON, it goes straight into the transcript.
• There's no streaming, no cancellation, no retry, no observability, no cost counting.
• MAX_ITERATIONS = 20 is the only thing standing between this loop and an unbounded cost runaway.

That is the point — we are not going to pretend to be surprised when it breaks.

2.4 The First Run

Let's make it work before we make it fail. A calculator tool, a mock scenario.

# examples/ch02_calculator.py
from harness.agent import run
from harness.providers.base import ProviderResponse
from harness.providers.mock import MockProvider


def calc(expression: str) -> str:
    # dangerous in real life; fine for a mock
    return str(eval(expression, {"__builtins__": {}}, {}))


mock = MockProvider([
    ProviderResponse(
        kind="tool_call",
        tool_name="calc",
        tool_args={"expression": "2 + 2"},
        tool_call_id="call-1",
    ),
    ProviderResponse(kind="text", text="2 + 2 is 4."),
])

tool_schemas = [{
    "name": "calc",
    "description": "Evaluate a Python arithmetic expression.",
    "input_schema": {
        "type": "object",
        "properties": {"expression": {"type": "string"}},
        "required": ["expression"],
    },
}]

answer = run(
    provider=mock,
    tools={"calc": calc},
    tool_schemas=tool_schemas,
    user_message="What is 2 + 2?",
)

print(answer)  # -> "2 + 2 is 4."

Run it:

uv run examples/ch02_calculator.py

You should see 2 + 2 is 4. printed. Two turns — the model asks for the calculator, we run it, the model reads the result, the model produces a final answer — and that is an agent. A small, contrived, brittle one, but structurally the real thing; every harness in the rest of the book, and every production harness in the wild, is a variation on this same two-turn pattern with progressively more engineering layered between the asks.

Commit:

git add -A && git commit -m "ch02: minimum viable loop with mock provider"
git tag ch02-minimum-loop

2.5 Five Ways to Break It

Now the pedagogically useful part. We are going to feed the loop five specific failure scenarios, one at a time, and watch each one reveal a missing piece of engineering the naive version quietly assumed would not matter.

Break 1: The model asks for a tool that doesn't exist

Change the mock's first response to call a tool we didn't register:

ProviderResponse(
    kind="tool_call",
    tool_name="calculator",  # not "calc"
    tool_args={"expression": "2 + 2"},
    tool_call_id="call-1",
),

Run it and you get a RuntimeError: unknown tool: 'calculator'. The guard clause we added catches the missing name and the loop crashes deliberately, rather than stumbling into a silent KeyError deep inside the tool lookup — which is strictly better, because the error now names the actual problem. The model, though, has no chance to recover: the exception unwinds the whole loop, and a misnamed tool in turn three kills a session that had nine turns of useful work behind it.

What's missing. A dispatch layer that catches "unknown tool" and returns a structured error to the model as a tool result, so the model gets one more chance to call the right tool. Chapter 4 introduces the ToolRegistry that does this.

Break 2: The model's tool arguments don't match the schema

ProviderResponse(
    kind="tool_call",
    tool_name="calc",
    tool_args={"expr": "2 + 2"},  # wrong key name
    tool_call_id="call-1",
),

You get TypeError: calc() got an unexpected keyword argument 'expr', and the loop dies. Same class of failure as Break 1, but one level deeper: a model that misnamed a parameter never gets the chance to see what it did wrong, because the exception unwinds the loop before the next turn can happen.

What's missing. Schema validation before dispatch. The loop should notice the args don't match, return a validation error to the model, and give it a chance to correct. Chapter 6 builds this.

Break 3: The tool itself raises

def calc(expression: str) -> str:
    return str(eval(expression, {"__builtins__": {}}, {}))

# And the mock asks for:
ProviderResponse(
    kind="tool_call",
    tool_name="calc",
    tool_args={"expression": "1 / 0"},  # guaranteed ZeroDivisionError
    tool_call_id="call-1",
),

ZeroDivisionError, and again the loop unwinds. That alone would be fine to handle locally, but consider the subtler versions that show up in any non-trivial system: a network tool that times out, a file read that hits a permission error, a shell command that returns exit code 1, a remote API that returns a 503. All of these are expected failures in a harness that does anything interesting, and the loop currently has no place to put them other than "crash the session."

What's missing. A tool-dispatch wrapper that converts tool exceptions into structured tool-result errors, visible to the model. Chapter 6 again.

Break 4: The model never stops

mock = MockProvider([
    ProviderResponse(kind="tool_call", tool_name="calc",
                     tool_args={"expression": "1"}, tool_call_id=f"call-{i}")
    for i in range(100)
])

The model keeps calling the tool, iteration after iteration, and MAX_ITERATIONS = 20 catches it — but we raise a RuntimeError that discards the partial transcript along with everything useful for debugging. The number itself is arbitrary, too: twenty is too low for a real task and too high for a true runaway. We need a principled bound, one rooted in cost rather than iteration count, and we need to preserve the transcript when the bound fires so a human can figure out what went wrong.

What's missing. Two things. A token budget that triggers termination based on cost, not iteration count (Chapter 20). An observability layer that preserves the transcript for debugging when we do terminate (Chapter 18). And — looking ahead — a dedup check that notices the model is calling the same tool with the same args over and over, and halts before the budget fires (Chapter 6).

Break 5: The tool returns a novel

def calc(expression: str) -> str:
    # imagine a "read_file" tool that reads 60,000 tokens of JSON
    return "X" * 200_000  # 200KB of X

The loop still works — technically — but the transcript now has 200KB of X in it, and the next turn sends that whole transcript back to the provider. By turn five we're well past the context window, the provider returns an error, and the session crashes in a way that looks mysterious only because nobody was tracking the cost of what the tool was returning.

This is the central problem that shapes the rest of the book. The loop has no awareness of how much context it's using, no strategy for summarizing or truncating tool outputs, no concept of a scratchpad for state that shouldn't be in context at all — and no way to tell the difference between 200 bytes of useful signal and 200KB of noise that happens to look similar at the protocol level.

What's missing. Context accounting (Chapter 7), compaction (Chapter 8), external state (Chapter 9), retrieval (Chapter 10), and deliberate tool design that avoids producing these blobs in the first place (Chapter 11).

2.6 The Itinerary

Those five breaks are the book. Look at them laid out:

Every chapter from here to Chapter 11 is motivated by one of these five breaks. The chapters after that — orchestration, observability, evals, cost control, resumability — extend the harness into production territory once the core is solid.

If at any point the design feels over-engineered, come back to this table. Every piece of machinery is there because we watched the absence of it crash a loop.

2.7 A Quick Look at How Real Harnesses Handle This

Before we close the chapter, a sanity check: do real harnesses actually look like this — a while loop with a dispatch inside? The answer, across the ones worth looking at, is yes.

Claude Code, per Anthropic's public documentation, has an agent loop described in roughly 88 lines internally. The core is exactly what we wrote: the model produces a response, if the response contains tool calls the harness dispatches them and appends results, then it loops; otherwise, it returns. What Claude Code adds on top is production-grade error handling, a permission gate in front of every side-effecting tool, and the compaction and checkpointing we'll build in later chapters.

smolagents, Hugging Face's open-source agent library, fits in about a thousand lines total. Its MultiStepAgent.run() method is a for _ in range(self.max_steps) loop with the same three decisions we just wrote, plus a richer error taxonomy and an observation-formatting layer.

mini-swe-agent, the minimal variant of SWE-agent, is about a hundred lines and uses the same structure with a single bash tool instead of a registry — a useful reference for how thin a working harness can get when the problem shape is narrow enough to assume a single tool.

LangGraph looks different on the surface — it's a compiled graph, not a while — but the graph compiles down to a Pregel-style execution model in which a ReAct-style agent is a cycle between an LLM node and a tool node. Same three decisions as our naive loop, different packaging, and the "ReAct" naming here refers directly to Yao et al.'s 2022 paper that introduced the reason-act-observe paradigm the graph implements.

The loop is not a simplification for pedagogy; it is the actual shape of the thing, and Wang et al.'s 2024 "Survey on LLM-based Autonomous Agents" — which analyzed more than a hundred LLM-agent implementations across research and production — found the same core structure in the overwhelming majority of them. Everything else is engineering around it.

2.8 Try It Yourself

1. Reproduce the five breaks. Take the working calculator example, apply each of the five breakages in turn, and observe the specific failure mode each one produces. Note in your own words what a correct handling would look like — the articulation, before you know the answer, is the learning.
2. Add a sixth break of your own. Find one more way to break this loop that isn't in the table above. Some candidates worth trying: the provider itself raises (network error, rate-limit response), the mock returns a text response containing JSON tool-call syntax the loop doesn't parse, or the tool returns something that isn't a string. Confirm the failure, then decide whether it belongs to one of the five classes already named or constitutes a new category.
3. Write the smallest possible type-safe version. Re-implement run() with TypedDict message shapes instead of raw dicts. Does the type checker catch any of the five breaks at authoring time? Which ones does it not catch, and why? (The answer to the second question is where most of Chapter 3's motivation comes from.)

Chapter 3. Messages, Turns, and the Transcript

Previously: we built a forty-line loop against a mock provider and watched it break five ways. Break 5 — tool output overwhelming the transcript — hinted that the transcript was doing too much work as a pile of dicts. It's time to give it some structure, and at the same time plug in real providers.

A message is not a dict. A message is a typed record with a role, ordered content blocks, a provenance, a token cost, and a creation timestamp. A dict can hold all of those, but it can also hold none of them — and a loop that treats {"role": "user", "content": "..."} and {"role": "user", "content": [{"type": "text", "text": "..."}]} as interchangeable will sooner or later send the wrong shape to the wrong provider and get a 400 back.

By the end of this chapter, three things are true of your harness:

1. The transcript is a typed data structure, not a list of dicts.
2. It translates cleanly to and from at least three provider formats (Anthropic, OpenAI, and a generic OSS adapter).
3. The loop from Chapter 2 still works, unchanged in logic, but now routed through the adapter layer.

{
  "role": "assistant",
  "content": [
    {"type": "tool_use",
     "id": "t_1",
     "name": "calc",
     "input": {"expr": "2+2"}}
  ]
}

{
  "role": "assistant",
  "tool_calls": [
    {"id": "t_1",
     "type": "function",
     "function": {
       "name": "calc",
       "arguments": "{\"expr\":\"2+2\"}"}}
  ]
}

3.1 The Problem With Dicts

Three failure modes, all of which I've shipped to production at least once.

Shape drift. Anthropic uses content blocks: [{"type": "text", "text": "..."}]. OpenAI uses plain strings for user messages and structured objects for assistant tool calls. A dict-based transcript has to guess which shape is live at any moment. The guesses are usually right. The times they're wrong, you find out in the worst possible way: a parser that silently accepts malformed input, a model that hallucinates because a tool result came back empty, a test that passes locally and fails in production.

Role confusion. Is a tool result a user message (Anthropic's convention) or a tool message (OpenAI's)? If your code picks one and hard-codes it, you've locked yourself to that provider. If your code picks the wrong one, you've made your own life harder for no benefit.

Accounting becomes impossible. To track how much context you've consumed, you need to know what each message is — and a list of dicts requires re-parsing every time you want to answer that question, whereas a typed list exposes the structure up front and lets the accountant pattern-match on block kinds instead of guessing.

We're going to fix all three in the same file, and it's going to cost us about eighty lines of dataclasses plus a thin Transcript wrapper.

3.2 The Canonical Shape

Here are the types we'll use for the rest of the book — worth memorizing, because they show up in every chapter from this point on, and the vocabulary ("blocks", "role", "the ToolCall block", "the ReasoningBlock's metadata") recurs without reintroduction.

# src/harness/messages.py
from __future__ import annotations

from dataclasses import dataclass, field
from datetime import datetime, timezone
from typing import Literal
from uuid import uuid4


Role = Literal["user", "assistant", "system"]


@dataclass(frozen=True)
class TextBlock:
    text: str
    kind: Literal["text"] = "text"


@dataclass(frozen=True)
class ToolCall:
    id: str
    name: str
    args: dict
    kind: Literal["tool_call"] = "tool_call"


@dataclass(frozen=True)
class ToolResult:
    call_id: str
    content: str
    is_error: bool = False
    kind: Literal["tool_result"] = "tool_result"


@dataclass(frozen=True)
class ReasoningBlock:
    """Model-internal reasoning ("thinking" on Anthropic, "reasoning" on OpenAI).

    Emitted by reasoning-enabled providers before the final answer or tool
    call. `metadata` holds vendor-specific fields (notably Anthropic's
    opaque `signature`) that the adapter needs to round-trip.
    """
    text: str
    metadata: dict = field(default_factory=dict)
    kind: Literal["reasoning"] = "reasoning"


Block = TextBlock | ToolCall | ToolResult | ReasoningBlock


@dataclass
class Message:
    role: Role
    blocks: list[Block]
    created_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc))
    id: str = field(default_factory=lambda: str(uuid4()))

    @classmethod
    def user_text(cls, text: str) -> "Message":
        return cls(role="user", blocks=[TextBlock(text)])

    @classmethod
    def assistant_text(cls, text: str, *,
                       reasoning: ReasoningBlock | None = None) -> "Message":
        blocks: list[Block] = []
        if reasoning is not None:
            blocks.append(reasoning)
        blocks.append(TextBlock(text))
        return cls(role="assistant", blocks=blocks)

    @classmethod
    def assistant_tool_call(cls, call: ToolCall, *,
                            reasoning: ReasoningBlock | None = None) -> "Message":
        blocks: list[Block] = []
        if reasoning is not None:
            blocks.append(reasoning)
        blocks.append(call)
        return cls(role="assistant", blocks=blocks)

    @classmethod
    def tool_result(cls, result: ToolResult) -> "Message":
        # conventionally attached to the "user" role;
        # the adapter remaps this for providers that use "tool".
        return cls(role="user", blocks=[result])

Five things to notice.

Block is a union of four kinds. Everything a message can contain — plain text, a tool call the assistant wants to make, a result from a tool call, and an optional reasoning trace — is one of those four. The loop never has to ask "what shape is this?"; it pattern-matches on the kind discriminant.

All four block types are frozen. Messages are immutable once created. If you want to modify one, you replace it. This prevents a whole class of bugs where code downstream of the loop mutates a message and the next turn sends different content than was rendered.

Tool results live in a user-role message. This matches Anthropic's convention and is straightforward to remap for OpenAI. The role is a transport detail; the block type is the semantics.

ReasoningBlock rides alongside text or tool calls on an assistant turn. The factory methods assistant_text(..., reasoning=...) and assistant_tool_call(..., reasoning=...) take an optional ReasoningBlock that gets placed before the primary block. That's why the blocks list can have two entries on a reasoning-enabled turn: the reasoning trace first, then the tool call or final answer. More on reasoning below.

Each message gets a UUID and a timestamp. These cost nothing at creation time and save hours when debugging. If your compaction policy later drops a message, the UUID tells you which one.

A closer look at ReasoningBlock

Reasoning models — Anthropic's Extended Thinking, OpenAI's o-series and gpt-5 reasoning models, DeepSeek R1, and several others — produce a chain-of-thought trace before the final answer or tool call. Anthropic calls these blocks thinking, OpenAI calls them reasoning, and the underlying mechanism is the same. The technique traces back to Wei et al.'s 2022 paper "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models," which demonstrated that explicit step-by-step reasoning in the model's output substantially improved performance on arithmetic and commonsense tasks. The 2024–2025 generation of reasoning models turned that prompt-engineering trick into a dedicated internal phase: the model now produces a reasoning trace before its visible answer as a matter of course, and providers expose the trace as a first-class block type rather than asking callers to parse it out of free-form text. We name our internal type after the broader industry term rather than either vendor's.

Three things to know about reasoning tokens before we write any adapter code.

They're billed as output tokens. A task that used to cost 200 output tokens may now cost 2,000 — most of it reasoning. Chapter 7's accountant counts reasoning text against the "history" component when it ends up in the transcript; Chapter 20's budget enforcer sees reasoning as output spend.

They're usually not fed back to the model. Reasoning is the model's private scratchpad. OpenAI's Responses API keeps reasoning server-side and replays it via previous_response_id; if you don't use that flow (and we don't — our architecture is stateless), the model starts fresh each turn and previous reasoning is dropped from input. Anthropic will accept reasoning round-tripped in messages, but only with the opaque signature field attached and only when extended thinking stays enabled. The metadata dict on ReasoningBlock is where the signature lives.

Anthropic requires round-trip when thinking + tools are on. If you enable extended thinking and the model uses a tool, the next request must include the assistant's thinking block (with signature) alongside the tool_use block. Drop the thinking, and the API rejects the request. The Anthropic adapter (a few pages down) handles this: with thinking on, reasoning stays in the transcript and serializes back out; with thinking off, nothing is ever generated.

The matching change on the ProviderResponse side is an extra field that carries whatever reasoning the provider emitted this turn:

# src/harness/providers/base.py (preview — full definition in §3.3)

@dataclass(frozen=True)
class ProviderResponse:
    text: str | None = None
    tool_call_id: str | None = None
    tool_name: str | None = None
    tool_args: dict | None = None
    reasoning_text: str | None = None   # the trace, if any
    input_tokens: int = 0
    output_tokens: int = 0
    reasoning_tokens: int = 0           # subset of output_tokens, broken out

The loop doesn't branch on reasoning — it dispatches on tool call vs. text as before. Reasoning shows up on the response so adapters can persist it, the accountant can count it, and observability (Chapter 18) can surface it.

One factory method on Message ties this together. The loop calls it every turn; adapters decide on translation whether to round-trip the reasoning. Add it inside the Message class — alongside assistant_text, assistant_tool_call, and the rest — not as a module-level function:

# src/harness/messages.py (add to the Message class)

class Message:
    # ... role, blocks, created_at, id, the four existing factory methods ...

    @classmethod
    def from_assistant_response(cls, response) -> "Message":
        """Build an assistant Message from a ProviderResponse.

        Reasoning (if emitted) comes first as a ReasoningBlock; the
        primary output (text or tool call) follows. Vendor-specific
        metadata (OpenAI's encrypted reasoning items, Anthropic's thinking
        signature) is merged into `ReasoningBlock.metadata` so adapters
        can round-trip reasoning on the next turn.
        """
        reasoning = None
        has_reasoning = (
            bool(response.reasoning_text)
            or bool(getattr(response, "reasoning_metadata", None))
        )
        if has_reasoning:
            meta: dict = {"provider_tokens": response.reasoning_tokens}
            meta.update(getattr(response, "reasoning_metadata", None) or {})
            reasoning = ReasoningBlock(
                text=response.reasoning_text or "",
                metadata=meta,
            )
        blocks: list[Block] = []
        if reasoning is not None:
            blocks.append(reasoning)
        if response.tool_calls:
            # One assistant message carries every ToolCall block from
            # this turn — both providers accept multi-tool_use messages
            # on round-trip.
            for ref in response.tool_calls:
                blocks.append(ToolCall(id=ref.id, name=ref.name,
                                        args=dict(ref.args)))
        else:
            blocks.append(TextBlock(text=response.text or ""))
        return cls(role="assistant", blocks=blocks)

That's the whole data model. §3.4's adapters translate each block type to and from the two vendor wire formats; §3.5's loop uses from_assistant_response every turn.

The Transcript is a thin wrapper:

# src/harness/messages.py (continued)

@dataclass
class Transcript:
    messages: list[Message] = field(default_factory=list)
    system: str | None = None

    def append(self, message: Message) -> None:
        self.messages.append(message)

    def extend(self, messages: list[Message]) -> None:
        self.messages.extend(messages)

    def last(self) -> Message | None:
        return self.messages[-1] if self.messages else None

    def __len__(self) -> int:
        return len(self.messages)

System prompts are separate from the messages list. Every provider handles them slightly differently — Anthropic takes it as a top-level parameter, OpenAI as the first message in the list, some OSS models bake it into a template. Keeping it apart means the adapters decide how to inject it.

3.3 The Provider Protocol, Upgraded

Now that we have typed messages, the Provider protocol can use them directly. Replace the old base.py:

# src/harness/providers/base.py
from __future__ import annotations

from dataclasses import dataclass, field
from typing import Protocol

from ..messages import Transcript


@dataclass(frozen=True)
class ProviderResponse:
    """A provider's response to one complete() call.

    Exactly one of (text, tool_call) is set. `reasoning_text` is
    orthogonal — it may accompany either a text answer or a tool call
    when the provider is configured to emit reasoning. `reasoning_metadata`
    holds vendor-specific replay data (OpenAI's encrypted reasoning items,
    Anthropic's thinking signature) that `Message.from_assistant_response`
    folds into the `ReasoningBlock.metadata` so the adapter can round-trip
    reasoning on the next turn.
    """
    text: str | None = None
    tool_call_id: str | None = None
    tool_name: str | None = None
    tool_args: dict | None = None
    reasoning_text: str | None = None
    reasoning_metadata: dict = field(default_factory=dict)
    input_tokens: int = 0
    output_tokens: int = 0
    reasoning_tokens: int = 0  # subset of output_tokens, broken out for accounting

    @property
    def is_tool_call(self) -> bool:
        return self.tool_name is not None

    @property
    def is_final(self) -> bool:
        return self.text is not None and self.tool_name is None


class Provider(Protocol):
    name: str

    def complete(self, transcript: Transcript, tools: list[dict]) -> ProviderResponse:
        ...

Three additions since Chapter 2. First, input_tokens and output_tokens — the provider knows what it cost; we want that visible at the protocol level so Chapter 7's accountant doesn't have to estimate. Second, reasoning_text / reasoning_tokens / reasoning_metadata — see §3.2's discussion; every adapter populates these when the model emits a trace, and zero-fills them otherwise. Third, name as a discriminator, so logs and traces can identify which provider served a given response.

3.4 The Adapters

Three adapters. Keep them small; the job of an adapter is to translate, not to decide.

The Anthropic adapter

# src/harness/providers/anthropic.py
from __future__ import annotations

import os
from typing import Any

from ..messages import (
    Block, Message, ReasoningBlock, TextBlock, ToolCall, ToolResult, Transcript,
)
from .base import Provider, ProviderResponse


class AnthropicProvider(Provider):
    name = "anthropic"

    def __init__(self, model: str = "claude-sonnet-4-6",
                 client: Any | None = None,
                 enable_thinking: bool = False,
                 thinking_budget_tokens: int = 2000,
                 max_tokens: int = 4096) -> None:
        self.model = model
        self.enable_thinking = enable_thinking
        self.thinking_budget_tokens = thinking_budget_tokens
        self.max_tokens = max_tokens
        if client is None:
            # Import the specific symbol (not `import anthropic`) so there's no
            # ambiguity with this module's own name, `harness.providers.anthropic`.
            from anthropic import Anthropic  # external SDK
            client = Anthropic()
        self._client = client

    def complete(self, transcript: Transcript, tools: list[dict]) -> ProviderResponse:
        kwargs: dict[str, Any] = {
            "model": self.model,
            "max_tokens": self.max_tokens,
            "messages": [_to_anthropic(m, self.enable_thinking)
                          for m in transcript.messages],
            "tools": tools,
        }
        if transcript.system:
            kwargs["system"] = transcript.system
        if self.enable_thinking:
            kwargs["thinking"] = {
                "type": "enabled",
                "budget_tokens": self.thinking_budget_tokens,
            }
        # Parallel tool use stays on (Anthropic's default). Chapter 5's
        # `accumulate` collects every tool_use into `ProviderResponse.tool_calls`,
        # and the loop dispatches them sequentially in arrival order.

        raw = self._client.messages.create(**kwargs)
        return _from_anthropic(raw)


def _to_anthropic(message: Message, keep_reasoning: bool) -> dict:
    # Drop ReasoningBlocks when thinking isn't enabled — the API rejects
    # `thinking` blocks without the feature turned on. With thinking on,
    # reasoning (including its signature) must round-trip.
    content: list[dict] = []
    for block in message.blocks:
        if isinstance(block, ReasoningBlock) and not keep_reasoning:
            continue
        content.append(_block_to_anthropic(block))
    return {"role": message.role, "content": content}


def _block_to_anthropic(block: Block) -> dict:
    match block:
        case TextBlock(text=t):
            return {"type": "text", "text": t}
        case ToolCall(id=i, name=n, args=a):
            return {"type": "tool_use", "id": i, "name": n, "input": a}
        case ToolResult(call_id=i, content=c, is_error=err):
            return {"type": "tool_result", "tool_use_id": i,
                    "content": c, "is_error": err}
        case ReasoningBlock(text=t, metadata=meta):
            out: dict[str, Any] = {"type": "thinking", "thinking": t}
            if (sig := meta.get("signature")) is not None:
                out["signature"] = sig  # required on round-trip
            return out


def _from_anthropic(raw: Any) -> ProviderResponse:
    # Gather any thinking trace first — it may accompany either a tool_use
    # or a text answer, and we want to preserve it on ProviderResponse so
    # the loop's `Message.from_assistant_response` puts it in the transcript.
    thinking_texts = [b.thinking for b in raw.content if b.type == "thinking"]
    reasoning_text = "\n".join(thinking_texts) if thinking_texts else None

    for block in raw.content:
        if block.type == "tool_use":
            return ProviderResponse(
                tool_call_id=block.id,
                tool_name=block.name,
                tool_args=dict(block.input),
                reasoning_text=reasoning_text,
                input_tokens=raw.usage.input_tokens,
                output_tokens=raw.usage.output_tokens,
            )

    # No tool call → concatenate text blocks for the final answer.
    texts = [b.text for b in raw.content if b.type == "text"]
    return ProviderResponse(
        text="\n".join(texts),
        reasoning_text=reasoning_text,
        input_tokens=raw.usage.input_tokens,
        output_tokens=raw.usage.output_tokens,
    )

The match statement in _block_to_anthropic is the pattern we'll use throughout the book for discriminating blocks. It's exhaustive: adding a new block type and forgetting to handle it becomes a MatchError rather than silent data loss — and notice ReasoningBlock is a first-class case alongside text/tool_use/tool_result.

_from_anthropic does three passes over the response content — thinking first (so we can attach it regardless of which primary path fires), then tool_use, then falling back to text. This mirrors what Chapter 5's streaming version does; the only difference is that streaming emits ReasoningDelta events as they arrive rather than collecting them at the end.

One Anthropic-specific wrinkle the _block_to_anthropic case covers: a thinking block round-tripped on a subsequent turn must carry its opaque signature field. We stashed the signature in ReasoningBlock.metadata when we first captured it (the real harness streaming adapter reads it off the completed block); passing it back is how the API verifies the reasoning hasn't been tampered with. Miss the signature and the API rejects the request.

The OpenAI adapter

A word on which OpenAI API to target, and a short detour through how we got here. OpenAI introduced function calling as a first-class feature in Chat Completions in June 2023, establishing the pattern of tool calls as typed output blocks that every major vendor has since adopted. Before that release, tool use in production systems was a prompt-engineering exercise: you'd ask the model to emit JSON in a particular format, then parse free-form text looking for it, and a substantial fraction of what people called "tool-use failures" were actually parser failures — the model hadn't produced invalid output, the downstream code had simply misread it. The typed-block approach — picked up by Anthropic in 2024 and now the de facto standard across the industry — is the protocol-level shift that makes this book's ToolCall and ToolResult types possible in the first place.

OpenAI currently ships two APIs on top of that foundation. Chat Completions (client.chat.completions.create) is the 2023-era surface. Responses (client.responses.create, introduced in 2025) is the newer one, and OpenAI now actively recommends Responses for agentic use — it is stateful, supports built-in tools like web_search and code_interpreter, and powers OpenAI's own Agents SDK. Chat Completions remains available but is no longer the preferred surface for new work.

We use Responses, for two reasons worth naming.

First, vendor direction. When the platform owner says "this is the supported agentic surface going forward," a book teaching agent harnesses that ignores the recommendation ages badly. Chat Completions will keep working, but new capabilities — structured outputs on tool calls, built-in tools, the richer streaming event vocabulary — are shipping on Responses first.

Second, coverage is closing fast on the OSS side. vLLM and Ollama both speak Responses now, and more open-source servers ship support each quarter. Our LocalProvider — the subclass pointing AsyncOpenAI at a local endpoint — works against any server that implements /v1/responses. If you hit a server that only exposes /v1/chat/completions, add a sibling OpenAIChatCompletionsProvider against the same Provider protocol; that's exactly what the adapter seam is for, and it's the kind of change that touches one file and nothing else.

The Responses API is a little more verbose than Chat Completions — input items are typed (function_call, function_call_output, message) rather than role-tagged strings — but the typing absorbs ambiguity that the Chat Completions shape papered over. Tool calls and tool results become first-class input items instead of array-nested dict keys, which is the same philosophical move our Transcript made, and the adapter has correspondingly less translation work to do.

# src/harness/providers/openai.py
from __future__ import annotations

import json
from typing import Any

from typing import Literal

from ..messages import (
    Block, Message, ReasoningBlock, TextBlock, ToolCall, ToolResult, Transcript,
)
from .base import Provider, ProviderResponse


ReasoningEffort = Literal["minimal", "low", "medium", "high"]


class OpenAIProvider(Provider):
    name = "openai"

    def __init__(self, model: str = "gpt-5", client: Any | None = None,
                 reasoning_effort: ReasoningEffort | None = None) -> None:
        self.model = model
        self.reasoning_effort = reasoning_effort
        if client is None:
            # Import the specific symbol (not `import openai`) so there's no
            # ambiguity with this module's own name, `harness.providers.openai`.
            from openai import OpenAI  # external SDK
            client = OpenAI()
        self._client = client

    def complete(self, transcript: Transcript, tools: list[dict]) -> ProviderResponse:
        input_items: list[dict] = []
        for m in transcript.messages:
            input_items.extend(_to_responses_input(m))

        responses_tools = [_tool_to_responses(t) for t in tools] if tools else None
        kwargs: dict[str, Any] = {"model": self.model, "input": input_items}
        if transcript.system:
            kwargs["instructions"] = transcript.system  # system prompt, top-level
        if responses_tools:
            kwargs["tools"] = responses_tools
            # Parallel tool calls stay on (Responses default). Chapter 5's
            # `accumulate` handles the batch.
        if self.reasoning_effort is not None:
            kwargs["reasoning"] = {"effort": self.reasoning_effort}
            # Ask Responses for the encrypted reasoning blob so we can replay
            # it across turns without `previous_response_id`. We run stateless
            # — `store=False` opts out of server-side conversation storage.
            kwargs["include"] = ["reasoning.encrypted_content"]
            kwargs["store"] = False

        raw = self._client.responses.create(**kwargs)
        return _from_responses(raw)


def _tool_to_responses(tool: dict) -> dict:
    # Our canonical tool shape is Anthropic-flavoured: {name, description, input_schema}.
    # Responses flattens function tools: {type, name, description, parameters}.
    return {
        "type": "function",
        "name": tool["name"],
        "description": tool.get("description", ""),
        "parameters": tool.get("input_schema", tool.get("parameters", {})),
    }


def _to_responses_input(message: Message) -> list[dict]:
    # Tool results become function_call_output items (no role — typed directly).
    if any(isinstance(b, ToolResult) for b in message.blocks):
        return [
            {"type": "function_call_output", "call_id": b.call_id, "output": b.content}
            for b in message.blocks if isinstance(b, ToolResult)
        ]

    # Reasoning items get replayed to Responses so chain-of-thought carries
    # across turns. We stashed the opaque `id` + `encrypted_content` in
    # metadata on the way in; if the metadata is missing (e.g. the
    # ReasoningBlock came from Anthropic, or reasoning wasn't enabled on
    # the provider that produced it), we skip — Responses won't accept a
    # raw text reasoning item.
    items: list[dict] = []
    for b in message.blocks:
        if isinstance(b, ReasoningBlock):
            for spec in b.metadata.get("openai_items") or []:
                item: dict[str, Any] = {
                    "type": "reasoning",
                    "summary": spec.get("summary") or [],
                }
                if rid := spec.get("id"):
                    item["id"] = rid
                if enc := spec.get("encrypted_content"):
                    item["encrypted_content"] = enc
                items.append(item)

    # Assistant tool calls become function_call items.
    if any(isinstance(b, ToolCall) for b in message.blocks):
        for b in message.blocks:
            if isinstance(b, ToolCall):
                items.append({
                    "type": "function_call",
                    "call_id": b.id,
                    "name": b.name,
                    "arguments": json.dumps(b.args),
                })
        return items

    # Plain text keeps its role/content shape.
    text = "\n".join(b.text for b in message.blocks if isinstance(b, TextBlock))
    return [{"role": message.role, "content": text}]


def _from_responses(raw: Any) -> ProviderResponse:
    # Extract reasoning first so it attaches to whichever primary output fires.
    # Two things to collect from each reasoning item: the summary text (for
    # humans / logs) and the opaque id + encrypted_content (for replay on
    # the next turn via `_to_responses_input`).
    reasoning_parts: list[str] = []
    openai_items: list[dict] = []
    for item in raw.output:
        if item.type == "reasoning":
            for summary in getattr(item, "summary", []) or []:
                text = getattr(summary, "text", None)
                if text:
                    reasoning_parts.append(text)
            openai_items.append({
                "id": getattr(item, "id", "") or "",
                "encrypted_content": getattr(item, "encrypted_content", "") or "",
                "summary": [],  # send back empty; summaries aren't required on replay
            })
    reasoning_text = "\n".join(reasoning_parts) if reasoning_parts else None
    reasoning_metadata = {"openai_items": openai_items} if openai_items else {}

    # Responses breaks reasoning tokens out under usage.output_tokens_details.
    details = getattr(raw.usage, "output_tokens_details", None)
    reasoning_tokens = int(getattr(details, "reasoning_tokens", 0) or 0) if details else 0

    # If the model issued a tool call, return the first one.
    for item in raw.output:
        if item.type == "function_call":
            return ProviderResponse(
                tool_call_id=item.call_id,
                tool_name=item.name,
                tool_args=json.loads(item.arguments),
                reasoning_text=reasoning_text,
                reasoning_metadata=reasoning_metadata,
                input_tokens=raw.usage.input_tokens,
                output_tokens=raw.usage.output_tokens,
                reasoning_tokens=reasoning_tokens,
            )

    # Otherwise, concatenate the output_text blocks from any message items.
    texts: list[str] = []
    for item in raw.output:
        if item.type == "message":
            for block in item.content:
                if block.type == "output_text":
                    texts.append(block.text)
    return ProviderResponse(
        text="\n".join(texts),
        reasoning_text=reasoning_text,
        reasoning_metadata=reasoning_metadata,
        input_tokens=raw.usage.input_tokens,
        output_tokens=raw.usage.output_tokens,
        reasoning_tokens=reasoning_tokens,
    )

Same shape as _from_anthropic: reasoning comes first, then the primary path branches on tool call vs. text. The difference is how the two APIs surface the count — Anthropic rolls reasoning tokens into usage.output_tokens, OpenAI breaks them out under usage.output_tokens_details.reasoning_tokens. Our reasoning_tokens field captures whichever the provider exposes, which lets Chapter 20's router and Chapter 18's traces show the breakdown without caring which vendor produced it.

One more difference: _from_responses captures the opaque id and encrypted_content from every reasoning item into reasoning_metadata.openai_items. Those fields are what make stateless reasoning replay possible — the next turn's _to_responses_input emits a matching {type: "reasoning", id, encrypted_content} item so the model sees its own chain-of-thought from the previous turn. Without this, reasoning models effectively "forget" their thinking each turn. The include=["reasoning.encrypted_content"] request flag and store=False (we saw both up in complete()) are what make the encrypted blob show up in the first place; together they give us chain-of-thought continuity without relying on OpenAI's server-side previous_response_id conversation storage. Anthropic handles the same round-trip via signature on thinking blocks; our harness-internal ReasoningBlock.metadata dict holds whichever convention applies.

Notice _to_responses_input still returns a list, not one item. One of our Message objects can expand into multiple Responses input items — a function_call followed by a function_call_output on the next turn, or two function calls from a single assistant turn. The adapter absorbs the asymmetry; the rest of the harness never sees it.

Also notice the translation is almost mechanical. Transcript already distinguishes ToolCall from ToolResult as typed block subclasses; Responses already distinguishes function_call from function_call_output as typed input items. Both sides agree that tool calls and tool results are different things with different shapes — all the adapter does is rename the fields.

The OSS adapter

For open-source models served through a local endpoint (llama.cpp, vLLM, Ollama), we use OpenAI-compatible mode — almost every serious local server supports it. The adapter is one line:

# src/harness/providers/local.py
from __future__ import annotations

from .openai import OpenAIProvider


class LocalProvider(OpenAIProvider):
    name = "local"

    def __init__(self, model: str = "llama-3.1-8b-instruct",
                 base_url: str = "http://localhost:8000/v1") -> None:
        # Import the specific symbol so there's no ambiguity with the sibling
        # module `harness.providers.openai`.
        from openai import OpenAI  # external SDK
        client = OpenAI(base_url=base_url, api_key="not-needed")
        super().__init__(model=model, client=client)

LocalProvider inherits all the OpenAI translation. This works for any server that speaks OpenAI's chat-completions protocol, which is the de facto OSS standard.

Turning reasoning on

§3.2 introduced ReasoningBlock as a first-class block type and both adapter snippets above already translate it. What we haven't shown is how a caller turns reasoning on. Both knobs are plain constructor arguments; the rest of the harness is unaffected.

from harness.providers.anthropic import AnthropicProvider
from harness.providers.openai import OpenAIProvider

anthropic = AnthropicProvider(
    enable_thinking=True,
    thinking_budget_tokens=4000,
    max_tokens=16000,     # must be larger than the thinking budget
)

openai = OpenAIProvider(
    reasoning_effort="medium",   # "minimal" | "low" | "medium" | "high"
)

The two providers agree on the outcome: reasoning tokens stream, accumulate into ProviderResponse.reasoning_text, the loop's Message.from_assistant_response puts them in the transcript as a ReasoningBlock, and the adapter decides whether to round-trip them on the way back out. Same loop code, reasoning-agnostic. That's the adapter seam doing its job.

3.5 Updating the Loop

The Chapter 2 loop used raw dicts. Now it uses Transcript and typed messages. The logic is identical; the types tighten:

# src/harness/agent.py
from __future__ import annotations

from typing import Callable

from .messages import Message, TextBlock, ToolCall, ToolResult, Transcript
from .providers.base import Provider


MAX_ITERATIONS = 20


def run(
    provider: Provider,
    tools: dict[str, Callable[..., str]],
    tool_schemas: list[dict],
    user_message: str,
    system: str | None = None,
) -> str:
    transcript = Transcript(system=system)
    transcript.append(Message.user_text(user_message))

    for _ in range(MAX_ITERATIONS):
        response = provider.complete(transcript, tool_schemas)

        if response.is_final:
            # from_assistant_response preserves reasoning (if any) alongside
            # the final text as a single assistant Message. With reasoning off
            # it's equivalent to Message.assistant_text(response.text).
            transcript.append(Message.from_assistant_response(response))
            return response.text or ""

        # Same story on the tool-call branch: reasoning rides with the
        # ToolCall blocks in one assistant message.
        transcript.append(Message.from_assistant_response(response))

        # Dispatch each call in arrival order. One tool_result message per
        # call; Chapter 5 keeps the same loop shape with the registry.
        for ref in response.tool_calls:
            try:
                result_text = tools[ref.name](**ref.args)
                result = ToolResult(call_id=ref.id, content=result_text)
            except KeyError:
                result = ToolResult(call_id=ref.id,
                                    content=f"unknown tool: {ref.name}",
                                    is_error=True)
            except Exception as e:
                result = ToolResult(call_id=ref.id, content=str(e),
                                    is_error=True)
            transcript.append(Message.tool_result(result))

    raise RuntimeError(f"agent did not finish in {MAX_ITERATIONS} iterations")

Three things earned by the refactor. The transcript now has a system field — Anthropic will pass it at the top level, OpenAI as the first message, your OSS provider however it wants. The loop doesn't care. Message.from_assistant_response(response) is the one-liner that persists both the primary output (text or tool call) and any ReasoningBlock the provider emitted, in a single assistant Message — the loop stays reasoning-agnostic. And the try/except around tool dispatch addresses Break 1 and Break 3 from Chapter 2 in a minimal way — the loop no longer crashes on unknown tools or exceptions; it returns a structured error to the model and lets it recover. This is a preview; Chapter 6 does it properly with schema validation and dedup.

3.6 Swapping Providers

The payoff. The Chapter 2 mock still works — it uses Transcript now, but the shape of the call hasn't changed:

# src/harness/providers/mock.py (updated)
from ..messages import Transcript
from .base import Provider, ProviderResponse


class MockProvider:
    name = "mock"

    def __init__(self, responses: list[ProviderResponse]) -> None:
        self._responses = list(responses)
        self._index = 0

    def complete(self, transcript: Transcript, tools: list[dict]) -> ProviderResponse:
        if self._index >= len(self._responses):
            raise RuntimeError("mock ran out of responses")
        response = self._responses[self._index]
        self._index += 1
        return response

The real providers drop in behind the same interface:

# examples/ch03_real_provider.py
import os
import sys

from harness.agent import run
from harness.providers.anthropic import AnthropicProvider
from harness.providers.openai import OpenAIProvider
from harness.providers.local import LocalProvider


def calc(expression: str) -> str:
    return str(eval(expression, {"__builtins__": {}}, {}))


tool_schemas = [{
    "name": "calc",
    "description": "Evaluate a Python arithmetic expression.",
    "input_schema": {
        "type": "object",
        "properties": {"expression": {"type": "string"}},
        "required": ["expression"],
    },
}]


# Choose the provider once. The rest of the script doesn't care which one.
provider_name = os.environ.get("PROVIDER", "anthropic")
required_env = {
    "anthropic": "ANTHROPIC_API_KEY",
    "openai": "OPENAI_API_KEY",
    "local": None,  # local servers don't need a key
}
env_var = required_env.get(provider_name)
if env_var and not os.environ.get(env_var):
    sys.exit(
        f"error: PROVIDER={provider_name} requires {env_var}. "
        f"Set it and re-run. For the local provider, use PROVIDER=local."
    )

provider = {
    "anthropic": AnthropicProvider,
    "openai": OpenAIProvider,
    "local": LocalProvider,
}[provider_name]()

answer = run(
    provider=provider,
    tools={"calc": calc},
    tool_schemas=tool_schemas,
    user_message="What is 17 * 23, minus 100?",
)
print(answer)

Before running it, set the API keys. The Anthropic SDK reads ANTHROPIC_API_KEY from the environment; the OpenAI SDK reads OPENAI_API_KEY. If either is missing, the SDK raises TypeError: Could not resolve authentication method … deep inside its HTTP layer — an error whose stack trace is long enough that it's worth learning to recognise on sight.

export ANTHROPIC_API_KEY=sk-ant-...
export OPENAI_API_KEY=sk-...

The LocalProvider path needs no key — it points at an OpenAI-compatible local server (llama.cpp, vLLM, Ollama, LM Studio). Set OPENAI_API_KEY=not-needed or leave it; the local server ignores it.

Run it three times:

PROVIDER=anthropic uv run examples/ch03_real_provider.py
PROVIDER=openai    uv run examples/ch03_real_provider.py
PROVIDER=local     uv run examples/ch03_real_provider.py  # assumes local endpoint

Three different models, same loop, same tool, same transcript type, no code change. That's the seam paying for itself.

Commit:

git add -A && git commit -m "ch03: typed transcript + Anthropic/OpenAI/local adapters"
git tag ch03-transcript

3.7 Why Tool Schemas Are Still Dicts

You may have noticed that tool_schemas is a list[dict]. That's deliberate, and temporary. Chapter 4 introduces a Tool class that owns its schema, its callable, its side-effect declaration, and its validator. At that point the adapter stops taking schemas as dicts and takes them as typed objects.

Why wait? Because the shape of a JSON schema is not complicated enough to earn its own abstraction yet, and every abstraction you add before its pain has been felt is debt. Chapter 4's Tool is motivated by the fact that two breaks from Chapter 2 (unknown tool, schema mismatch) are still being handled ad hoc in the loop's try/except. We fix them properly when we have the right shape to hang the fix on.

3.8 Try It Yourself

1. Write a fourth adapter. Pick a provider we haven't covered — Gemini, Cohere, AWS Bedrock, Together AI, Groq. Read its docs. Write an adapter implementing the Provider protocol. Run the calculator example against it. How many lines did you need? Which parts were trivial? Which were friction?
2. Break the translation deliberately. Mutate a TextBlock after creating it. What happens? (Hint: it's frozen, so the attempt raises.) Now write a subtly-wrong OpenAI translator that forgets to serialize tool_calls arguments as JSON strings. Run the example. Observe the provider's error. Note what the error told you and what it didn't.
3. Add tracing to the adapter layer. Before and after each complete() call, log the input token count, the output token count, and the wall-clock duration. You've just built a minimal version of what Chapter 18 will formalize as observability. Keep it — we'll replace it with real OpenTelemetry spans later.

Chapter 4. The Tool Protocol

Previously: typed messages, typed transcripts, three provider adapters. The loop no longer crashes on unknown tools, but its fix is ad hoc — a try/except in the dispatch. We owe ourselves a proper tool abstraction.

A tool is a contract. It has a name a model can guess, a description a model can read, a schema a model must match, a callable we execute on a match, and a side-effect profile that determines who has to ask permission before we run it. The tools you ship are the surface through which your agent reaches into the world, and every well-documented production failure in this space — hallucinated tool calls, output truncation, tool cliff, prompt injection — is a failure of the tool surface.

This chapter builds the Tool abstraction we'll use for the rest of the book. By the end, three things are true:

1. Tools carry their own schema, description, and side-effect declaration.
2. A ToolRegistry dispatches calls and rejects unknown names before they reach your code.
3. The tool schema sent to the provider is derived from the tool, not hand-maintained.

We still don't validate argument shapes before dispatch — that's Chapter 6. We're building the object; Chapter 6 attaches the validator.

4.1 The Tool as an Interface, Not a Function

The research arc here is short but worth naming. Schick et al.'s 2023 "Toolformer: Language Models Can Teach Themselves to Use Tools" was the paper that opened the tool-use research area as a distinct subfield — it demonstrated that language models could learn to call external APIs (calculators, translators, search) in the middle of text generation, which in turn established tool use as a research problem rather than a prompting trick. Yang et al.'s 2024 "SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering" made the sharp design point that followed from Toolformer's opening: tool design is interface design for a non-human user with very specific cognitive constraints. A model reads your description before it decides what to call, it doesn't remember your README, it can't ask a clarifying question, and it infers behavior from names. Three consequences follow:

Names are semantic. send_message is very different from send_email. read_file implies non-destructive; open_file is ambiguous. If two tools could be confused, they will be.

Descriptions are contracts. "Sends a notification" doesn't tell a model whether the notification costs money, reaches production customers, is rate-limited, or requires a subject line. A well-specified tool description covers what it does, what it does not do, what it requires as preconditions, and what side effects it produces. "Sends a notification" is a bug report waiting to happen.

Schemas are hard edges. A tool with an optional argument that's actually required, or a string argument that should have been an enum, gets misused in proportion to how soft its edges are. Every string where an enum would fit is a chance for the model to be creative in ways you don't want.

We'll encode all three — name, description, schema — as first-class fields of Tool. Side effects get a field too, because the permission layer in Chapter 14 will need to gate on them.

4.2 The Tool Dataclass

# src/harness/tools/base.py
from __future__ import annotations

from dataclasses import dataclass, field
from typing import Callable, Literal


SideEffect = Literal["read", "write", "network", "mutate"]


@dataclass(frozen=True)
class Tool:
    """A callable exposed to the model.

    name        -- stable identifier the model calls by.
    description -- contract text the model reads. Must state scope,
                   preconditions, and side effects in plain English.
    input_schema -- JSON Schema for the arguments dict.
    run         -- the callable. Accepts kwargs matching the schema;
                   returns a string (what the model will see as the result).
    side_effects -- declared effect tags. Used by the permission layer.
    """
    name: str
    description: str
    input_schema: dict
    run: Callable[..., str]
    side_effects: frozenset[SideEffect] = field(default_factory=frozenset)

    def schema_for_provider(self) -> dict:
        """The dict shape providers expect (Anthropic-flavored)."""
        return {
            "name": self.name,
            "description": self.description,
            "input_schema": self.input_schema,
        }

Four tags in SideEffect:

• read — the tool only reads state. Safe to retry, safe to run in parallel, never needs an idempotency key.
• write — modifies local state (files, scratchpad). Needs write ownership; usually safe to retry with idempotency.
• network — reaches an external service. Needs egress permission; retries require vendor-side idempotency.
• mutate — has externally-visible irreversible side effects (sending an email, charging a card, deleting a row). The permission layer may require human approval; retries need real idempotency keys.

These tags aren't load-bearing yet. In Chapter 14, the PermissionManager uses them to decide what gets gated. In Chapter 21, the Checkpointer uses them to decide what needs idempotency protection. We declare them now so every tool we write has them from the start.

4.3 The Decorator

Building Tool instances by hand every time is tedious, and the boilerplate obscures what's actually specific to each tool. A decorator gives us lighter ergonomics and lets the tool's function signature, docstring, and type hints do most of the work:

# src/harness/tools/decorator.py
from __future__ import annotations

import inspect
import typing
from typing import Callable, get_type_hints

from .base import SideEffect, Tool


def tool(
    name: str | None = None,
    description: str | None = None,
    side_effects: set[SideEffect] | frozenset[SideEffect] = frozenset(),
) -> Callable[[Callable[..., str]], Tool]:
    """Turn a plain function into a Tool.

    The input schema is inferred from type hints. The function's docstring
    is used as the description if not provided explicitly.
    """
    def wrap(fn: Callable[..., str]) -> Tool:
        actual_name = name or fn.__name__
        actual_description = description or (fn.__doc__ or "").strip()
        if not actual_description:
            raise ValueError(f"tool {actual_name!r} has no description")

        schema = _schema_from_signature(fn)

        return Tool(
            name=actual_name,
            description=actual_description,
            input_schema=schema,
            run=fn,
            side_effects=frozenset(side_effects),
        )
    return wrap


def _schema_from_signature(fn: Callable[..., str]) -> dict:
    sig = inspect.signature(fn)
    hints = get_type_hints(fn)
    properties: dict[str, dict] = {}
    required: list[str] = []
    for pname, param in sig.parameters.items():
        if pname == "self":
            continue
        hint = hints.get(pname, str)
        properties[pname] = _type_to_schema(hint)
        if param.default is inspect.Parameter.empty:
            required.append(pname)
    return {
        "type": "object",
        "properties": properties,
        "required": required,
    }


def _type_to_schema(t: type) -> dict:
    origin = typing.get_origin(t)
    if origin is typing.Union or origin is types_union():
        args = [a for a in typing.get_args(t) if a is not type(None)]
        if len(args) == 1:
            return _type_to_schema(args[0])
    if t is str:
        return {"type": "string"}
    if t is int:
        return {"type": "integer"}
    if t is float:
        return {"type": "number"}
    if t is bool:
        return {"type": "boolean"}
    if origin is list:
        return {"type": "array", "items": _type_to_schema(typing.get_args(t)[0])}
    return {"type": "string"}  # fallback


def types_union():
    import types
    return types.UnionType

_schema_from_signature is deliberately simple. It handles str, int, float, bool, list[T], and Optional[T] — enough to cover every tool we write in the book. If you need enums, nested objects, or constraints (minLength, pattern, minimum), you can either extend this function or pass an explicit input_schema to a manual Tool(...) call. Most serious production harnesses use Pydantic or pydantic.TypeAdapter for this; we'll adopt that in Chapter 6 when schema validation starts mattering.

Usage:

from harness.tools.decorator import tool


@tool(side_effects={"read"})
def calc(expression: str) -> str:
    """Evaluate a Python arithmetic expression.

    Accepts: standard Python syntax with +, -, *, /, **, parentheses.
    Rejects: imports, function calls, attribute access.
    Side effects: none.
    """
    import ast
    tree = ast.parse(expression, mode="eval")
    for node in ast.walk(tree):
        if not isinstance(node, (ast.Expression, ast.BinOp, ast.UnaryOp,
                                 ast.Num, ast.Constant, ast.operator,
                                 ast.unaryop, ast.Load)):
            raise ValueError(f"not allowed in expression: {type(node).__name__}")
    return str(eval(compile(tree, "<expr>", mode="eval"), {"__builtins__": {}}))

Notice the docstring is doing real work. The model reads it; "accepts / rejects / side effects" is a pattern that makes the tool hard to misuse. Compare that to "evaluates math" and you can see the difference it makes to the model's selection behavior.

4.4 The Registry

The registry holds tools, renders their schemas for providers, and dispatches calls by name.

# src/harness/tools/registry.py
from __future__ import annotations

from dataclasses import dataclass

from ..messages import ToolResult
from .base import Tool


class UnknownToolError(Exception):
    pass


@dataclass
class ToolRegistry:
    tools: dict[str, Tool]

    def __init__(self, tools: list[Tool] | None = None) -> None:
        self.tools = {}
        for t in tools or []:
            self.add(t)

    def add(self, tool: Tool) -> None:
        if tool.name in self.tools:
            raise ValueError(f"duplicate tool name: {tool.name}")
        self.tools[tool.name] = tool

    def schemas(self) -> list[dict]:
        return [t.schema_for_provider() for t in self.tools.values()]

    def dispatch(self, name: str, args: dict, call_id: str) -> ToolResult:
        if name not in self.tools:
            return ToolResult(
                call_id=call_id,
                content=(f"unknown tool: {name}. "
                         f"available: {sorted(self.tools.keys())}"),
                is_error=True,
            )
        tool = self.tools[name]
        try:
            content = tool.run(**args)
        except TypeError as e:
            return ToolResult(
                call_id=call_id,
                content=f"argument error for {name}: {e}",
                is_error=True,
            )
        except Exception as e:
            return ToolResult(
                call_id=call_id,
                content=f"{name} raised {type(e).__name__}: {e}",
                is_error=True,
            )
        return ToolResult(call_id=call_id, content=content)

The registry's contract is narrow: it knows tools, it knows schemas, it dispatches. What it does not do yet: validate arguments against the schema before dispatch, detect loops, enforce permissions, measure cost. All of those are chapters of their own. The registry is the seam they'll plug into.

Two design choices worth naming.

dispatch returns a ToolResult, never raises. The loop should not have to wrap every call in try/except; that's what the registry is for. An unknown tool is not an exception; it's a structured error the model can read and recover from. This is the explicit handling the Chapter 2 try/except was approximating.

The error messages name the available tools. When the model calls calculator instead of calc, the error tells it so. That's not decoration — it's how the model corrects itself on the next turn. A registry that says "unknown tool" without naming alternatives is throwing away a free learning signal.

4.5 The Loop, Threaded Through the Registry

The Chapter 3 loop built a dict[str, Callable] and called it directly. Now it uses the registry.

# src/harness/agent.py
from __future__ import annotations

from .messages import Message, Transcript, ToolCall
from .providers.base import Provider
from .tools.registry import ToolRegistry


MAX_ITERATIONS = 20


def run(
    provider: Provider,
    registry: ToolRegistry,
    user_message: str,
    transcript: Transcript | None = None,
    system: str | None = None,
) -> str:
    if transcript is None:
        transcript = Transcript(system=system)
    transcript.append(Message.user_text(user_message))

    for _ in range(MAX_ITERATIONS):
        response = provider.complete(transcript, registry.schemas())

        if response.is_final:
            transcript.append(Message.from_assistant_response(response))
            return response.text or ""

        # One assistant message with every ToolCall block from this turn.
        transcript.append(Message.from_assistant_response(response))

        # Dispatch each call in arrival order (Chapter 5 details the
        # ProviderResponse.tool_calls tuple; here it's usually one call).
        for ref in response.tool_calls:
            result = registry.dispatch(ref.name, ref.args, ref.id)
            transcript.append(Message.tool_result(result))

    raise RuntimeError(f"agent did not finish in {MAX_ITERATIONS} iterations")