The judgment calls

Chapter 1 handed you Yan Wang’s ruler, so here is the short version. In his October 2 guest lecture he said “Every hot noun bundles a lasting problem with packaging from its year”¹. He separates the two with two questions: “If the model were 10x better, would this still matter?” and “What problem is it actually solving, and do I still have it?”¹ Understanding a lasting problem “appreciates”; packaging “depreciates”¹.

In chapter 1 we pointed the ruler at protocols. Here we point it at decisions, and the same deck shows why that works. When Yan Wang split LangChain, the lasting problem he named was “Deciding what the model sees, what it may do, when it stops, and when to ask a human”¹. His note under it reads: “The loop is short; these decisions are the work”¹. Those are this chapter’s judgments. What the model sees is judgments 1 and 2. What it may do, and when a human is asked, are judgments 3 and 4. How the tool reaches the model at all is judgment 5.

His essays add a map for where the judgments live. He splits putting a model into a product into four layers: model, protocol, runtime and contract². The protocol layer asks questions such as “MCP or RESTful API?”, and the contract layer asks “What does success actually look like?”². Most packaging sits in the protocol layer. Most of this chapter’s judgments sit in the runtime layer, where tools get invoked, and in the contract layer, where you decide what counts as safe and done. His contract layer is about success criteria; it is not the tool contract from chapter 2, which judgments 1 and 2 below refer to.

He also offers a habit to keep a judgment once you have made it: “externalize first, then execute”³. Each new chat “starts from a blank slate plus your prompt”³. A decision your team made but did not write down will likely be made again next week, possibly differently.

Each judgment below has four parts, in the same order: the decision, the layer and property it changes, the evidence, and the 10x verdict. The verdicts are specific to this course. They are judgments, not measurements. Each comes with a reason you can check against the evidence and discuss.

Your support app shows a customer’s order on screen, and the user asks for a refund. You could give the model a tool that takes an order ID. OpenAI’s function-calling guide says not to: “Don’t make the model fill arguments you already know”⁴. Its example is to define submit_refund() with no parameters and pass the order ID in your code, and the same passage adds: “Combine functions that are always called in sequence”⁴.

The decision. Design actions for the task itself, not for the API. Keep work that code can do reliably within your code. For each tool, ask if the agent truly needs it.

The layer and property. This is the contract, the part of the tool the model reads (chapter 2). The property it sets is what the model can do at all. Every argument the model fills is a value it can get wrong. Every tool it holds is a door it can open.

The evidence. Anthropic describes a tool as “a contract between deterministic systems and non-deterministic agents”⁵: the caller might skip the tool, misuse it, or “hallucinate”⁵. This is why we move certain work into code. Anthropic’s advice on agents points the same way: “we recommend finding the simplest solution possible, and only increasing complexity when needed”⁶. The Claude Code team keeps about 20 tools and says “The bar to add a new tool is high, because this gives the model one more option to think about”⁷. The shape of the actions also changes results. SWE-agent’s authors gave a model “a small set of simple actions for viewing, searching through and editing files” and report that interfaces built for language models “outperform existing user interfaces (UIs) designed for human users, such as the Linux shell”⁸. Those are 2024 models (chapter 2 gives the numbers). Sierra states the security side plainly: “The best way to make sure that an agent can’t do something unexpected with sensitive data is to avoid giving it that data in the first place”⁹.

The 10x verdict: still matters. A better model chooses better among the actions you provide. It does not reduce the scope of what those actions can reach. The action set determines the blast radius—how much damage one wrong call can do—and that is a property of your system. One honest limit: specific tools do expire. The Claude Code team writes that “As model capabilities increase, the tools that your models once needed might now be constraining them”⁷. The tool list changes over time; you still have to decide what goes on it.

Connect five common MCP servers, for GitHub, Slack, Sentry, Grafana and Splunk, and Anthropic’s documentation says their definitions “can consume ~55k tokens” before Claude does any work¹⁰. The model must read this menu before each choice it makes.

The decision. Show the model the smallest set of tools that can do the job. First, subtract tools you don’t need. Then, merge tools that always go together, and group tools by domain. Only after these steps should you load tools on demand or let the model call them from code. Chapter 5 covers all four of these moves.

The layer and property. This is the contract again, but now it’s about what fills the context. The property is what the model sees at one time.

The evidence. Longer menus make selection harder. RAG-MCP reports that retrieving only the relevant tool descriptions “more than triples tool selection accuracy (43.13% vs 13.62% baseline)” on its benchmark tasks¹¹. ToolMenuBench, across seven model backends, reports task success rising from 32.1% when every tool is visible to 85.7% when using its filter, with about 98% fewer tokens¹². Anthropic states “Claude’s ability to pick the right tool degrades once you exceed 30–50 available tools”¹⁰. OpenAI’s guide says “Aim for fewer than 20 functions available at the start of a turn at any one time, though this is just a soft suggestion”⁴, and it suggests namespaces of “fewer than 10 functions”¹³. None of these vendors publishes the evaluation data behind its numbers, and each offers guidance specific to its own models^4,10. A Studio tool server with three to eight tools is below the thresholds these vendors provide, and Anthropic itself says standard tool calling “is a better fit when you have fewer than 10 tools”¹⁰.

The 10x verdict: half and half. One half may shrink. Anthropic’s internal evaluation of tool search shows how much the baseline moved between models: without tool search, Opus 4 scored 49% and Opus 4.5 scored 79.5%¹⁴. A stronger model may pick well from a longer list. The other half stays, because it describes your system, not the model. Function definitions “count against the model’s context limit and are billed as input tokens”⁴. In Anthropic’s prompt cache, changing tool definitions “invalidates the entire cache”¹⁵, which is why OpenAI’s tool search adds newly found tools “at the end of the context window”¹³. And every visible tool can be misused: ToolMenuBench counts “risky-tool exposure” as a cost of its own¹². A smarter model pays the same bill, and the tool it should never have seen is still within reach.

In July 2025, Replit’s CEO wrote that its “agent in development deleted data from the production database” and called it “Unacceptable and should never be possible”¹⁸. A user affected by this wrote a separate complaint. They said that during a code freeze, the agent kept changing code “Despite repeated explicit instructions—including eleven separate warnings in ALL CAPS”¹⁹. The same account concludes: “Enforcing a true code freeze was simply impossible within Replit’s architecture”¹⁹. Keep these two points separate: the user’s account states the warnings were about the freeze, not the database¹⁹. Both failures teach the same lesson.

The decision. Let reads pass. Send every write through a gate in your code. Give every write tool an answer to one question: what if it runs twice?

The layer and property. This is the harness (chapter 2). The property is who decides whether a requested call runs.

The evidence. Claude Code’s documentation draws the line: “Permission rules are enforced by Claude Code, not by the model. Instructions in your prompt or CLAUDE.md shape what Claude tries to do, but they don’t change what Claude Code allows”²⁰. Its rules are checked in a fixed order, “deny, then ask, then allow”²⁰, and a hook that exits with code 2 blocks a call even if its own output says allow²¹. Replit’s fixes were structural in the same way: “automatic DB dev/prod separation to prevent this categorically” and a “one-click restore” from backups¹⁸. OpenAI keeps the same line when the model writes code: for “Writes or approval-sensitive actions”, its guide says “Use direct tool calling by default to preserve a clear authorization boundary”²².

Then there is the second half of the decision. In 1984, Birrell and Nelson wrote that a failed remote call “will have been invoked either once or not at all—the user is not told which”¹⁶. As of October 2026, the tool-calling guides from OpenAI, Anthropic and Google, searched in full as fetched on October 3, say nothing about what to do when a call may already have run or may run twice^4,23,24,25. That part is left to you. An old answer exists. RFC 1057, the 1988 Sun RPC specification, says a server “may choose to remember this ID after executing a call and not execute calls with the same ID”¹⁷.

The 10x verdict: fully matters. A smarter caller still cannot undo a write. It still cannot see whether a lost call ran, because the reply never arrived. It can also be wrong about its own actions. After the Replit deletion, the agent “initially denied that rollback functionality existed”¹⁹. The gate has to sit where the model’s judgment cannot move it.

In 2025, Invariant Labs put instructions in an issue on a public GitHub repository²⁷. A user’s agent, connected through GitHub’s MCP server to an account that could also read private repositories, read the issue while handling a normal request²⁷. It could then be “coerced into pulling private repository data into context, and leaking it in an autonomously-created PR in the public repository”²⁷. The researchers emphasize that “this is not a flaw in the GitHub MCP server code itself”²⁷.

The decision. Treat anything a tool returns as data, never as an instruction. Do not let one session hold private data, untrusted content, and a way to send data out at the same time.

The layer and property. This is about the harness and scope of a session. The property is whose words count. Chapter 2’s fourth question asked who wrote the contract and the result. This judgment is what you do with that answer.

The evidence. OpenAI’s 2023 launch post for function calling already warned that “untrusted data from a tool’s output can instruct the model to perform unintended actions”²⁸. A 2023 paper identifies the cause: “LLM-Integrated Applications blur the line between data and instructions”²⁹. Simon Willison explains why the model cannot resolve this alone: “LLMs are unable to reliably distinguish the importance of instructions based on where they came from”²⁶. His lethal trifecta consists of private data, exposure to untrusted content, and the ability to send data out. With all three, he writes, “an attacker can easily trick it”²⁶.

Dialogs did not hold. Claude Desktop asks before each call by default, but “many users already opt for an ‘Always Allow’ confirmation policy”²⁷, and in Cursor the arguments sat “hidden behind an overly simplified UI representation”³⁰. Structure did hold. Invariant’s fix is a policy that allows one repository per session²⁷. Sierra hands its agent “a separate GitHub token that only gives read-only access to specific repositories”⁹. The CaMeL defense separates the plan from untrusted data and reports 77% of AgentDojo tasks solved “with provable security”, against 84% for an undefended system³¹. That is a real cost in capability, paid on purpose.

Vendors now incorporate this rule into products. Claude Managed Agents’ automatic evaluator “does not read intent from a tool result, a fetched webpage, an MCP server’s response, or a message between session threads”³², but end-user text you relay in user messages counts as your intent³².

The 10x verdict: fully matters. A better model still reads one stream of tokens: “Everything eventually gets glued together into a sequence of tokens and fed to the model”²⁶. An attacker only needs one success. This is why Willison, discussing guardrail products that claim to stop 95% of attacks, says that in web application security that number is “very much a failing grade”²⁶. In the cases in this chapter, prompts and dialogs failed, but structural limits held.

Chapter 4 discussed this judgment when we looked at what a packaging adds beyond a local function. Here is a short version, placed next to the other four judgments.

The decision. Begin with the simplest packaging that works for the agents using the tool. Add an MCP server when several hosts need the same tool, or when a server must hold a credential that the agent should never access.

The layer and property. This refers to packaging, and the property describes how the tool reaches the harness and where its credentials are stored.

The evidence. CMU’s tool-use lecture states “MCP is an alternative agent-facing interface—not a requirement for calling REST APIs”³³. When MCP truly adds value, it is through credential brokering: “the server accepts one credential and uses a different credential upstream. The model sees neither secret”³³. A commenter in a Hacker News discussion about CLIs versus MCP made this same point: “Authorization, in my opinion, is the big problem you need MCP for”³⁴. If you do build a server, FastMCP’s documentation warns that models do “significantly better” with curated servers “than with auto-converted OpenAPI servers”³⁵, though it offers no benchmark. Yan Wang’s default approach is a sequence: “Start with a CLI and an instruction file; add MCP when distribution requires it”¹.

The 10x verdict: mostly packaging, but the switching cost is real. A better model does not care much which format carried the definition. You still pay to change between packagings. The 2026-07-28 MCP revision removed protocol sessions and the opening handshake³⁶, and one partner quoted in the release said “The biggest changes are breaking ones”³⁷. The same revision notes that a client whose response stream breaks “MUST re-issue it as a new request with a new request ID”³⁶. Packaging details can reach back into judgment 3: if the lost request was a write that already ran, the re-issued one arrives with a new request ID, so only your own tool-level ID can catch the repeat.

Cloudflare describes an LLM as “the most brilliant and enthusiastic junior hire you’ve ever had”, and today’s decentralized approach to MCP as “giving that junior hire a master key to every office and server room on their first day”³⁹. One developer can make the five judgments in their own code. A company with hundreds of agents needs them made once, in one place.

The decision. Put the judgments in one shared gateway, review every server before anyone can use it, and let identity follow the work.

The evidence. A listing is not a review. The MCP Registry “hosts metadata that points to those packages”, and its metadata is “deliberately unopinionated”³⁸; maintainers “can manually remove spam or malicious servers” after the fact³⁸. Sierra built one gateway; by its July 2026 post, 89% of its employees used it to connect agents to 45 services⁹. It keeps an audit log, blocks access to sensitive data about more than one customer in a session, and requires approval out of band when someone has a legitimate reason to cross that line⁹. Its rule for identity: “Interactive work runs as the user. Scheduled or shared workflows run as service accounts with only the permissions they need”⁹. Cloudflare’s launch post describes portals where “Administrators can now review and approve MCP servers before making them available to users”, and where users sign in with the corporate identity provider³⁹. Cloudflare’s post also concedes a gap: the portal does not yet fully prevent direct access, around the portal, to a server that has no authorization of its own³⁹. Chapter 7 covers these patterns in more detail.

The 10x verdict: fully matters. Who may use which tool, with whose identity, and who can audit it afterward are facts about your organization. A better model changes none of them.

Look at the verdict column. The two rows that fully matter, who decides a step and whose words can steer it, are the ones where the cases in this chapter were decided by structure: a gate in code, a session bound to one repository. The row that is mostly packaging is the one most likely to change, as the 2026 MCP revision demonstrated. When you have limited time, focus on rows 3 and 4.

This lecture focuses on a single agent and its tools. We will cover splitting a task across multiple agents in the next two lectures. CMU’s planning lecture shows an agent calling launch_subagent(goal) to run a copy of itself on a subtask, and notes that “Delegation is an action, so it can be trained”⁴⁰. One type of orchestration begins with a plan that a person reviews: “Cursor, Claude Code, and Codex all introduced a Plan Mode: write a persistent plan, have a human verify it, carry it out (possibly with subagents, in parallel)”⁴⁰. Lectures 6 and 7 will address sub-agents and their orchestration. Chapter 6 of this book shows how a plan can also function as a security boundary. We will also discuss memory later. CMU distinguishes memory, which is saved from an agent’s own interactions, from a skill, which is reusable knowledge about how to act⁴¹. Chapter 2 already showed that a memory write is a write, so the five judgments apply to it.

1. For your Studio tool server, write the five decisions as one line each, then add the 10x verdict for each, with your own reason.
2. Pick the riskiest of the five for your server. Name the structural limit you would put in code for it, and say which incident in this chapter it answers.
3. A teammate says the models are getting better, so the write gate can go. Explain in three sentences why judgments 3 and 4 do not depend on the model.