Lecture 5 · COMS W4995 Agentic Engineering

Agent tools in pictures

Opening

A model writes the order, a program cooks it

By the end of these pages you can say why a model never runs a tool itself, name the four questions every tool raises, and tell what is forty years old about tool calling and what is new.

After this chapter you can

  1. Explain in plain words why a model never runs a tool itself, and what decides whether a requested call happens.
  2. Name the four questions every tool raises and who owns each one.
  3. Say what is old and what is new about tool calling.

7 minutes to read4 sections4 quiz questions

Start reading →

Ask an assistant for the weather in Pittsburgh. Look at what the model produces. It does not contact a weather service. Instead, it writes a short piece of structured text: a tool name, get_weather, and an argument, a city called Pittsburgh2. Something else, not the model, decides whether any weather service is actually called.

Section 1

A model never runs a tool

A tool call in four steps: the model writes the call, the harness decides whether to allow it, the tool runs, and the result returns to the model's context. Only the first step belongs to the model. The model's part Software around the model Model writes the call: name + inputs Harness checks and decides: deny, ask or allow Tool runs the code Result goes back into the context the model reads the result and writes its next step Stanford's version: select and arguments (the model), then validate and execute (software).
The model’s part ends when it writes the call. A program decides, runs the tool and hands the result back.

A language model generates text. When it “uses” a tool, it generates a request: the tool’s name and the inputs to pass to it. CMU’s agents course summarizes this relationship: “The model proposes; external software decides whether and how to execute”2.

Google’s guide for Gemini function calling agrees: “The model doesn’t execute the function itself”3. Anthropic’s guide states that Claude “returns a structured call that your application executes (client tools) or that Anthropic executes (server tools)”4. In both cases, a program runs the tool.

Stanford’s course outlines a tool call in four steps: select a tool from the schemas in the context, fill in the typed arguments from the task, validate those arguments, and then execute the tool. The result then returns to the context5. The model performs the first two steps. Software handles the last two. We call this software the harness: it’s the program surrounding the model that reads its requests, determines what actually happens, and feeds the results back.

Why do we emphasize this distinction? Because safety and reliability reside on the harness side. Writing never delete the database into the prompt adds more text; it does not add a security lock. Claude Code’s documentation states this explicitly: “Permission rules are enforced by Claude Code, not by the model”1.

Check yourselfThe model writes the call delete_database("prod"). What prevents it from running that call?

Only the program designed to run it. Until the harness executes the request, it remains just text. Rules enforced by the harness are applied “by Claude Code, not by the model”1; a sentence in the prompt is not such a rule.

Section 2

Four questions, four owners

Four questions about any tool, each with its owner and the chapter that answers it: what the model sees (the contract, chapters 2 and 3); who says yes and runs it (the harness, chapter 6); how the tool is plugged in (the packaging, chapter 4); who controls it across a company (gateway and admin, chapter 7). Question Owner Chapter 1 What does the model see? the contract 2 and 3 2 Who says yes, and who runs it? the harness, in code 6 3 How is the tool plugged in? the packaging 4 4 Who controls it across a company? gateway and admin 7
Four questions decide whether a tool call goes well. Each has a different owner, and this book answers each in its own chapter.

Once the model only writes requests, a tool is no longer a single thing. It becomes four questions. Each question is answered by a different part of the system. We sort them this way; it is not an industry standard.

1. What does the model see? This is the contract. For a Claude tool, the contract includes a name, a description “of what the tool does, when it should be used, and how it behaves”, and an input_schema, “A JSON Schema object defining the expected parameters for the tool”6. Anthropic describes tools as “a contract between deterministic systems and non-deterministic agents”7. The model only knows a tool through this contract. Chapters 2 and 3 will explain this in detail.

2. Who says yes, and who runs it? This is the harness, which is code. Claude Code checks its rules in a fixed order: “deny, then ask, then allow”1. And “Instructions in your prompt or CLAUDE.md shape what Claude tries to do, but they don’t change what Claude Code allows”1. Chapter 6 focuses on this question.

3. How is the tool plugged in? The packaging. The same capability can arrive as an MCP server, where a host application “establishes connections to one or more MCP servers”8; as a skill, which is “a directory containing, at minimum, a SKILL.md file”9; or as a command the model runs through one general tool. The Claude Code team frames the choice this way: “Do you give it one general-purpose tool like bash or code execution? Or fifty specialized tools, one for each use case?”10 Chapter 4 compares five ways to hand a tool over.

4. Who controls it across a company? This is the gateway and the admin console. In ChatGPT’s workspace controls, “Role access controls who can use an app. Actions controls what the app can do. Permissions controls when ChatGPT asks before using an app”11. Sierra routes its agents through one company gateway. By its own account, 89% of its employees use this gateway to connect agents to 45 services12. Chapter 7 covers this layer.

Each owner can fail independently. A clear contract cannot protect you if a harness is set to approve everything. A careful harness cannot vouch for a server that nobody reviewed.

Check yourselfYour company bans one MCP server for every employee. Which of the four questions does this relate to?

Question 4: the company decides which tools are allowed at all. The harness, which is question 2, then enforces that ban on each call. In Claude Code, managed settings are at the top: “no other level, including command line arguments, can override a managed permission rule”1.

Section 3

The problem is forty years old; the caller is new

A timeline from 1984 to 2025: remote procedure calls in 1984, CORBA in 1991, SOAP and REST in 2000, the Language Server Protocol in 2016, OpenAPI 3.0 in 2017, function calling in 2023 when the caller became a model, MCP in 2024, and adoption by OpenAI, Google and Microsoft in 2025. The caller is a program The caller is a model 1984RPC 1991CORBA 2000SOAPREST 2016LSP 2017OpenAPI 3.0 2023functioncalling 2024MCP 2025OpenAI,Google andMicrosoftadopt it Spacing is not to scale.
Programs have called code on other machines since 1984. In 2023 calling by a model became an API; in 2024 MCP arrived, and within six months the large vendors had adopted it.

Calling code that lives somewhere else is not new. In 1984, Birrell and Nelson published their remote procedure call package and stated its guarantee: if a call returns, the procedure ran exactly once; otherwise “the procedure will have been invoked either once or not at all—the user is not told which”13. CORBA 1.0 followed in October 1991 with an Interface Definition Language and an Interface Repository14. In 2000, SOAP defined “a convention for representing remote procedure calls and responses”15, and Fielding’s dissertation defined REST16. In 2016, the Language Server Protocol let a language server “be implemented once and have different tools take advantage of it”17. In 2017, OpenAPI 3.0 described HTTP APIs so that “both humans and computers” can “discover and understand the capabilities of a service without requiring access to source code”18.

In all these cases, a program was the caller. Research systems began handing the call to a model in 202219, and in 2023 it became an API. On June 13, 2023, OpenAI let developers describe functions so the model could “intelligently choose to output a JSON object containing arguments to call those functions”20. Now, the caller reads a description in plain language and guesses what to send. It might pick the wrong tool, invent an input, or obey something it read. OpenAI’s launch post already warned that “untrusted data from a tool’s output can instruct the model to perform unintended actions”20.

On November 25, 2024, Anthropic open-sourced the Model Context Protocol (MCP), to replace a world where “Every new data source requires its own custom implementation”21. Within six months the other large vendors had adopted it. OpenAI’s Agents SDK added MCP support on March 26, 202522. Google launched its Agent Development Kit on April 9, 2025 with MCP tools in its tool list23. Microsoft announced “broad first-party support” for MCP at Build on May 19, 202524. OpenAI’s Responses API added remote MCP servers the next day25. Google’s agent-to-agent protocol, A2A, describes itself as one that “complements Anthropic’s Model Context Protocol”26.

So the plumbing is old, and so are its unsolved questions, such as whether a call that never answered actually ran. The caller is new: it reads, it guesses, and it can be misled.

Check yourselfAn agent’s tool call times out. Did the tool run?

Maybe. The 1984 answer still holds: a failed call ran “either once or not at all—the user is not told which”13.

Before you go on

Close the book

  1. Explain to a friend why the model cannot delete a file by itself. Name the part of the system that can.
  2. Pick one tool you have used with an agent, such as web search or a calendar. For each of the four questions, say who answers it in that product.
  3. Say in two sentences what is forty years old about calling a tool, and what changed when the caller became a model.

Opening · quiz

Test yourself

Most questions have more than one right answer, so select every option that is true. Check each question, then open “Answer and why” for the reasoning.

Question 1An agent answers a weather question by using a get_weather tool. Which of these does the model itself do?Select all that apply.
Answer and why
  • RightWrites the tool’s name and the inputs to pass. A call is a structured request the model writes; external software decides whether to execute it2.
  • WrongSends the network request to the weather service. The application or the vendor executes the call; the model doesn’t execute the function itself3,4.
  • WrongDecides, through its rules, whether the call is allowed. Permission rules are enforced by the harness, not by the model1.
  • RightReads the result once it is placed back in its context. After execution, the result returns to context, where the model reads it5.
  • RightChooses which tool to call from the descriptions it can see. Selecting a tool from the schemas in context is the first step of a call5.
Question 2A team writes ‘Never delete production data’ in the system prompt and considers the agent safe. What is wrong with that reasoning?Select all that apply.
Answer and why
  • RightThe prompt shapes what the model tries to do, not what the harness allows. Instructions in the prompt shape what Claude tries to do, but they don’t change what Claude Code allows1.
  • WrongThe model will ignore the instruction most of the time. The flaw is not how often the model obeys; even an instruction it always follows is not a lock, because enforcement lives in the program that runs the call2.
  • WrongModels cannot read system prompts. They can; the problem is that reading an instruction is not the same as being prevented.
  • WrongNothing is wrong; a clear instruction is as strong as a permission rule. Permission rules are enforced by the harness, not by the model1.
Question 3Match each situation to the right question. Which pairings are correct?Select all that apply.
Answer and why
  • RightA tool’s description is vague, so the model keeps picking it for the wrong task: question 1, what the model sees. The description is part of the contract the model reads6.
  • RightClaude Code blocks a command because a deny rule matches: question 2, who says yes. The harness evaluates deny, then ask, then allow1.
  • RightA team ships the same capability as a skill folder instead of an MCP server: question 3, how it is plugged in. A skill is a directory with a SKILL.md file; an MCP server is a program a host connects to; both are packaging8,9.
  • WrongAn admin limits which employees may use an app: question 1, what the model sees. That is company-wide control, question 4: role access controls who can use an app11.
  • RightAn admin limits which employees may use an app: question 4, who controls it across a company. Role access controls who can use an app11.
Question 4Which of these are new with model callers, rather than problems remote calls already had in 1984?Select all that apply.
Answer and why
  • WrongNot knowing whether a call that failed actually ran. Birrell and Nelson already wrote that a failed call ran once or not at all, and the user is not told which13.
  • RightThe caller chooses a tool by reading a plain-language description. Function calling lets the model choose to output a JSON object with arguments after reading described functions20.
  • RightText returned by a tool can instruct the caller to do something unintended. OpenAI warned on launch day that untrusted data from a tool’s output can instruct the model20.
  • WrongCalls travel over a network and can be slow or fail. Timing and independent failure were already named as the fundamental difficulties of distributed systems in 198413.
  • WrongDescribing a remote interface so a stranger can call it. CORBA’s IDL (1991) and OpenAPI both did this for programs14,18.

Sources

  1. Configure permissions (Claude Code). Anthropic. code.claude.com/docs/en/permissions ↗
  2. Tool Use for Language Model Agents (Lecture 2 slides). CMU LTI / Graham Neubig. www.cmu-agents.com/slides/lecture-02-tool-use.pdf ↗
  3. Function calling with the Gemini API - Interactions API | Google AI for Developers. Google. ai.google.dev/gemini-api/docs/function-calling ↗
  4. Tool use with Claude - Claude Platform Docs. Anthropic. platform.claude.com/docs/en/agents-and-tools/tool-use/overview ↗
  5. Stanford CS329Z Lecture 1: Intro to Agentic Systems. Diyi Yang, Michael Ryan, John Yang. web.stanford.edu/class/cs329z/slides/lecture01.pdf ↗
  6. Define tools. Anthropic. platform.claude.com/docs/en/agents-and-tools/tool-use/implement-tool-use ↗
  7. Writing effective tools for AI agents—using AI agents. Anthropic. www.anthropic.com/engineering/writing-tools-for-agents ↗
  8. Architecture overview, MCP docs (round-1 source, B7/B14 URL; page version 2026-07-28). modelcontextprotocol.io (assigned reading; now serves the 2026-07-28 docs). modelcontextprotocol.io/docs/learn/architecture ↗
  9. Agent Skills specification. agentskills.io. agentskills.io/specification ↗
  10. Seeing like an agent. Anthropic (Thariq Shihipar). claude.com/blog/seeing-like-an-agent ↗
  11. Admin controls, security, and compliance for plugins and apps. OpenAI Help Center. help.openai.com/en/articles/11509118-admin-controls-security-and-compliance-for-plugins-and-apps ↗
  12. Building Sierra's MCP Gateway: An engineering iceberg. Sierra (Mihai Parparita). sierra.ai/blog/building-sierras-mcp-gateway-an-engineering-iceberg ↗
  13. Implementing Remote Procedure Calls (Birrell & Nelson), ACM TOCS 2(1). ACM Transactions on Computer Systems (copy hosted at cmu.edu). www.cs.cmu.edu/~dga/15-712/F07/papers/birrell842.pdf ↗
  14. History of CORBA. Object Management Group. www.omg.org/gettingstarted/history_of_corba.htm ↗
  15. Simple Object Access Protocol (SOAP) 1.1, W3C Note. W3C (member submission: DevelopMentor, IBM, Lotus, Microsoft, UserLand). www.w3.org/TR/2000/NOTE-SOAP-20000508 ↗
  16. Fielding Dissertation, Chapter 5: Representational State Transfer (REST). Roy T. Fielding, UC Irvine. ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm ↗
  17. A Common Protocol for Languages. Microsoft, Visual Studio Code team blog. code.visualstudio.com/blogs/2016/06/27/common-language-protocol ↗
  18. OpenAPI Specification v3.0.0. OpenAPI Initiative / Linux Foundation. spec.openapis.org/oas/v3.0.0.html ↗
  19. Yao et al. "ReAct: Synergizing Reasoning and Acting in Language Models." ICLR (2023).. arXiv (Yao et al.). arxiv.org/abs/2210.03629 ↗
  20. Function calling and other API updates. OpenAI. openai.com/index/function-calling-and-other-api-updates ↗
  21. Introducing the Model Context Protocol. Anthropic. www.anthropic.com/news/model-context-protocol ↗
  22. Release v0.0.7 · openai/openai-agents-python. OpenAI (GitHub release). github.com/openai/openai-agents-python/releases/tag/v0.0.7 ↗
  23. Agent Development Kit: Making it easy to build multi-agent applications. Google for Developers Blog. developers.googleblog.com/en/agent-development-kit-easy-to-build-multi-agent-applications ↗
  24. Microsoft Build 2025: The age of AI agents and building the open agentic web. Microsoft (Official Microsoft Blog). blogs.microsoft.com/blog/2025/05/19/microsoft-build-2025-the-age-of-ai-agents-and-building-the-open-agentic-web ↗
  25. Changelog | OpenAI API (round-1 source, A28 URL). OpenAI. developers.openai.com/api/docs/changelog ↗
  26. Announcing the Agent2Agent Protocol (A2A). Google for Developers Blog. developers.googleblog.com/en/a2a-a-new-era-of-agent-interoperability ↗